The Army’s research lab has developed software that lets robots understand verbal instructions, carry out a task, and report back. The potential rewards are tremendous. A robot that can understand commands and has a degree of machine intelligence would one day be able to go ahead of troops and check for IEDs or ambushes. It could also reduce the number of human soldiers needed on the ground.
Young gives the example of the command “Go behind the farthest truck on the left.” As well as recognizing objects and their locations, the robot has to decipher “behind” and “left,” which depend on where the speaker is standing, facing, and pointing. Its hard-coded knowledge of the environment gives it further conceptual clues as to how to carry out its task.
The robot can also ask questions to deal with ambiguity. If it is told to “go behind the building,” it might come back with: “You mean the building on the right?”
“We have integrated basic forms of all of the pieces needed to enable acting as a teammate,” says Stump. “The robot can make maps, label objects in those maps, interpret and execute simple commands with respect to those objects, and ask for clarification when there is ambiguity in the command.”