Computer scientists are developing a system by which humans could teach an autonomous car to identify “blind spots” in its driving ability.
Researchers from MIT and Microsoft are working out a model for autonomous systems to identify their own “unacceptable” actions by comparing them to “acceptable” human actions. Through machine-learning techniques, this model then enables systems to identify gaps in their responses to real-world situations, a potentially useful tool for improving the safety of artificial intelligence technology like self-driving vehicles.
While existing autonomous systems can receive input from humans, they don’t use that input to identify behavioral blind spots, according to MIT News.
“The model helps autonomous systems better know what they don’t know,” said Ramya Ramakrishnan, a graduate student at MIT and author on a recent paper discussing the model. “The idea is to use humans to bridge that gap between simulation and the real world, in a safe way, so we can reduce some of those errors.”
The brief history of self-driving cars has been spotted with tragedy.
In March, a self-driving car owned by Uber struck and killed a pedestrian in Arizona after the company’s engineers disabled the car’s emergency breaks to “reduce the potential for erratic vehicle behavior.”
The first known death involving a self-driving car happened in 2016. The car’s driver was watching a Harry Potter movie when his car attempted to drive underneath a semi-trailer, unable to distinguish its bright-white side from the skyline.
Distinctions between real-world scenarios that are obvious to human operators can be challenging for autonomous systems. For instance, a self-driving car might have trouble telling the difference between a white van and an ambulance — and not know to pull over to the side of the road as the ambulance passes by.
Ramakrishnan and her fellow researchers reportedly tested their autonomous system successfully using a video game. The next step will be to try the system within the context of regular training simulations for autonomous cars and robots.
Human operators provide information to the model through a few different means, including demonstration.
The artificial intelligence might signal it plans to act in one way, prompting the human operator to take control and plot a different course. The system then takes note of the difference between its own plan and the human operator’s action.
The trouble is prompting the system to distinguish between situations and objects that might differ only slightly — such as an ambulance and a white van.
“At that point, the system has been given multiple contradictory signals from a human: some with a large car beside it, and it was doing fine, and one where there was an ambulance in the same exact location, but that wasn’t fine. The system makes a little note that it did something wrong, but it doesn’t know why,” Ramakrishnan said. “Because the agent is getting all these contradictory signals, the next step is compiling the information to ask, ‘How likely am I to make a mistake in this situation where I received these mixed signals?’”
The model then uses probability calculations to compute whether a situation might actually constitute a blind spot in its understanding of “acceptable” or “unacceptable” responses.
“When the system is deployed into the real world,” Ramakrishnan said, “it can use this learned model to act more cautiously and intelligently. If the learned model predicts a state to be a blind spot with high probability, the system can query a human for the acceptable action, allowing for safer execution.”
Safety concerns about self-driving cars have prompted some public skepticism over the technology — and even hostility.
According to the Arizona Republic and New York Times, Arizona police have documented more than a dozen instances of people attacking or threatening Google-affiliated self-driving vehicles over the last two years.
Image from Pexels.com