Testing self-driving cars & Machine Learning Models

by Kate Libbie | June 15, 2020 1:37 pm

Tesla’s Autopilot, Waymo, Uber… The autonomous revolution is already here, but its proper functioning is a big concern. The main goal behind self-driving cars testing is simple: to ensure cars can handle riding on various conditions and provide safety for people inside as well as outside. But can we trust an automated chauffeur? To check this, QATestLab prepared a little investigation on how do self-driving cars work and how we can test them.

How Machine Learning Makes Self-Driving Cars the Reality

To understand the process of testing self-driving cars, one should first get a better idea of how self-driving cars work. All-in-all, the thing that turned a science-fiction dream into reality is Machine Learning – the hot term nowadays and the most popular subset of Artificial Intelligence. It uses statistics to find patterns in massive amounts of data and improve its mechanism automatically through experience.

Autonomous vehicles are associated with artificial intelligence. But when we say artificial intelligence, we mean a broad field that combines multiple machine learning algorithms under its umbrella and especially in an even narrower sense, a variety of deep learning technologies. So, the central algorithm of machine learning, on which self-driving cars are built on is Deep Learning and, more precisely, Deep Neural Networks.

What is Deep Learning and Deep Neural Network? 

Right now, Deep Learning is the most popular Machine Learning algorithm. This algorithm helps to create an AI-powered layered neural network, much like a simplified replica of the human brain. Diving deep into myriads of networks, deep learning techniques allow to extract much more insights from data and solve more complex tasks than any other Machine Learning technique.

The power of deep learning for Self Driving Cars

Thanks to deep learning, we can reproduce all the functions done by human drivers. For this, such technologies like object and image recognition and collision avoidance comes to play. But to make a car drive autonomously, it needs to learn on its own and can quickly resolve complex situations. How do the process of machine learning happen?

Each situation must be analyzed separately. For example, let’s consider the process of parking a car. To teach an autonomous vehicle to park without problems, you need to train the ML model and repeat a specific algorithm of actions many times. Each pass is based on a system of reinforcements for proper functioning. If the algorithm is executed correctly – reinforcement learning rewarded with a high score, the error values decrease significantly. The higher accuracy, in the end, the more possibilities of ML model o work properly.

Driverless Vehicle Testing: Points Needed to Be Checked

Approaching the new reality caused by driverless vehicles, we are facing more questions about the safety of this cutting-edge technology than ever before. That’s why autonomous vehicles are tested on public roads on a large scale. What concerns on testing needed to be taken into account? Training machine learning models that are incorporated in the heart of driverless cars is held in accordance with the following aspects:

• Vehicle safety: The first criterion for the safety of self-driving cars is safety inside the vehicle itself. Programs that equip the vehicles must be focused on ensuring that the passenger is brought to the destination safely.
• Safety of others: The biggest concern of autonomous cars is making sure that other drivers on the road, pedestrians, and just buildings down the way will not be damaged.
• Emergency situations: Another risk that can be tracked with autonomous cars is a misunderstanding of how a car will react to unexpected situations. For example, construction zones that require slower speeds and maneuvering around people, recognition, and yielding to emergency vehicles, etc.
• Cybersecurity: Autonomous cars are amassing vast amounts of fo data, but are these snippets of information, especially one of a private matter, protected enough? What if the databases can be hacked and private data can be revealed. All these need to be checked.

So, all-in-all, the central goal of testing self-driving cars is to ensure that vehicles will function the same way as a human driver would even in response to unexpected events on the road. 

Testing ML Models and Deep Neural Networks

The principle of operating behind self-driving cars is not so simple. For instance, autonomous vehicles take inputs from a variety of sensors, and actuate car systems such as steering, braking, and accelerating. Each of these sensors forms its own neural networks that needed to be checked in accordance with inputs/output scenarios. So, there is a great number of sensors, but options for combinations of data amassed from these sensors are much higher. It is clear that human-QA engineer is unlikely to cope with so many test cases. That’s why there are special programs that train ML models and thus test the DNNs.

If you delve into the technical side of autonomous vehicles testing, you can find many options for this purpose. For example, one of the programs developed to ‘think like a tester’ – is DeepXplore[1] that trains MLmodels and test DNNs, which are used everywhere and in particular in self-driving cars. This program uses model ensembles to find differences in outputs that suggest bugs. In addition, there is another program based on the previous one – DeepTest – it focuses exclusively on autonomous vehicles and is a system that checks several sources of information, thanks to which autonomous vehicles work.

You can read more about testing DNNs with DeepTest here[2].

Bottom line: Thinking about safety

Soon self-driving vehicles will transform urban environments everywhere, and it’s fascinating how far we have come in such a short period of time. The only question remains to be open: can autonomous vehicles be safe enough? Today, thanks to such programs like DeepExplore and DeepTest it becomes possible to check all possible scenarios of autonomous vehicles. Still, we can not be secured to 100%. 

On the other hand, we should perceive autonomous vehicles from different angles. Just like driverless cars can pose a risk to us, they also can bring us safety. Since robots programmed to obey certain rules, we can track things that are impossible for a person to track. For example, you can program the robot so that it does not increase speed and is not distracted by text messages, and so on. Another advantage over live drivers,  thanks to a computer vision autonomous vehicles are able to recognize objects even at night or in low-light areas.

So, autonomous vehicles have both things that we can worry about, and vice versa, those that provide more security than before. However, the essential thing in that is ensuring the expected response of the drone, which can be done through proper testing. 

……………………………

If your products are requiring testing, we are ready to hedge your team and ensure the high quality. We are participating in the life of the product, and do our work with a passion for your success. Let’s make great products together.

Visit our blog[3] to read more about QA & testing. Feel free to contact us[4] for collaboration.

Related Posts:

• Testing the Brains of Tomorrow’s Tech – AI and ML[5]
• QA for AI: Why to Test Already Smart Artificial Intelligence?[6]
• New job title: The Ethical Hacker on Artificial Intelligence[7]

Source URL: https://blog.qatestlab.com/2020/06/15/testing-self-driving-cars/

---