While all types of vehicles are rigorously tested, the testing process for self-driving technology goes beyond the requirements for traditional vehicles. Autonomous vehicle testing requirements are more than just mechanical performance, because it’s also necessary to validate the software decisions that are happening.

It’s possible for a car to “drive itself” safely when the conditions are ideal, but a catastrophic failure can occur from something as small as a sensor being confused by rain or a pedestrian who moves unpredictably in the road. AV testing simulation goes beyond engineering and also looks at other factors that will determine whether the vehicle is road-ready.

Not only do the manufacturers and vehicle owners have an interest in the success of these vehicles, but regulators, insurance companies, and the general public also have a stake in the safety and performance of self-driving cars.

From Virtual Roads to Real Ones: How Autonomous Vehicles Are Tested Phase by Phase

There are several stages of testing that must occur before an autonomous vehicle is determined to be roadworthy.

Phase 1: AV Testing Simulation

Simulation is the starting point, which means that the software goes through millions of virtual scenarios before anything is tested physically. Many of the scenarios that are tested are too dangerous to recreate in real life, such as unexpected obstacles, wrong-way drivers, or a sudden sensor failure during the middle of a trip.

For example, Waymo has reported over 20 billion simulated miles. All of this simulated testing allows the teams to find patterns in the failures, which would take decades of real-world driving to identify the same issues.

But there are limitations to using simulation for autonomous vehicle safety testing. For example, it’s not possible to create a full replication of the unpredictable conditions that happen in real-world environments. Which is why simulations are the starting point, and then the AV testing must move on to additional phases.

Phase 2: Closed-Course Testing

The next stage of autonomous testing methods involves physical testing in a closed-course environment. Places like MCity and GoMentum Station are designed to recreate a variety of circumstances, including intersections, school zones, construction areas, and emergency scenarios.

Every AV must pass a list of standardized scenarios before it is approved for real-world use. Examples of the necessary tests include a person crossing the street suddenly, emergency braking, or merging onto a busy road at speed.

During the physical testing phase, it’s possible to catch things that might have been missed in the simulation stage. Some of the smallest details can affect the vehicle’s performance, such as how the sensors respond to the heat shimmer from the asphalt or the way the brakes perform when the roads are wet.

There are many valuable insights that can be gained when failure occurs during this phase of autonomous vehicle testing. The goal is to identify gaps in the vehicle’s decision-making logic as soon as possible, so that real people aren’t at risk when the vehicle is actually on the road.

Phase 3: Public Road Testing and Shadow Mode Validation

The next stage of testing is known as shadow mode, which means that the AV system is running alongside a human driver in the car. The system is making decisions without acting on those decisions. Then, the engineers can compare the car’s decisions with what the human driver did in each scenario.

Shadow mode validation is the way that some of the major players in the industry (Tesla, Waymo, and GM) identified real-world issues without putting passengers at risk.

State-issued permits are required before public road testing can begin, and the local requirements vary depending on the location. In the state of California, they have some of the strictest rules. Additionally, disengagement data (when a human driver had to take over the controls) is published from every permitted operator.

Autonomous Vehicle Testing Methods for Sensors, Cybersecurity, and Compliance

All functions need to be included in autonomous vehicle testing, including sensors, cybersecurity, compliance, and more. At first, each sensor is tested in isolation, then testing moves to the integrated system.

These testing methods introduce known failure conditions, such as snow, heavy rain, direct glare, faded lane markings, road changes due to construction, and more. In order for a manufacturer to get regulatory approval, the international functional safety standards (ISO 26262) must not only be tested, but every step needs to be documented throughout the testing process.

Additionally, cybersecurity is a big concern since autonomous vehicles are connected to maps, cloud infrastructure, other vehicles, and traffic systems. Every time there is a connection point, it could possibly be an entry point for attackers to access the system. There are global cybersecurity regulations that AV manufacturers must follow (UNECE WP.29), and all threats must be documented, analyzed, and monitored on an ongoing basis.

Autonomous vehicle testing should always include penetration testing, with ethical hackers attempting to hijack the vehicle controls or spoof GPS data. The same methods that real attackers would use are tested on all of the vehicle systems so that potential vulnerabilities can be identified and fixed.

FAQs

What are the main methods used to test self-driving cars?

Autonomous vehicle testing goes through four stages: simulation, closed-course, public road testing, and ongoing shadow mode validation.

Is simulation testing enough to certify an autonomous vehicle?

No. While simulation testing is the first step of this process, it’s also essential to test the physical performance and real-world road testing. Every AV needs to be tested to see how it performs in real, unpredictable conditions.

What happens when an AV disengages during a public road test?

If this happens, then there is a human driver in the vehicle who will take over control. Each event is logged. In certain states, such as California, these disengagements must be reported publicly.

What is the difference between ADAS testing and full autonomy testing?

ADAS testing is for cars that have built-in driver-assist features, such as lane keeping and emergency braking. On the other hand, full autonomy testing is for vehicles that are designed to operate with no human involvement at all.

Customized Technology Solutions for Your Organization

There’s no question that technology is a critical factor for businesses of all sizes. As industry advancements continue to change, you need a team that can help you stay relevant in a digital world. Tech Critic is here to help, offering a full range of solutions to support your business. Contact us to learn more about the services that are available.