How to ensure the safety of Self-Driving Cars: Part 3/5

So now that you’ve read Part 2, you should feel OK about how well autonomous vehicles can sense both the world around it and itself. What does the car now do with this information? Well, it thinks about it and spits out some command, usually in the form of a steering, braking, and acceleration amount. But before it gets to brake, acceleration, and steer outputs, the vehicle must first plan its path or route. This is called “Path Planning,” and requires all the information from the sensing stack to get correct.

Figure 1: Path Planning (Source)

Part 3: What’s the best path plan for a self-driving car?

How can we be sure that the vehicle made the best decision regarding what path to take in any given scenario? First, we need to determine the evaluation criteria. We choose to break this into four categories:

Performance: What is the limit of the vehicle’s capability in any given scenario and how can we be sure to stay within that limit?

Safety: What happens if we exceed the limit of the vehicle? What adjustments must be made to both get back within the limit of the vehicle? How do we deviate the smallest amount from the planned path while avoiding obstacles?

For example, if the vehicle starts to slide, how fast can it regain traction, how close can it stay to the original path, and how well can it avoid obstacles in its new, unintended path?

Comfort: While the vehicle is driving in non-emergency conditions, how can we make sure the passengers are as comfortable as possible?

Efficiency: How does the vehicle use the least amount of electrical power as possible to allow itself to go for longer?

We’ll dive into each of these criteria and determine how the engineers can best architect the appropriate solution, and how we can verify that it is the appropriate solution.

Performance: What is the limit of the vehicle?

Imagine you’re driving your car and someone immediately jumps in front of you. What do you do? In your mind, you’re probably deciding whether you should slam on the brakes or try to swerve around them. You next try to determine if the vehicle can stop in time, or if you’d slide out of control if you tried to swerve. You make your best guess based on your knowledge of the vehicle and scenario to avoid an accident.

This is a key area where an autonomous vehicle can do better than a human. An autonomous vehicle can know exactly what it is capable of at any given instance in time. However, there’s a lot of math involved in determining this, and more math means more software lines of code, which translates to more time and energy for the hardware to compute results. Time is of the essence in an emergency scenario.

You may be asking “how can a car know what it is capable of at any given instance in time?” The answer lies in a field of study called “Vehicle Dynamics,” namely “Multibody Vehicle Dynamics” in this case. This is the study of objects (wheeled vehicles) in motion. We call it “multibody” because a car isn’t a rigid object, it has joints that move and bend, and suspension components that suspend the wheels from the chassis of the vehicle. To determine the capabilities of the vehicle, we need to break the entire car into its separate bodies and solve the physics equations around each of those components.

Vehicles contact the ground through their tires. Those tires deform as the car drives, which allows the car to propel the vehicle.

Figure 2: Tire Deformation while Driving (Source)

There is a limit to how much a tire can deform before it loses traction with the ground. This is a function of the design of the tire itself; the temperature, wear, and air pressure of the tire; the position of the tire on the road; the load on that tire; and the friction between the ground and the tire in various scenarios. We call the diagram that shows the capability of the tire at any given time the “friction circle:”

Figure 3: Friction Circle (Source)

Once the vehicle has an idea of what each of the tires are capable of, it needs to understand the forces on that tire, namely how much weight is on each tire. It can do this by knowledge of the vehicle specifications (vehicle curb weight) and by checking how much the suspension components are compressed. There are certain complexities regarding the forces on each tire, because they can be influenced by what is happening with the chassis and suspension of the vehicle. For example, if the car is going through a turn, the suspension may collapse and put more force on the outside front wheel. But because the suspension is a spring, it will eventually extend again, shifting weight on each tire. Likewise, the aerodynamics of the vehicle can cause the chassis to move in all sorts of directions, again changing the load on each tire. The gas shifting in the tank will also have an effect.

Along with estimating tire loads, the vehicle needs to know the scenario it’s in. In this case, it looks back to the sense section of the AV stack, and determines the road angle, the vehicle speed and acceleration, the amount the vehicle is rotating, and takes a guess at the coefficient of friction based on what it knows of the scenario. For example, if a vehicle knows it’s below freezing outside and can see some reflectivity on the ground, it may be able to determine if the ground is icy. This also helps us understand if the vehicle is in a scenario where the load on each tire is about to change.

The vehicle can combine its knowledge of the tire information, vehicle scenario, and the loads on each tire and do some math to determine what percentage of the tire friction circle is currently used up and how much there’s left. It can also determine what adjustments can be made within the friction circle, such as adding brake, acceleration, or steering to maneuver the vehicle. It can also determine what the capability of that tire will be in the near future given the current vehicle scenario, like the rate of collapse of the suspension.

There’s one more piece of the pie here though, and that’s the transition period from the current scenario the vehicle is in to the desired one. This is more complex because the vehicle needs to predict what it can do during a transition. The vehicle must quickly run through a bunch of scenarios and choose the one that is best for the specific use case. For example, a vehicle might want to try to make an emergency stop, where it will apply braking force to the edge of the tire friction circle. It will simulate a scenario where it continuously applies brake force to the edge of the friction circle, adjusting the force with the expected friction circle changes over time. If the vehicle determines that it can indeed stop in time, it may make that decision and omit all others.

However, if the vehicle determines that an emergency brake scenario will not cut it, it may try to simulate a “swerve” scenario. Perhaps the vehicle can only turn left in this case because of a cliff on the right. It then simulates a few different “left” inputs of the steering wheel within the friction circle of the tire, adjusting with what it thinks will be the new friction circle of the tire over the entirety of the maneuver. Once it is happy with a proposed swerve solution, it commits to this action.

So, that’s a lot of simulations and math to do in an emergency scenario, right? How can we be sure it made the right decision and did so in time? Engineers use vehicle dynamics software simulation tools like CarSim and MSC Adams to build simulated vehicles and run them through various scenarios, to test their capabilities:

Figure 4: CarSim Vehicle Simulation Software (Source)

This is a form of SIL called “Model in the Loop,” or MIL, because the specific software being tested is a model of a vehicle. The engineer can add the software that handles emergency scenarios to this simulation model to test their stack. Once they are happy with the decisions the car is making under various simulated emergency scenarios, they can move the software to the vehicle hardware. They can then do HIL testing by providing the hardware simulated information and benchmark how fast and accurately the vehicle made decisions. They can even test the real vehicle out, if they don’t mind the possibility of destroying their test car.

The engineers will take data from their MIL, SIL, HIL, and in-vehicle tests and determine if the math the car did was accurate, if the decisions the car made were truly the best possible decisions, and if the decisions were made in time to execute the plan.

Applying the field of multibody vehicle dynamics to autonomous vehicles is still relatively new. Many companies are not concerned about this for the time being since they’re building vehicles for very specific use cases. They don’t anticipate emergency maneuvers being necessary, or they can offload that responsibility to the human driver. The ones that are deploying vehicle dynamics models typically simplify the math to a point that the hardware onboard can process the math quick enough. There’s still a ways to go to get a perfectly accurate answer out of the vehicle.

Figure 5: Simplifying a 4-Wheeled Vehicle to a Bicycle Model (Source)

There’s also an ongoing conversation around how much of the above should be mathematically solved for and how much should be learned by the machine over time. This is a complex debate, and not one we’ll get into. It’s worth noting that some deep learning models in the future may be able to calculate the vehicle’s capabilities just as good as doing all the math in a much quicker and efficient way.

So, should this concern the public? The answer is yes. Though we should be confident that smart engineers will find a way to pick the right amount of simplification on the right hardware for all the math involved so that the vehicle will be able to make objectively correct decisions in the right amount of time. We will ultimately benchmark vehicles against human drivers to prove statistically that the vehicle made better decisions.

Safety: What if I go beyond the limit of the vehicle?

OK, so you know the tires have a limit where they begin to slide. I’m sure you can envision a scenario where something unexpected occurs, like hitting an unseen ice patch, and the vehicle starts to slide. How would an autonomous vehicle handle this? How does it make sure it doesn’t overcorrect and start sliding the other way? How do we know that the decision a car makes in this scenario is objectively the correct scenario?

Again, we go back to our multibody vehicle dynamics study. Instead of trying to stay within the friction circle, we try to get back into it so the vehicle can take control again. This needs to happen as quickly as possible and the vehicle needs to adjust its path plan to get back to a safe route once traction comes back. Not only this, but the vehicle should also understand what adjustments it can make during the slide so that it can maneuver away from obstacles if it can’t regain traction it time.

You may think that this seems really complicated, and it can be. But you should know that most likely your current vehicle has some of this capability. Ever heard of “ABS,” or “Anti-lock (or skid) Braking System?” This is a piece of hardware controlled by software that detects when a vehicle is about to skid or is already in a skid due to excessive braking and releases the brake enough to get back within the tire friction circle. It may do this may times over a few seconds to just be sure the vehicle is back within the friction circle.

This same concept must be expanded to deal with all skid scenarios. Professor Chris Gerdes of Stanford leads a team building an autonomous electric drift DeLorean (you read that right) named MARTY (Multiple Actuator Research Test bed for Yaw control, and doubly after Marty McFly from Back to the Future). They are conducting research to determine just how an autonomous car can control a drift and recover traction through machine learning.

Figure 6: Stanford’s Autonomous Drift DeLorean MARTY (Source)

So, some combination of vehicle dynamics software and machine learning algorithms will ultimately get deployed to hardware. How do we make sure this works correctly? Same as before. We use models, math, SIL, and HIL to run through various scenarios. We benchmark how fast the car makes each decision, how long it took to regain traction, how long it took to return to a safe path, and how well the vehicle stuck to the path planned during the slide.

There’s a lot of work to be done in this area, and again the public should be concerned about research going on here. But there is a lot of opportunity to think outside the box since there will be a robot controlling the vehicle that can multitask. Perhaps the vehicle can decide that the best way to get out of a skid is to turn each tire on the vehicle a separate angle. This is not something a human could possibly do since we don’t have a steering method for each wheel (nor could we multitask like that). There is a lot of promise in the life saving potential with this challenge, and it is clearly an area where the robot should beat the human.

Comfort: What’s best for the passengers?

This may seem like an area that is not as important as the others, but it’s one the average passenger of an autonomous vehicle is going to experience every second they’re traveling. How do we make sure that the vehicle is comfortable? There’s lots that can be done within the vehicle, like picking the right seats, having the right air temperature, playing the right music, etc. However, there’s a unique opportunity here to apply the field of multibody vehicle dynamics to make the car as objectively comfortable as possible.

What are we talking about here? We’re talking about the holy grail of never being motion sick in a car, of not feeling any bumps, and not being thrown around in your seat as the car maneuvers. And this, as you can probably imagine, comes down to designing the best suspension for the vehicle.

What is suspension though? We typically think of springs when we discuss suspension. As more force is applied on the spring, it compresses, then when the force is lessened, it extends. This is what gives vehicles a “plush” ride. There are typically a couple of “springs” in the car between the passenger and the road: the tire, the suspension spring, and the seat cushion. For this discussion, we will stick to the suspension spring, but they are all important when dealing with comfort.

If you can imagine releasing a compressed slinky while holding one side in your hand, you can imagine the slinky bouncing up and down. It will continue to oscillate up and down due to the “natural frequency” of the spring. Different springs have different “natural frequencies” and in a vehicle suspension, different natural frequencies have different effects on the comfort and performance of the vehicle. Typically, stiffer springs have higher natural frequencies and are meant for higher performance vehicles that are not very comfortable. A typical passenger car has suspension with a natural frequency of between .5–1 Hz, while an F1 car has can exceed 5 Hz.

Figure 7: Spring Natural Frequency (Source)

What does this mean for us? We want a car that doesn’t feel harsh, so in this case, we’d stick to a low natural frequency. This can have other implications in the performance section though.

The other major component of the vehicle suspension system is the “damper.” This is what slows down the speed at which a spring compresses or extends. This helps the vehicle not oscillate due to the suspension springs and adds a more gradual feeling to harsh road conditions. Dampers operate as a function of the speed of compression and extension, and are generally not constant. Some dampers have different performance under sharp bumps, gradual bumps, and during extension and compression.

Figure 8: Spring-Mass-Damper System on a 1/4 Vehicle View (Source)

With the electrification of the vehicle, new technology has allowed us to make more “active” suspension components. The car can adjust its damping and even its springs in realtime to account for road conditions and driving scenarios. There’s even a startup out of Boston, ClearMotion, building a fully active suspension system that doesn’t have springs and dampers at all, just a pushrod actuator that connects the body of the vehicle to the wheels.

Figure 9: Fully Active Suspension Visual (Source)

So, if an engineer wants to use or leverage a passive suspension system with standard springs and dampers for their autonomous vehicle, they need to run through vehicle dynamics simulations and in-vehicle tests. The engineer will look at how much acceleration and vibration the passenger endured and select the correct springs and dampers for that optimization.

If an engineer wants to leverage a semi or fully active suspension, then they will need to write code that controls the actuation of the active suspension components, and test that with MIL and SIL. Then they’ll have to verify again that the entire suspension system works appropriately through simulation, MIL, SIL, HIL, and in-vehicle tests. The engineers will look to have the smallest acceleration effects on the passengers.

It’s worth noting that with active suspension, the active components in the suspension now become additional outputs from the AV stack on top of steering, acceleration, and braking.

The end goal for the engineers tasked with optimizing for comfort is a vehicle that feels like it’s floating in air (without turbulence). The name for this end goal is often called “skyhook theory,” since it’s often likened to a vehicle suspended by an imaginary hook in the sky. We ultimately think the body of a vehicle will need to maneuver similarly to how an airplane maneuvers in the sky, banking the chassis into a turn to ensure all of the force of the turn is taken through the center of the passenger’s body and not throwing them off to the side.

Ultimately, this shouldn’t be a dramatic concern to the public when we think about safety of a vehicle. However, optimizing for comfort is what will allow the vehicle to become something more than just a mode of transportation. It will become an office space, or a conference room, or a shopping center. If the bumps on the road can be eliminated, and the passengers can’t even tell they’re in a moving vehicle, the possibilities are endless.

Efficiency: How do I get the most mileage while doing everything else?

This one’s fairly straight forward. An autonomous vehicle needs to do all the above without dramatically reducing its mileage or driving distance capability. This means optimizing for power efficiency.

There will be engineers from vehicle component suppliers that provide the power consumption of each of their components of the autonomous vehicle. Engineers that integrate the components into the autonomous vehicle will test the power consumption of those systems. The engineers will optimize for the best components and architectures to ensure driving range is prioritized.

There are other things that engineers can do to optimize efficiency as well. For example, Peloton has a technology that allows big rig trucks to “platoon” together to save gas by eliminating the wind resistance from the trucks behind the first:

Figure 10: Peloton’s Fuel Savings Technology (Source)

There’s no reason autonomous vehicle on the road can’t do this as well. Anytime an engineer implements a vehicle-to-vehicle communication link, not only do they have to test the software and hardware on a single vehicle, they must test the communication between vehicles as well. This means more simulation and more “in-the-loop” testing.

This is another one where the public should largely be unconcerned, because the cost savings from more efficient vehicles will fall on the vehicle suppliers and the service providers, and they will be incentivized to improve this element.

Conclusion

Whew, that was a lot! What should be apparent is that there is a field of study that can be leveraged along with advancements in machine learning to determine just how well a vehicle makes decisions in emergency scenarios. Keep in mind that the amount the vehicle designers care about performance, safety, comfort, and efficiency largely varies depending on the type and purpose of the vehicle they build:

Figure 11: Types of Autonomous Vehicles, Evangelos Simoudis Concept and Jason Marks Visual

You can imagine that parents want to know for sure how safe the autonomous school bus that picks up their kids is. If that bus is only allowed on neighborhood roads, engineers may have more leeway to select hardware for their safety systems since they don’t need to act as fast as if the bus were traveling on the highway.

We should all feel safe knowing that these challenges are surmountable, and many engineers have taken steps to guarantee the safety of their autonomous vehicles. But, there’s much more work to do, and there’s no standard that the industry agrees on regarding autonomous vehicle safety.

Read the Rest of the Series: How to ensure the safety of Self-Driving Cars

Part 1 — Introduction