Key takeaways:
- MSC Software brings to Hexagon Manufacturing Intelligence (Hexagon MI) a deep understanding of the challenges faced by the automotive industry in developing safe and reliable autonomous vehicles without physically driving billions of miles.
- Hexagon MI offers numerous simulation tools as part of its autonomous road vehicles development platform, VTD, for representing vehicle dynamics, traffic and road scenarios, active safety sensors, and motion-control intelligence.
- Hexagon MI’s simulation offerings have been widely applied in the development of automated vehicles to: 1) study realistic traffic conflict scenarios; 2) produce reliable vehicle response simulations at different levels of detail; and 3) enable quick design iterations, based on accurate modeling of active safety sensors and practicable control strategies.
The Validation Challenge of Autonomous Road Vehicles
The proliferation of autonomous road vehicles, enabled mainly by the integration of advanced sensors and real-time artificial intelligence, is expected to help create new mobility experiences and new mobility-related business models, while improving traffic flow and crash-avoidance, as well as reducing environmental pollution and occupant-stress. The biggest hurdle to leveraging these opportunities, however, is the validation of safe operation of autonomous vehicles (AVs) in complex, continuously-changing traffic scenarios that include autonomous, human-driven and parked vehicles, bicycles, pedestrians, traffic signals, and detour postings, in different light and weather conditions.
Over the past decade, AV-prototypes have been tested on public roads to better understand their interaction with the current driving environment. Waymo,3 for example, has covered 8 million miles of AV-driving on public roads since 2009. Additionally, Waymo has also covered 5 billion miles of AV- driving through simulation. This combined strategy of driving AVs on public roads and simulations appears imperative for meeting the predicted need for several billion driven miles to demonstrate the reliability of AVs.4 Moreover, with simulated driven miles being a large multiple of the public road driven miles (over 600x in the case of Waymo), hints at the accelerated understanding and validation that could be achieved by relying on the simulated driven miles.
The validation of robust operation of AVs in all possible driving environments and traffic conflict scenarios can be accomplished much more efficiently and cost-effectively through simulation than driving prototype AVs on public roads. Public roads testing of the AV prototypes should be carried out only after simulation-based design iteration cycles have successfully been completed. In other words, simulation-guided design iterations should be used for wide-ranging experimentation with scenarios, maneuvers, and design variables, then public road driving of prototype AVs should be undertaken, mainly for confirmation of actual performance in critical scenarios and maneuvers.
Simulation Needs of AV Development
AVs need active safety sensors such as RADAR; LiDAR; as well as cameras to identify road markers, other vehicles, pedestrians, signals, sign posts, and obstacles; in addition to sensors that monitor the changing dynamic state of the vehicle.
Active safety sensors are the eyes of the AVs whose signals need to be processed and interpreted quickly and reliably for the vehicle to function according to the artificial intelligence built in it. A major aspect of enhanced and robust perception of the driving environment is sensor-fusion, which leverages multiple sensors to overcome the functional restrictions of individual sensors, and to ensure that unreliable performance of individual sensors, due to natural restrictions or faulty outputs, does not cause unintended motion. Finally, the ability to plan motion and make decisions about motion control in a continuously changing environment is a human capability, which needs to be enabled in AVs through artificial intelligence.
Many large and small organizations globally are working aggressively to enable the three capabilities mentioned above in AVs, for different passenger segments. However, the need to substantially accelerate and validate these efforts remains, and the consensus in the AV development world is that simulation is imperative for better and faster learning, redesign, and validation. With the computational power and simulation tools available today, it is feasible to carry out design, verification, and validation of AVs in large part based on simulation. In fact, due to the sheer complexity and unknowns involved, simulation for developing AVs is considered the first essential step that must be taken.
The virtual design, verification, and validation of AVs require the following:
- Active safety sensor models: Realistic, validated mathematical models of sensors such as RADAR, LiDAR, and cameras with accurate representations of the effects of weather and other factors such as material reflectivity, speed, contrast, and granularity, on their performance.
- Vehicle dynamics models: Several levels of vehicle dynamics mathematical models in terms of detail, to enable the use of appropriate levels of refinement for the driving environment and tradeoffs between accuracy and simulation speed.
- Driving environment models: Realistic and computationally efficient representation of the roads, intersections, shoulders, traffic signals, and the dynamic states and potential motion of other road occupants as well as the interaction with the AV being studied.
- Motion planning and control algorithms: Realistic and robust models for planning the motion of the virtual AVs within the simulated driving environment and controlling their motion based on interaction with its virtual elements.
The investigation of billions of simulated driven miles of AVs needs realistic representation of thousands of driving environments with each driving environment having hundreds, if not several thousand traffic scenarios. Combination of the large number of driving environments and traffic scenarios is expected to create the envelopes around potential critical conditions in which the AVs must perform satisfactorily. The unknowns involved in the validation of AVs are so large that all potential critical cases are not likely to be covered, based on simulation alone. However, it would be reasonable to expect that a very large set of critical scenarios and maneuvers would be covered based on numerical experimentation, and a much smaller set of critical cases is likely to surprise us during prototype testing on public roads and test beds.
Hexagon MI’s Open Development Platform for Simulating Driven Miles
Over the last two years Hexagon has acquired MSC Software, VIRES GmbH, and AutonomouStuff, which, integrated with its prior expertise in sensors, mapping, and positioning intelligence, has put Hexagon MI in a leading position to help accelerate the validation of AVs through simulation.
Hexagon MI’s open Vehicle Test Drive (VTD) platform leverages the simulation and algorithmic elements imperative for AV development such as for vehicle dynamics, sensor-response, driving environment representation and animation, and vehicular traffic scenarios. The platform allows integration of externally developed sensor-fusion algorithms and driving intelligence applications, enabling users to examine the effectiveness of their proprietary solutions. The platform coordinates the interaction between the information generated and used by the different areas mentioned above.
VTD includes reliable mathematical models for predicting the responses of a wide range of active safety sensors such as RADAR, LiDAR, mono and stereo cameras, and ultrasonic sensors including the degradation in their performance due to weather and operating conditions. These models are constantly refined by Hexagon MI based on the knowhow residing in sensor producing divisions including Leica’s and NovAtel’s LiDAR and Global Navigation Satellite System (GNSS) technology.
The virtual driving environments needed for validating AVs can be created and animated on the VTD platform to realistically represent the interaction between the AVs and the elements of the driving environment. The virtual environment can either be generated inside VTD, or by digitizing roads using Hexagon’s Leica Pegasus mapping capability. The changes in the driving environment due to the motion of the AV, included in that virtual environment, are computed. In effect, the driving environment is recreated based on sensor signals and the motion of the surrounding vehicles as well as other occupants to reproduce realistic interactions. The information regarding the motion of the AV, other vehicles, and pedestrians or obstacles could be leveraged by external artificial driving intelligence applications developed by users or third-parties to perform sensor-fusion for object-detection and object-tracking, as well as for motion planning and motion control, to assess the performance of the virtual AVs.
Hexagon MI’s simulation offering for representing vehicle dynamics response at different levels of refinement is crucial for quickly carrying out a large number of design iterations. A detailed multibody dynamics model with high-fidelity tire modeling and refined suspension modeling (i.e., an Adams model from MSC Software) is needed to assess the AV response in severe lateral maneuvers for collision avoidance such as lane changes, with potential rollover, uncontrollable oversteering, or serious occupant discomfort risk. Whereas, for simpler scenarios such as hard braking of the AV in a straight-line motion, with very little yaw instability or rollover concern, a lower order lumped parameter model is enough, allowing many more iterations at the level of design variables. Even when yaw instability could become a concern, perhaps without rollover risk, a higher order lumped parameter model could be used without resorting to a full-fledged Adams model.
The VTD open platform, with published and fully documented application programming interfaces (APIs), allows users to include the artificial driver intelligence that they want to investigate into the AV simulation ecosystem. In this way, users can quickly iterate over the changes to the driver intelligence algorithms based on the response of active safety sensors under critical weather and operating conditions. In addition, a separate artificial driver intelligence capability is available as part of Hexagon’s AutonmouStuff offering. Consequently, it is possible to directly compare the simulated AV response with the prototype AV physical response by using the same artificial driver intelligence in both cases. This should be very helpful for design validation, because design iterations based on simulations alone, at some point, could be drifting away from the physical reality, and direct comparison at different levels of design maturity would help increase confidence in the overall AV response simulations. Essentially, a digital twin of the AV in the driving environment can be developed, based on the vehicle, the driving environment measurements, and the vehicle dynamics simulations connected to the real-time re-creation of the changing driving environment. The digital twin helps refine the simulation models of the AVs at different levels due to the test results obtained at several intermediate levels of design evolution (Figure 1).
Figure 1—Hexagon MI’s Open VTD Platform for Developing Autonomous Vehicles Based on Digital Twins (Courtesy of Hexagon)
Summary
CIMdata closely monitors the trends in autonomous mobility as the technology evolves to make it economically viable for individually owned and shared AVs. The design evolution of AVs based purely on prototype testing on public roads appears to be too time-consuming for commercially viable AVs to become reality within the next decade. The main reason is that the safe and acceptable performance of AVs cannot be demonstrated in a convincing manner. However, a large number of conflict scenarios could be studied quickly through simulation, speeding up the design and the understanding of AV response, while reducing the number of physical prototypes needed along the design evolution path.
Numerical simulation has reduced the design time and cost substantially in the development of conventional vehicles, whose response is quite well understood. In the case of AVs, simulation is needed to accelerate experimentation with a large number of potential conflict scenarios to improve the artificial driver intelligence given the limitations of sensors included in the AVs. It amounts to the tuning of the artificial driver intelligence algorithms to compensate for the shortcomings of the sensor-systems, once a vehicle and the sensors are chosen for developing the AV. Without any standards or regulations dictating the development of sensors or the integration of sensor-systems, each AV design needs the driver intelligence system to be calibrated for that specific vehicle.
Hexagon MI offers several simulation capabilities for AVs, centered around its open VTD platform, such as sensor signals acquisition and processing and sensor response models, high definition maps, AV test data acquisition and processing, Adams vehicle dynamics modeling, as well as realistic driving environment modeling, for rapid learning and design iterations.
Hexagon MI’s VTD platform, with its mathematical simulation and test data processing offerings, has been applied globally in the development of automated vehicles of different levels of automation, either individually or in an integrated fashion. In CIMdata’s opinion, Hexagon MI’s capabilities to orchestrate the data flow between these areas in real-time is much-needed for quickly evolving the design of AVs based on simulated driven miles, instead of public road driven miles. Automotive OEMs, suppliers, and startups should evaluate Hexagon MI’s VTD platform for evolving their AV designs mostly based on simulations with prototype testing at a limited number of stages.
(1) Waymo’s autonomous cars have driven 8 million miles on public roads, The Verge, June 2018
(2) Driving to Safety: How Many Miles of Driving Would it Take to Demonstrate Autonomous Vehicle Reliability, RAND Corporation, 2016