Editor’s Note: The National Highway Traffic Safety Administration (NHTSA), an agency of the Department of Transportation (DoT), released last December its advance notice of proposed rulemaking (ANPRM) for autonomous vehicles. We asked Egil Juliussen, a veteran automotive industry analyst and EE Times’ resident columnist (“Egil’s Eye”), to break it down for us. This is his second column on the subject.
The ANPRM indicates where NHTSA wants to go with self-driving technology. The following table summarizes the NHTSA automated driving systems (ADS) safety framework document. The left column is a simplified version of the agency’s table of contents. The middle column has more information details with additional content in the right column. I have shrunk 60+ pages to eight pages in this column.
In the executive summary, NHTSA explains that ADS includes the autonomous vehicle hardware and software that perform all driving functions. NHTSA emphasized that developing a safety framework is totally different approach from how existing Federal Motor Vehicle Safety Standards (FMVSS) automotive regulation are developed.
The rational for an ADS safety framework is that the technology for ADS vehicles will advance tremendously and much future innovation will happen. NHTSA wants to make sure it does not impede such valuable future progress with early restrictive regulation.
NHTSA is Defining Safety for Self-Driving Cars, But It Has Questions For You (Part 1 of this series)
Federal Register: Framework for Automated Driving System Safety — NHTSA’s ADS Safety Framework document. Public comments can be filed and read here:
Development of ADS
The development of ADS is well under way and will continue to increase. In July 2020, NHTSA identified on-road testing and development activities in 40 States and the District of Columbia. California is one of the main hubs of testing activity in the world, and 66 companies had valid State permits to test ADS vehicles with safety drivers on public roadways.
ADS development does not start with public, on-road testing. Much of the early testing of prototype ADS is conducted in simulation and/or closed-course testing environments. Public road testing of a prototype ADS typically begins after significant engineering and safety analysis are performed to understand safety risks. Mitigation strategies are put in place to address those risks. It is important to note that the development process is generally both iterative and cyclical. A developer does not “graduate” from simulation to track test, and then to on-road testing, and then deployment.
Instead, developers will continue simulation testing throughout the development process to gain new experience with scenarios that may be encountered rarely in the real world. Similarly, track testing is designed to resemble rare scenarios or that would be dangerous to attempt on public roads until later stages of readiness. This process is repeated, even as on-road testing is occurring.
Experiences gained from on-road testing will often lead to simulation and/or test track replication of situations encountered on public roads to improve the ADS. In other words, the fact that a vehicle is being tested on public roads does not mean that the vehicle or ADS is nearing deployment readiness. Conversely, the fact that a vehicle is still undergoing simulation or track testing does not mean is it not safe to be tested on public roads.
Potential benefit of ADS
NHTSA’s mission is to save lives, prevent injuries, and reduce economic costs due to road traffic crashes, through education, research, guidance, safety standards, and enforcement activity. ADS can aid in achieving that mission, given their potential to prevent, reduce, or mitigate crashes involving human error or poor choices. This potential stems from the substantial role that human factors, distraction, impairment, fatigue, errors in judgment, and decisions not to obey traffic laws, play in contributing to crashes. In addition, they have the potential to enhance accessibility through allowing personal transportation to people with disabilities or people incapable of driving. ADS can improve productivity by allowing people to work while being transported and allowing platooning or entirely automated operation of commercial trucks. Accordingly, NHTSA is placing a priority on the safe development and testing of ADS that factors safety into every step toward eventual deployment.
Activity to remove unnecessary FMVSS rules
To date, NHTSA’s regulatory notices have focused on ADS vehicles without traditional manual controls by assessing the modifications to existing FMVSS that may be necessary. ADS differing safety needs may mean that the installation of some features currently required by the FMVSS are not needed. Examples are mirrors, dashboard controls and some displays.
Need for ADS safety framework
NHTSA typically begins the process of promulgating a FMVSS by identifying the aspect of performance that may need regulation. This is the safety need. NHTSA analyzes real-world crash data and other available information in order to identify safety issues and quantify the size of the safety problems.
Next NHTSA researches potential solutions or countermeasures to the identified safety issues, and then develops performance or related requirements intended to either resolve or mitigate the crash risk identified.
Manufacturers are then required to self-certify, by whatever reasonable means they choose, that their vehicles or equipment meet the performance requirements. Finally, NHTSA assesses vehicle or equipment compliance with those established requirements through the validated test procedures that it has developed.
ADS are in the development stages, and market-ready, mature ADS do not yet exist. Accordingly, no data exist about the on-road experience of ADS that can be analyzed to determine the safety need that potentially should be addressed. Or which aspects of performance need regulation, what would be reasonable, practicable, or appropriate for regulation, or the minimum thresholds for performance. There are no vehicles equipped with mature ADS that can be purchased by NHTSA and tested to validate the effectiveness of a contemplated standard in addressing the safety needs of those vehicles.
NHTSA has no desire to issue regulations that would needlessly prevent the deployment of any ADS vehicle, as this could inhibit the development of a promising technology that has the potential to result in an unprecedented increase in safety.
It is not too soon, however, for NHTSA, with input from stakeholders, to begin identifying and developing the elements of a framework that meets the need for motor vehicle safety and assesses the degree of success in manufacturers’ efforts to ensure safety, while also providing sufficient flexibility for new and more effective safety innovations.
NHTSA seeks to develop a safety framework of standards and/or guidance that manufacturers of ADS would follow to evaluate and demonstrate the safety of their new systems, as produced and, at least in some cases, throughout the lifetime of those systems. In addition, NHTSA seeks to identify the best administrative mechanisms for establishing and implementing engineering and process measures and facilitating agency safety oversight.
Safety Framework: Engineering Measures
Both NHTSA and the ADS industry have done lots of research to prepare for developing an ADS safety framework. These efforts are summarized in this section.
Core ADS safety functions
The core ADS functions include the four pillars of driving: Sensing, Perception, Planning and Control. Sensors on an ADS vehicle might include cameras, radar, lidar, GPS, V2V and/or V2X devices, among other technologies. Sensing also involves scanning the driving environment with emphasis on the travel direction of the ADS.
Perception includes detection and identification of relevant static features and objects (road edges, lane markings, and traffic signs) and dynamic objects (vehicles, cyclists, and pedestrians) detected by sensors within proximity of the vehicle. Perception provide the ADS with information necessary to predict the future behavior of relevant static and dynamic objects that may create collision risk. Perception provides necessary information to the ADS to successful complete all driving tasks.
Planning is the ability of an ADS to establish and navigate the route it will take to its intended destination. The planning function of an ADS builds on the sensing and perception functions.
Control includes implementing the driving plan by delivering appropriate control inputs—such as steering, propulsion, and braking—to follow the planned path. It includes adjusting the plan as necessary based on continuous acquisition and processing of new data concerning the state of the vehicle and surrounding environment.
Other safety functions
ADS safety also depends on a wide array of other functions and capabilities of the system and how it interacts with the humans both inside and surrounding the ADS vehicle.
One safety-related aspect is the vehicle’s ability to communicate with vehicle occupants, other vehicles and people in the driving environment, especially vulnerable road users. The human-machine interaction is expected to have an impact on the operational safety of an ADS, and also on the public acceptance of such systems. ADS capability to detect the malfunction of its own system or other systems in the vehicle accurately and reliably is another important consideration. The ADS must also ensure safe transitions between operational modes developed to respond to any detected issues or malfunctions such as fail safe or limp home modes.
Other aspects that could impact the ability of an ADS to operate in a safe and reliable manner include:
- Identifying reduced system performance and/or ODD in the presence of failure.
- Operating in a degraded mode within reduced system constraints.
- Performing the essential task of transporting occupants or goods from starting point to destination.
- Recognizing and reacting appropriately to communications from first responders, including fire, EMS, and law enforcement.
- Receiving, loading, and following over-the-air software updates.
- Performing system maintenance and calibration.
- Addressing safety-related cybersecurity risks.
- System redundancies.
NHTSA notes that its authorities under the Safety Act are limited to motor vehicle safety. NHTSA is not authorized to regulate general privacy and cybersecurity unrelated to safety.
Federal engineering measure development
One key example of NHTSA’s efforts to develop safety performance models and metrics is the Instantaneous Safety Metric (ISM)—a research document published in 2017. ISM calculates physically possible trajectories that an ADS vehicle and other roadway users could take given a set of possible actions (e.g., steering wheel angles, brake/throttle) within a preset, finite period of time in the future and calculates which trajectory combinations could result in a multi-actor crash.
An updated approach, the Model Predictive Instantaneous Safety Metric (MPrISM), builds upon the ISM concept and modifies its assessment method. MPrISM considers the subject vehicle’s range of fully controllable actions and calculates crash implications under the scenario of best response choices by the subject vehicle and worst choices by other actors in the scene.
Other engineering measures efforts under considerations
In 2018, Rand Corporation issued a report proposing a partial framework for measuring safety in ADS-equipped vehicles. In developing that framework, Rand considered how to define ADS safety, how to measure ADS safety, and how to communicate what is learned or understood about ADS. The Rand report discusses how safety can be measured in a technology- and company-neutral way.
Nvidia published a framework called the Safety Force Field (SFF) that is articulated as a computational method to assess through simulation whether an ADS is monitoring its surrounding environment successfully and not taking unacceptable actions. The SFF goal is avoiding crashes, and it seeks to accomplish this through setting a driving policy that analyzes the surrounding environment and predicts actions by other road users. Based on this analysis, the system would determine potential actions that avoid creating or contributing to unsafe conditions that could lead to a crash.
Intel’s Mobileye published framework called Responsibility Sensitive Safety. RSS address issues with multi-agent safety (defined as safe operation and interaction with multiple independent road users in a given environment). RSS is a mathematical model for multi-agent safety that incorporates common-sense rules of driving while interacting with other road users in a way that minimizes the chance of causing a crash, while operating within normal behavioral expectations. The method is constructed with respect to “right-of-way” rules, objects avoidance, and safe distance maintenance, both longitudinally and laterally. Mobileye also claims that special traffic conditions are covered in the discussion including intersection with traffic lights, unstructured roads, and collisions involving pedestrians (or other road users).
With Mobileye’s CES 2021 presentation on its strategy using RSS and two independent and redundant sensing systems, NHTSA may give Mobileye additional attention.
Safety Framework: Process Measures
SOTIF (Safety of the Intended Functionality) or ISO 21448 works in tandem with ISO 26262 to help a manufacturer assess and mitigate a variety of risks during the development process. ISO 26262 focus on mitigating failure risk and ISO 21448 mitigating foreseeable system misuse.
ISO 21448 is applied to intended functionality where proper situational awareness is critical to safety, and where that situational awareness is derived from complex sensors and processing algorithms. It is especially relevant to emergency intervention systems (e.g., active safety braking systems) and ADAS functions with SAE driving automation Levels 1 and 2.
UL 4600 is a process focused standard that is intended for use by the manufacturers in developing ADS. UL 4600 was developed primarily for ADS.
NHTSA is considering how to use these process standards (ISO 26262, ISO 21448, UL 4600) in developing a new framework for ADS, based either in regulation or providing guidance. Traditional FMVSS may not be suitable for addressing certain critical safety issues relating to the core safety functions of perception, planning, and control. NHTSA requests comment on how Functional Safety, SOTIF, and/or UL 4600 could be adopted, either modified or as-is, into a mechanism that NHTSA could use for minimum performance of an ADS or a minimum risk threshold an ADS must meet for Vehicle Safety Act requirements.
Administration of Safety Framework: Voluntary
NHTSA can establish various mechanisms to gather or generate information:
- How developers are analyzing the safety of their ADS.
- How developers are identifying potential safety risks of those systems.
- What methods developers are choosing to mitigate those risks.
The array of available mechanisms roughly falls into either of two categories: (1) voluntary mechanisms for monitoring, influencing and/or encouraging greater care; and (2) regulatory mechanisms. The first group includes voluntary disclosure, the New Car Assessment Program, and guidance. The second group includes FMVSS and any other compulsory requirements.
AV 2.0 provided guidance to stakeholders regarding the safe design, testing, and deployment of ADS. This document identified 12 safety elements that ADS developers should consider when developing and testing their technologies.
AV 2.0 also introduced the concept of a Voluntary Safety Self-Assessment (VSSA), which is intended to encourage developers to demonstrate to the public that they are: considering the safety aspects of an ADS; communicating and collaborating with the U.S. DOT; encouraging the self-establishment of industry safety norms; and building public trust, acceptance, and confidence through transparent testing and deployment of ADS. See: Voluntary Safety Self-Assessment | NHTSA.
NHTSA believes that VSSAs are an important tool for companies to showcase their approach to safety without needing to reveal proprietary intellectual property. As of January 2021, 26 developers and automakers have published VSSAs, which represents a significant portion of the industry.
Another voluntary reporting mechanism aimed at transparency is NHTSA’s AV Test Initiative, which involves a series of events throughout the country where NHTSA, State and local governments, automakers, and ADS developers share information about activities. AV Test is now a website for companies to share information with the public about their vehicles, including details of on-road testing (website now live: AV TEST Initiative | Automated Vehicle Tracking Tool | NHTSA)
One type of administrative mechanism under consideration is to use guidance to encourage the development of a safety case by manufacturers. In this ANPRM, a safety case is “a structured argument, supported by evidence that provides a compelling, comprehensible, and valid case that a system is safe for a given application in a given operating environment.” For NHTSA’s purposes, “valid” in this context means “verifiable.” Such an administrative mechanism might be implementable more quickly than others and could allow vehicle and equipment manufacturers flexibility in documenting the competence of their ADS in performing sensing, perception, planning, and control of its intended functions.
An ADS competency evaluation could be added in NCAP. While a standalone FMVSS obstacle-course performance test, would likely be inadequate to evaluate ADS competence, such a test might form a useful foundation for consumer information under the NCAP program. This evaluation could be developed and used to measure the relative performance of an ADS in navigating a variable environment (within established operational ranges). The evaluation course would have complex interactions with stimulus road users (e.g., dummy vehicles, pedestrians, and cyclists). The test would have notes describing variances in the manner in which the course was completed. All ADS-equipped vehicles could be expected to avoid collisions, while adhering to a driving model that minimizes the risks of getting into crash-imminent situations and observing operational limitations, such as limits on rates of acceleration and deceleration and limits on absolute speed. This is similar to a driving test for human drivers.
The information NCAP empowers consumers to compare the relative safety of new vehicles and to make informed vehicle-purchasing decisions.
At the current development stage of ADS technologies, the specific areas where regulatory intervention might be most needed remain uncertain and the appropriate regulatory performance metrics and safety thresholds remain unknown. NHTSA has therefore sought to enhance safety through voluntary guidance, instead of mandatory requirements. NHTSA is requesting comment on whether developing further guidance on engineering and process measures remains the most appropriate approach.
Administration of Safety Framework: Regulatory Mechanisms
NHTSA believes that eventually, ADS regulation will be necessary and is exploring ways it could appropriately regulate ADS. The vast majority of vehicle recalls are issued for safety related defects that having nothing to do with FMVSS.
Mandatory reporting & disclosure
NHTSA has taken steps to require the disclosure and reporting of certain information in the context of exemptions. An example is the petition for exemption from Nuro for a 25-mph maximum speed, electric-powered goods-only delivery vehicle that will be operated by an ADS. The terms include post-crash reporting, periodic reporting, cybersecurity, and other general requirements.
NTSA FMVSS setting authority
The Safety Act of 1966 gives NHTSA broad jurisdiction over motor vehicle safety, with a purpose to “reduce traffic accidents and deaths and injuries resulting from traffic accidents.”
Specifically, “‘motor vehicle safety’ means the performance of a motor vehicle or motor vehicle equipment in a way that protects the public against unreasonable risk of accidents occurring because of the design, construction, or performance of a motor vehicle; and against unreasonable risk of death or injury in an accident, and includes nonoperational safety of a motor vehicle.”
NHTSA can issue FMVSS for motor vehicles and its equipment that include recall and remedy of motor vehicles and equipment failing to comply with a FMVSS or containing a defect that poses an unreasonable risk to safety. The FMVSS are intended to be uniform national standards so that compliant vehicles can be sold throughout the United States.
FMVSS are divided into three categories: crash avoidance (100-series), crashworthiness (200-series), and post-crash survivability (300-series). Federal Motor Vehicle Safety Standards – Wikipedia.
NHTSA believes that, at some point, regulation of the ADS will likely be necessary and is exploring ways to regulate ADS. NHTSA could create new FMVSS regulation or modify existing FMVSS regulation for ADS vehicles.
NHTSA has typically used its FMVSS authority in two ways:
- Either to mandate the installation of a proven technology by way of performance standards to address a safety need and subject the technology to minimum performance requirements.
- Or to regulate voluntarily installed technology by subjecting the technology to minimum performance safety requirements.
Applying FMVSS framework to ADS
NHTSA believes that the critical relationship between the safety of an ADS’s design and the vehicle’s decision-making system makes it necessary to evaluate the safety of ADS performance considering appropriate and well-defined Operational Design Domain (ODD) for any system below Level 5.
State and local authorities also play critical roles in roadway safety. Such authorities may establish new rules of the road to address ADS-equipped vehicles.
Reforming FMVSS with rapid tech changes
As the functions and capabilities of motor vehicles are increasingly defined and controlled by software, vehicles will continue to change and improve through software updates that occur during the lifetime of the vehicle. The more quickly vehicle systems can change, the greater the risk that the current regulatory requirements may unnecessarily interfere with innovation. The slow pace of the regulatory process to address unnecessary barriers may also delay the introduction of new safety improvements.
If a new generation of safety standards and other safety regulations is determined to be needed for ADS, they might be written, to the extent allowed by the law, so that they do not have the effect of inadvertently locking future ADS into today’s hardware and software technologies.
In other words, NHTSA should not assume that specific technologies used in today’s vehicles will be used in future vehicle designs. Future standards—particularly those that mandate vehicles be equipped with a certain technology—may be better approached by focusing on objective vehicular functionality as opposed to the performance of a specific discrete system.
A new generation of FMVSS should give the manufacturers of vehicles, sensors, software, and other technologies needed for ADS sufficient flexibility to change and improve without the need for frequent modifications to the regulations.
Multiple FMVSS regulatory approaches: ADS
NHTSA provides 3 examples of potential regulatory approaches for ADS:
- FMVSS requiring obstacle course-based validation in variable scenarios and conditions. This is equivalent to the driver test to get a driver’s license.
- FMVSS requiring vehicles to be programmed to drive defensively in a risk-minimizing manner in any scenario within their ODD. This is similar to the driving policies and metrics described in Mobileye’s RSS, NVIDIA’s Safety Force Field, and NHTSA’s MPrISM.
- FMVSS drafted in a highly performance-oriented manner. This been the traditional approach to drafting standard. This approach is challenging due to complexity of sensors, software and other ADS components.
Timing & Phasing of FMVSS
NHTSA expects a phased approach to regulation of safety performance that may need regulation, given limited NHTSA resources and the evolving technology and business models of ADS development.
NHTSA has already begun the process of providing oversight and guidance—including encouraging disclosure and highlighting key safety aspects relevant for all ADS developers. Where appropriate, the Agency has granted, and will continue to consider granting, exemptions from FMVSS to allow for limited deployment or research in a manner that mitigates safety risk and advances agency technical knowledge.
Critical Factors Considered in Designing, Assessing, and Selecting Administrative Mechanisms
To aid commenters in providing useful information to NHTSA on the administrative mechanisms described above, the following are critical factors that NHTSA will weigh in exploring the strengths and weaknesses of those mechanisms:
- Consistent and Reliable Assurance of Safety. There should be criteria for assessing objectively whether the methods of each manufacturer should meet a common standardized level of rigor, including documentation, and a common standardized minimum level of safety.
- Technology Neutrality/Performance-Based. NHTSA wants to ensure that any mechanism it uses does not pick winners and losers among available and anticipated technologies. Any new standards and regulations should be drafted, to the extent possible, in performance-oriented terms to give manufacturers broad choices among available technologies and flexibility to develop and introduce new technologies without the need to seek amendments or exemptions to those standards.
- Predictability. In developing vehicles and ADS, manufacturers should be able to anticipate what types of performance outcomes they will need to make to demonstrate the safety of their products so they can design their products accordingly.
- Transparency. To build public confidence and acceptance, the methods used by manufacturers to demonstrate the safety of their products should be made known and explained to the public.
- Efficiency. Given that there is neither enough time nor resources for NHTSA to develop physical test procedures for all conceivable driving scenarios, an effort should be made to determine which physical tests have the greatest likelihood to minimize safety risk in an effective manner.
- Equity. All manufacturers should be treated fairly and equally in NHTSA’s assessing of the sufficiency of their safety showings. The mechanism(s) chosen by NHTSA should provide some means to validate that each manufacturer’s demonstration of safety meets or exceeds a common level of rigor and comprehensiveness and that each vehicle meets or exceeds a common minimum level of safety.
- Consistent with Market-Based Innovation. To ensure that innovation is recognized and valued, governmental actions should be consistent with market-based innovation and ensure NHTSA’s actions facilitate and do not unnecessarily inhibit innovation to the extent possible.
- Resource Requirements. Return (measured in added safety) on investment (e.g., efficient use of available resources) is especially important in choosing mechanisms and in deciding which of the core elements of ADS safety performance NHTSA should prioritize.
NHTA Questions to ADS Industry
NHTSA included 25 questions to the ADS industry where it wants expert inputs on how to proceed with ADS safety framework. Over half of the questions was about the safety framework content. Seven question were about administrative mechanisms and four questions on NHTSA statuary authority.