Ford Motor Company introduced its Blue Cruise, an active advanced assisted-driving system (ADAS) function that enables hands-free highway driving, but only on specified roads. Ford will roll out the feature in the third quarter of 2021 with an over-the-air update to some F-150 trucks and Mustang Mach-E crossovers.
Part of the significance of BlueCruise is that it exemplifies the rapidly growing importance the industry is placing on driver monitor systems (DMS). An oft-cited critical element in safety of active ADAS is DMS with eye-tracking technology.
Ford’s DMS is based on an infrared camera; the system is mounted just above the infotainment screen.
DMS skeptics have dismissed it as “nanny tech” while proponents deem it a crucial safety advancement. Such issues were part of a panel discussion at “Roadmap to the Next-Gen EV&AV” a conference held by EE Times last month.
You might be surprised to see how far DMS has evolved. The driver monitoring system (DMS), overshadowed by the hype surrounding autonomous vehicles, has been underreported and underappreciated – especially among U.S.-based carmakers and consumers.
This neglect, however, is about to change, according to panelists who participated in a DMS panel at our recent conference, “Roadmap to the Next-gen EV & AV.” Regulatory pressures are mounting on carmakers to adopt DMS – especially in Europe and China. As more automated features – such as hands-free highway driving – emerge, auto companies recognize DMS as paramount to keeping human drivers alert and collaborating with cars while driving.
EE Times assembled leading DMS experts, including designers of DMS software and hardware, suppliers of chips and modules, a system integrator and an industry analyst. The panel’s goals were 1) to understand the basic science behind the technology, 2) to learn design options carmakers are exploring for DMS implementation, and 3) to discuss how companies planning to sell DMS to consumers.
Nick Difiore acknowledged, “There has been a relatively slow uptake” among some OEMs and tier ones in “really understanding the human factors.” Difiore is senior vice president, general manager, Automotive at Seeing Machines, one of the world’s leading DMS algorithm and system developers.
By “human factors,” he refers to the science of understanding and predicting human behavior in vehicles, devising the best way to help people drive safely. Despite “a bit of a slow start, interest [in DMS] is becoming really ubiquitous now,” observed DiFiore. The auto industry, he said, “wants to make semi-autonomous vehicles safer and make drivers [to become] safer drivers.”
Today, few people have DMS in their vehicles, noted Caroline Chung, senior business development manager at Veoneer. DMS-equipped vehicles are limited to “GM’s Super Cruise, and only a few others.”
But Chung sees changes afoot. “Since the beginning of the year, the National Highway Traffic Safety Administration (NHTSA) has come out with Request for Information (RFI) on DMS.” This, she said, indicates that the U.S. is likely to follow Europe’s lead.
Andy Hanvey, director of automotive marketing, Omnivision, concurred. “Every day, I am getting more and more RFQs and RFIs [for DMS]. Almost countless… the volume is increasingly rapidly, and the demand is global.” Hanvey also pointed out that many carmakers want not just DMS but also occupant monitoring systems (OMS).
Safety is the key reason for DMS, but OMS also has a safety role, according to Willard Tu, senior director, Xilinx. “Every year [in the United States], about 40 infants die from being left in a hot car. For the deaths of pets, you double the number.”
But it is the economic calculation that is driving OEMs to push both DMS and OMS. Seeing Machines’ DiFirore said, “Once you put in an expensive driver monitoring system, the instinct [among carmakers] is to immediately start looking for how else they can leverage this for comfort or convenience features they can charge for.”
Examples for consumer-oriented applications of OMS include: “a video call in a vehicle” for business, or “a selfie mode” of OMS that automatically takes a picture of occupants with a simple voice command. This is a feature most likely popular in China, explained Xilinx’ Tu. “They are gimmicky,” acknowledged Tu, but carmaker should consider anything that appeals to consumers.
DiFirore noted that in reality, “it’s very hard to charge for safety.” But the introduction of consumer-oriented OMS features could alter that dynamic. “We are seeing the market [for DMS and OMS] about to explode,” he said.
DMS vs. OMS
The technical differences between DMS and OMS are huge.
DMS is all about wanting “an epic amount of information about the driver… It probably demands 60 frames per second vision analytics,” explained Colin Barnden, lead analyst, Semicast Research. DMS monitors not just head pose, but also eye gaze. Driver monitoring measures levels of attention, distraction, cognition to understand the drivers’ attention, state and engagement level in the task of driving.
Occupant monitoring is different, he added. “You are looking at, maybe five to 20 frames per second,” as OMS looks at the wider cabin and positions of the occupants, Barnden explained.
Carmakers’ desire to push both DMS and OMS into one vehicle, however, forces change in the original DMS design. The system must address “resolution of your camera from near field of view to wide field of view,” said Barnden. It has to watch “not just occupants in the front passenger seat but also those in the rear seats.” He suspects that DMS software suppliers are “really trying to meet the needs of the automakers looking for these quite different technical requirements [by DMS and OMS]” of optical path and processing.
Omnivision’s Hanvey added that the traditional DMS camera is an NIR-based sensor, which is monochrome. But if consumers want to do a video conference inside a car, “we must figure out a way to offer additional processing to get them the color image.”
In-cabin sensing, however, doesn’t need to depend on just cameras, noted Veoneer’s Chung. “You can start talking about time-of-flight cameras in there…or maybe even radars or thermal sensing.”
Owl vs. lizard
The technology has evolved over time from an indirect DMS to watch the steering wheel and its rotation. Now the system primarily relies on head pose as the key to reading driver attention. Today, the latest DMS looks at driver attention based on “human factors’ studies, and actually studies of animals,” explained DiFiore.
Comparing an owl with a lizard, he said, “The owl focuses on objects by turning its head. It never moves its eyes.” In contrast, the lizard keeps its head steady while its eyes dart back and forth, as it focuses on different objects. As humans, he concluded, “We have a combination of lizard and owl behavior. When we focus on objects, it’s normally a combination of our eyes moving and our head moving, depending on the task and the range.”
Thus, the latest DMS seeks to know what’s going on in a driver’s head by collecting an awful lot of information including eye-gaze patterns and eyelid behaviors.
Veoneer’s Chung said, “I agree with Nick [DiFiore] that the eye gaze is the future and the highest performance.” But given a growing number of carmakers looking to implement DMS, she noted, “maybe it is just a head post system that’s acceptable in the short term DMS for mass market.”
DMS on a chip?
Presumably, DMS software could run on a chip designed for an infotainment system or even on a SoC designed for ADAS. Xilinx’ Tu, however, believes that the use of a standalone for DMS “is probably a better way to go for the next several years… and that’s what the RFQ tends to be indicating right now.” There are many advantages to a standalone unit, including that “it’s easier to do revisions or updates of DMS without having to address [how such changes might impact] a bigger domain controller.”
Reactive vs. Predictive
As Chung noted, one of the big benefits of DMS is its ability to predict and react by understanding when drivers get distracted or drowsy.
But if so, can DMS do more than issue an alarm? We asked: Why can’t DMS intervene?
DiFiore said, “This is coming.” He calls it “reverse of Level 2” system.
In a Level 2 system in which car and driver cooperate, you can say the car is controlling the vehicle, with the driver as backup, he explained. “But the reverse can also be true. In many cases, the car has the ability to keep itself in a lane and the car has the ability to brake when it needs to. So if the driver is in control, but is failing for whatever reason, why can’t the vehicle back the driver up, instead of the driver backing the vehicle? And this is clearly going to happen, in our opinion,” according to DiFiore.
No worries. Free registration is available here.
Sign in, go to conference, and watch the DMS panel.
The post DMS Evolves from Nanny Tech to ‘Blue Cruise’ Safety appeared first on EETimes.