Every year during extreme weather, infants, toddlers, and disabled adults are sickened or die overlooked in vehicles. While the numbers are not huge, each case is a tragedy for a family and community. Accordingly, regulators are moving toward requiring that new vehicles be able to detect the presence of a human left in an otherwise empty vehicle. New regulations are not a question of if, but of when and of how.
This presents vehicle manufacturers with a classic Goldilocks problem. There are three primary techniques for human-presence detection in an enclosed environment, presenting a range of cost points and capabilities.
The first alternative is infrared detection: simply looking for a change in the infrared signature of the back-seat region—a change that might indicate the presence of a warm body or of motion. Infrared technology is, to say the least, mature. And it is inexpensive. But it has proved extraordinarily difficult to derive accurate detection from infrared signatures, especially over a wide range of ambient temperatures and with heat sources moving around outside the vehicle.
In an application where frequent false positives will cause the owner to disable the system, and a steady false negative can cause tragedy, infrared is too little.
Then there are radars, cameras
Radar is the second alternative. Small, low-power radar modules already exist for a variety of industrial and security applications. And short-wavelength radar can be superbly informative—detecting not only the direction and range of objects, but even the shapes of surfaces and subtle motions, such as respiration or even heartbeat. If anything, radar offers the system developer too much data.
Radar is also expensive. At today’s prices it would be impractical to deploy it in any but luxury vehicles. Perhaps if infrared is too little, radar is a bit too much.
A closely related approach uses optical cameras instead of radar transceivers. But again, cameras produce a great flood of data that requires object-recognition processing. Also, they are sensitive to ambient light and outside interference, and they are powerless to detect a human outside their field of view or concealed by, say, a blanket.
Furthermore, the fact that cameras produce recognizable images of places and persons creates a host of privacy issues that must be addressed. So, camera-based approaches are also too much.
Looking for just right
Is there something in between? In principle there is. Nearly all new passenger vehicles today offer some sort of in-vehicle Wi-Fi. That means the interior of the vehicle, and its near surroundings, will be bathed from time to time in Wi-Fi signals, spanning multiple frequency channels.
For its own internal purposes, a modern Wi-Fi transceiver monitors the signal quality on each of its channels. The receiver records what it observes as a set of data called Channel State Information, or CSI. This CSI data comes in the form of a matrix of complex numbers. Each number represents the amplitude and phase on a particular channel at a particular sample moment.
The sampling rate is generally low enough that the receiver continuously collects CSI data without interfering with the normal operation of the Wi-Fi (Figure 1). In principle it should be possible to extract from the CSI data stream an inference on whether or not a human is present in the back seat of a vehicle.
Figure 1 To detect a human presence using Wi-Fi, a receiver records what it observes as a set of data called CSI, which can be done without interfering with the normal operation of the Wi-Fi. Even small changes in the physical environment around the Wi-Fi host and client will result in a change of the amplitude and state information on the various channels. Wi-Fi signals take multiple paths to reach a target, and by looking at CSI at different times and comparing them, we can understand how the environment is changing over time. Source: Synaptics
And since the Wi-Fi system is already in the vehicle, continuously gathering CSI data, the incremental cost to extract the inference could be quite modest. The hardware system would require only adding a second Wi-Fi transceiver at the back of the vehicle to serve as a client on the Wi-Fi channels. This might just be the middle ground we seek.
A difficult puzzle
The problem is that there is no obvious way to extract such an inference from the CSI data. To the human eye, the data stream looks completely opaque (Figure 2). There is no nice, simple stream of bearing, range, and amplitude data. There may not even be the gross changes in signature upon which infrared detectors depend. The data stream looks like white noise. But it is not.
Figure 2 Making accurate inferencing of what the CSI data is sensing in real-world scenarios is a key challenge as much of it looks the same. Using a multi-stage analysis pipeline, the Synaptics team combined spectral analysis, a set of compact, very specialized deep-learning networks, and a post-processing algorithm to continuously process the CSI data stream. Source: Synaptics
Complicating the challenge is the issue of interference. In the real world, the vehicle will not be locked in a laboratory. It will be in a parking lot, with people walking by, perhaps peering at the windows. Given the nature of young humans, if they were to discover that they could set off the alarm, they would attempt to do so by waving, jumping about, or climbing onto the vehicle.
All this activity will be well within the range of the Wi-Fi signals. Making accurate inferences in the presence of this sort of interference, or of intentional baiting, is a compounding problem.
But the problem has proven to be solvable. Recently, researchers at Synaptics have reported impressive results. Using a multi-stage analysis pipeline, the team combined spectral analysis, a set of compact, very specialized deep-learning networks, and a post-processing algorithm to continuously process the CSI data stream. The resulting algorithm is compact enough for implementation in modest-priced system-on-chip (SoC), but it has proved highly accurate.
Measured results
The Synaptics developers produced CSI data using Wi-Fi devices in an actual car. They performed tests with and without an infant doll and with babies, in both forward- and rear-facing infant seats. The team also tested with children and a live adult, either still or moving about. In addition to tests in isolation, they performed tests with various kinds of interference from humans outside the car, including tests in which the humans attempted to tease the system.
Overall, the system achieved 99% accuracy across the range of tests. In the absence of human interference, the system was 100% accurate, recording no false positives or false negatives at all. Given that a false negative caused by outside interference will almost certainly be transient, the data suggest that the system would be very powerful at saving human passengers from harm.
Using the CSI data streams from existing in-vehicle Wi-Fi devices as a means of detecting human presence is inexpensive enough to deploy in even entry-level cars. Our research indicates that a modestly priced SoC is capable, given the right AI-assisted algorithm, of achieving an excellent error rate, even in the presence of casual or intentional interference from outside the vehicle.
This combination of thrift and accuracy makes CSI-based detection a just-right solution to the Goldilocks problem of in-vehicle human presence detection.
Karthik Shanmuga Vadivel is principal computer vision architect at Synaptics.
Related Content
- Detecting Human Stress Using Sensors
- How Wi-Fi sensing simplifies presence detection
- AIoT Chip Slashes Power Consumption for Person Detection
- Human Vision Inspires a New Generation of Cameras—And More
- ‘Wi-Fi on Steroids’ Technology Allows Manufacturing Industry to Finally Cut the Cord
The post In-vehicle passenger detection: Wi-Fi sensing a ‘just right’ solution appeared first on EDN.