Image Credit: Radiant Vision Systems
Emphasis on safety systems amidst the shift toward semi-autonomous vehicles has led to in-vehicle sensing emerging as one of the main priorities for automotive innovators.
Driver distraction and drowsiness result in accidents that could be avoided with the assistance of sensing systems, which depend on visual and biometric indicators to keep hands on the wheel and eyes on the road.
Driver monitoring systems (DMS) and occupant monitoring systems (OMS) that utilize near-infrared light (wavelengths beyond human visual perception) supply unrestrictive, remote sensing of a person’s presence, eyelid movement, pupil position and gaze, as well as other indicators of driver alertness and occupant safety.
These systems are being quickly implemented into new vehicles, with near-IR light source output a key parameter for system performance. Radiant Vision Systems provides solutions for in-vehicle sensing applications through near-IR light measurement to deliver effective driver monitoring and identification.
Radiant’s portfolio contains automated test and measurement systems and software developed to provide scientific data to assist the design and manufacture of illuminated components such as displays, backlit components and light sources.
These light measurement solutions can be utilized for assessing conventional instrument clusters, backlit icons and center stacks, such as HVAC or volume control, or emergent state-of-the-art systems like head-up displays (HUD)—both standard and augmented-reality formats.
Radiant’s test and measurement solutions are being adapted to ensure performance for near-infrared driver monitoring systems (DMS) and occupant monitoring systems (OMS) as trends move toward light-based sensing for automotive applications.
Light-emitting components from displays to backlit buttons to head-up displays provide important vehicle-interfacing capabilities for the driver and passengers. Image Credit: Radiant Vision Systems
Driver and Occupant Monitoring Systems
There are two main systems applied for in-vehicle sensing—driver monitoring systems, or DMS, and occupant monitoring systems, or OMS. Within these systems, a number of technologies are utilized to supply data for sensing.
In the case of light-based systems, these are usually near-IR VCSELs (vertical-cavity surface-emitting lasers), LEDs and IR cameras that observe drivers or occupants within the vehicle.
There are different types of in-vehicle sensors —for example, thermal or infrared sensors, which monitor the heat signatures of occupants. Steering wheel sensors can, in a standard sense, monitor pressure identifying driver control or go surpass this with EKG functions to evaluate heart rate.
Additionally, for both DMS and OMS applications, other health and biometric monitoring systems include radar for detecting respiration and heart rate.
As a light measurement provider, Radiant’s sensing technologies address those that utilize near-IR light sources in conjunction with IR cameras.
Primary use cases of near-IR light in DMS and OMS applications include pupil detection and tracking eye, recognizing gestures, monitoring driver position and detecting the presence of occupants throughout the vehicle.
DMS and OMS are not new technologies, but these vehicle systems haven’t been rolled out on a global scale yet.
Observing the high end vehicle market, the German automotive marketplace, and in a number of Japanese vehicles, in-vehicle sensing systems have been in place for numerous years—going back more than a decade in some models. But, until recently, these have been the exception.
A Subaru DriverFocus system at the top of the center stack contains a camera and NIR light source to track where the driver is looking. Image Source: planetsubaru.com
New regulations are being devised to facilitate the expansion of these systems, and there is typically a focus on safety awareness in the development of vehicle systems into the future.
Driver distraction is a major concern. Automakers and regulators want to make sure that drivers keep their eyes on the road and maintain safe-control of the vehicle.
A number of emergent safety systems today are applied in conjunction with semi-autonomous features, such as the Tesla Autopilot or Cadillac Super Cruise System.
In Tesla’s system, steering wheel sensors are utilized, and in the Cadillac, both near-IR and steering sensors are employed to make sure that the driver is paying attention and that there are adequate warnings to return a driver’s gaze to the road or reclaim control of the vehicle.
How these systems evolve moving into level 5 autonomy still lacks clarity—but in the short term, systems must make sure that drivers remain focused and in control, especially while they're operating in today’s semi-autonomous scenarios.
At the legislative level, both the United States and European Union are developing regulations. These include the EU General Safety Regulation that defines DMS in new European vehicles as of 2022 and the SAFE (Stay Aware for Everyone) Act of 2020 that is also aimed at installing DMS in vehicles in the US in the future.
The industry has come to the point where there's no good reason not to incorporate systems like these, which drastically improve safety and reduce accidents on the road.
Drivers looking at their phones, talking to the passengers, or looking away, especially with the advent of autonomous driving, increases the risks of driver distraction, and DMS is a clear solution.
In addition to driver monitoring for presence and attentiveness, sensing systems enable occupant identification for vehicle personalization and other luxury applications. Image Credit: Radiant Vision Systems
It should be noted; while the motivation behind in-vehicle sensing technology is, of course, mainly driver and occupant safety, there are several luxury benefits that come with it.
Imagine a driver sits down in a car, the sensing technology scans their face, recognizes the driver, and says, "Hello, [name], welcome to your car," and accordingly adjusts the position of the driver seat, the steering wheel column and applies the preferred light settings.
Automakers can leverage sensing technology to add these improvements in the near future, with the regulation of DMS and OMS moving further in this direction.
Benefits of Near-IR Light for Sensing
Near-IR emitters have been utilized in vehicles for over ten years to supply remote and unobtrusive sensing. Automakers first incorporated these light sources in the steering wheel column, routinely using 890-nanometer (nm) light sources.
Unfortunately, these sources appeared as a deep red glow and could still be seen by the driver at this wavelength. When in direct view of the driver, especially at night, these light sources can become distracting.
Today’s near-IR light sources emit light at various wavelengths. There is a point where light is not so damaging to the eye, around 930-950 nm in the near-IR range, where irradiance from ambient sunlight is absorbed and where light begins to fade from human physical visual perception.
If near-IR light sources with a signature of around 940 nm are utilized, there is less interference from ambient sunlight, the light becomes almost invisible to the human eye and higher intensities of 940 nm light can be applied without presenting eye-safety risks.
In both day and night scenarios, the lights can be used without interference from sunlight and without being seen by the driver. This allows manufacturers to begin using additional emitters or more intensity to flood sensing areas, which creates general appeal for this wavelength range.
Near-IR light provides the illumination for IR cameras used in the Driver Sense driver monitoring system from Cipia. Image Source: Cipia.com
For example, applications of 940 nm sources for near-IR sensing in the consumer electronics market — eye tracking in new virtual and augmented reality devices or facial recognition in smartphones and laptops — supply data sufficient enough to support utilizing this same wavelength for in-vehicle sensing.
With such studies already complete, it's simple for the automotive industry to also transition to installing 940nm LEDs and lasers because they don't have to vet these systems against a full set of regulatory requirements in advance.
Automakers can take a well-developed technology and standardized and start integrating it to meet applications within the auto market.
Applying LEDs or Lasers for Sensing
The beams and distribution patterns of LEDs and lasers are distinct, leading them to be applied to meet various sensing objectives. A near-IR LED generates a very broad diffused distribution of light, whereas lasers produce a narrow beam, like a point source.
There are various elements that can be applied to diffuse the light of laser sources and spread it similar to the distribution of an LED.
Yet, LEDs are installed in more areas of the vehicle interior because most in-vehicle sensing applications can be sufficiently accomplished with LEDs—they generate a wide distribution of light that allows IR cameras to ‘see’ more of the interior, and these light sources can be installed much more economically than lasers.
Illustration of the application of near-IR LEDs to provide illumination for IR cameras. Image Credit: Radiant Vision Systems
As new applications come to depend on identifying the 3D qualities of an object, lasers with diffractive optical elements (DOE) offer precision better optimized than LEDs because they can generate specific patterns with thousands of separate points of information.
Utilizing this method, a diffracted pattern of laser light, also known as structured light or a dot pattern, is cast and bounced off a 3D surface. Deformities in the reflected pattern are deciphered by the sensing system to produce a map of the 3D surface with precise depth and shapes.
This is a major benefit for establishing the unique characteristics of an object, such as the unique shape of a person’s face for facial recognition. For basic detection of relative head, eye and body positions indicating user distraction (as in DMS), LED sources are still a satisfactory fit and possess a lower cost for integration.
Illustration of the application of near-IR lasers used for facial recognition, using structured light patterns. Image Credit: Radiant Vision Systems
Performance Considerations of Near-IR Light Sources for Sensing
Near-IR light sources must generate appropriate radiant intensity to facilitate the different capabilities of the sensing system. Manufacturers may want to know the distribution of a single emitter to cover a particular area from a specific distance.
They may also want to stack multiple emitters and understand how to safely achieve a larger flood of light and gather more information. In all applications, measurement of the light source is necessary to make sure the sensing system can supply key information.
To determine the output of a near-IR light source, measurement technology that can function within the source’s wavelength is necessary. For near-IR sources, this demands radiometric systems that detect light beyond the visible spectrum, generally above 750 nanometers.
Near-IR light sources in automotive applications range from 890 nm to 1050 nm, and within automotive interiors, this wavelength is normalizing at around 940 nm.
Measurement image (left) and cross-section (right) of the angular output of a near-IR LED. This cross-section is used to measure the full-width-half-max (FWHM) radiant intensity of the LED source. Images from TT-NIRI Software from Radiant Vision Systems
Once a suitable measurement system is in place, manufacturers must be able to determine the light source in a variety of ways to best understand how the source meets the needs of the application.
Generally, this includes assessing the geometric distribution of the source and measuring the output (radiant intensity) across or at various points of this distribution, for example, measuring the full-width-half-max (FWHM) intensity value of the distribution, maximum intensity of the light source emission, or the uniformity of the intensity across the distribution (inspecting the intensity of falloffs or hot-spots).
Being able to draw upon that information from the measurement system and software is crucial for understanding how the light source combines with other components of the sensing system for optimal performance.
Near-IR Light Measurement Solutions
Recently, Radiant Vision Systems launched a special camera/lens solution that is modeled toward measuring near-infrared emitters, like LEDs and lasers, utilized in near-IR sensing systems.
The NIR Intensity Lens is a conoscope lens applied in conjunction with a radiometric imager, applying Fourier optics to record the angular emission of a light source with maximum efficiency.
The lens solution was originally designed by Radiant for the consumer electronics market, where near-IR light is primarily used for facial recognition. Of course, what happens in the consumer electronics market bleeds over into automotive eventually.
Illustration of Fourier optics in the NIR Intensity Lens from Radiant Vision Systems (left) and the resulting angular measurement image to ±70° shown in a 2D plot in TT-NIRI Software (right). Image Credit: Radiant Vision Systems
With the move toward utilizing near-IR VCSEL and LED sensing technologies inside cars, the NIR Intensity Lens is now extensively applied for applications in the automotive industry.
Using the NIR Intensity Lens solution to measure its radiant intensity, manufacturers can rapidly understand the performance of these light sources by mapping each source’s full distribution pattern in a single measurement image.
The NIR Intensity Lens is pointed straight at the near-IR emitter to record its angular output. Light emissions out to ±70° are redirected by the lens’s Fourier optics to points on the camera’s internal sensor to generate a polar plot of angular intensity data, visualized as a 2D image.
Utilizing this solution, manufacturers can simultaneously measure across the entire angular distribution of the light source. This facilitates uniformity testing across the distribution while specifically measuring the radiant intensity at a solid angle and observing how that intensity drops-off at extreme angles.
For example, a source could be directly measured, with the measurement system perpendicularly aligned to the light source. Without rotating the measurement source or system, a manufacturer could record how the intensity of the distribution falls off as the extremity of the emission angle increases.
In order to understand where the light source is producing the appropriate amount of intensity for the sensing system to detect, a manufacturer may want to map this falloff, whether that's at 10° distribution, 20° distribution, 30° distribution etc.
Mapping the angular output in terms of scope and intensity is a way of guaranteeing the performance of sensing systems that depend on IR cameras to gather light from a number of angles reflected from an object.
A manufacturer may also want to measure a light source’s intensity at varying distances—for example, is there sufficient light at one foot, three feet, five feet, six feet?
Light used to detect the driver's position necessitates sensing an object three feet away from where the sensing technology is positioned within the dashboard or steering column.
An indirect measurement method could be used to supply information about the light source distribution in terms of its shape, intensity and size at different distances or positions.
Radiant provides IR filters for its portfolio of imaging systems (the Radiant Vision Systems ProMetric® imaging system family) that can be incorporated within each camera’s optical system to enhance measurement capability beyond the visible spectrum and precisely measure near-IR light.
Manufacturers can look at the total distribution of a near-IR light source cast on a wall or screen to see how it maps out over a specific area using one of these cameras with a basic Canon lens—for instance, the area intended to cover a human driver's face.
Using a conventional Canon lens and radiometric camera, the measurement system can be pointed at the distribution of a near-IR light source cast on a wall, enabling the source’s overall distribution area to be easily determined.
Understanding how the intensity may be dropping off at increased distances or how the spot size and spread changes, as opposed to observing the angular output, could be another critical performance parameter for the sensing system.
Illustration of an indirect light measurement setup, using a radiometric imaging system to capture the distribution of a near-IR light source cast on a screen. Image Credit: Radiant Vision Systems
Radiant’s measurement software is also crucial for these test applications. Radiant designed its TT-NIRI™ Software module with a special suite of tests and test parameters for evaluating near-IR light.
These include tests to assess angular distribution, dot or structured light patterns produced by diffracted lasers, time of flight (ToF) parameters and other qualities to determine the radiant intensity and geometry of near-IR light emissions. This information is crucial to understanding the technology incorporated in near-IR sensing applications.
A ProMetric® imaging system and TT-NIRI™ Software from Radiant Vision Systems provide radiometric measurement of near-IR light source distributions and intensity. Image Credit: Radiant Vision Systems
Having the proper measurement tool and analysis software is critical. A near-IR measurement system must first accommodate the wavelength and distribution geometry of the source.
Then, additional considerations may be made for the sensing application or for the stage in the manufacturing process where measurement is incorporated. Radiant brings a range of solutions to the table and makes near-IR performance testing easy to achieve across Tier 1, Tier 2, or OEM applications.
This information has been sourced, reviewed and adapted from materials provided by Radiant Vision Systems.
For more information on this source, please visit Radiant Vision Systems.