Breakthroughs to Affordability
Mass utilization of SAE automation level 3 (conditional automation) and above depends on the compute power needed to process the incoming sensor data, affordable sensor technologies, and the artificial intelligence required to carry out driving commands that deliver reliable and safe transportation in real-world conditions.
Figure 1: FLIR Thermal Sensors are affordable and integrated in a wide range of consumer, industrial, and military products today.
Autonomous vehicle (AV) and advanced driver assist system (ADAS) sensor technologies encompass radar, ultrasonic sensors, cameras, and light detection and ranging (LIDAR), but this integration of sensors falls short in creating a completely comprehensive and safe solution.
The unique capability to detect the long wave infrared (LWIR) energy given off by everything, means that thermal sensors can see through most fog, darkness, and smoke, and they are unaffected by sun glare. Their strength, to detect and classify pedestrians in obscured or highly-cluttered environments, means they are a crucial technology to help minimize the number of pedestrian fatalities, which amounted to 5,987 in 2017 in the U.S., of which 4,490, or 75%, occurred after dark1.
Breakthroughs in thermal imaging technology, increased manufacturing volume, and improved manufacturing techniques, mean that it is becoming possible to mass produce affordable thermal sensors2 for SAE automation level 2 and above. A common misconception is that thermal sensors, with their background in military use, are too expensive for automotive integration.
FLIR is focused on developing thermal imaging technology that is cost-effective. As the global leader in thermal imaging, FLIR has produced millions of thermal sensors, including over 500,000 sensors into driver warning systems which are installed on a number or automotive nameplates, including Peugeot, General Motors, BMW, Audi, Volkswagen, to name a few. Working closely with Veoneer, our tier-one automotive customer, FLIR has decreased the cost of thermal imaging technology for the automotive market as well as other emerging consumer markets, including mobile phones3 and consumer drones.4
Thermal cameras amounted to thousands of dollars each for VGA resolution or higher, until recently. Today they are an order of magnitude lower in price because of technology improvements and volume. FLIR continues to innovate and reduce camera costs further, which enables developers and engineers of AV and ADAS to add affordable thermal imaging to their sensor suites.
Lowering Costs in the Manufacturing Process
Basically, the fabrication of thermal imaging sensors is like that of silicon computing hardware. Manufacturing inputs include foundry costs, the silicon wafer, and yield. Per-sensor cost is calculated by dividing total costs by the number of chips which are available to sell. In a thermal camera, the main components are electronics, a sensor core (microbolometer), lens, and case.
Unfortunately, infrared image sensors cannot follow Moore’s Law. How small an infrared imager sensor’s pixel can be made is limited, because of a performance tradeoff as the pixel shrinks and approaches the wavelength of interest (LWIR sensors detect 8 to 14 microns wavelength radiation).
Yet, over the past decade, FLIR has decreased the pixel geometries from 50 x 50 microns to 12 x 12 microns, which means a reduction of 83% in area. This, in combination with increased scale and wafer-level packaging, and process optimization, has allowed FLIR to attain the lowest costs in the thermal-image sensor market.
What is Driving the Infrared Market?
A specialist in market research in infrared and thermal imaging, Maxtech International®, Inc., has supplied the data in Figure 3. Including the estimated global average end-user volume and price for all thermal cameras using uncooled microbolometer technology sold for commercial utilization.
Estimated market data is shown from 2003 through 2017 with forecasted amounts for 2018 onwards. Forecasted units per year include vehicle night vision systems from Veoneer (FLIR thermal sensors), but they do not include significant AV adoption, because it is likely to start in 2022 or 2023 and should be at a price point of a few hundred dollars.
Volume and price for non-commercial utilization is not included in Figure 3. The non-commercial (military) market advances technical and manufacturing innovation, but is not driving significant volumes and price reductions at present.
New efforts are being made to utilize uncooled sensors in a wide scope of soldier systems however, because of increased U.S. Department of Defense budgets. Microbolometer technology is being developed incrementally, which allows for continuing yield improvements, larger formats, digital-pixel ROICs, small pixels, etc.5
Infrared has become a mature technology, but constant ongoing yield improvements at FLIR and enhancements in smaller pixel design promise to lower costs even more. It is forecasted that an additional two times reduction in cost can be achieved over the next few years, based on current development plans. This positively compares with the necessary ten times cost reduction of LIDAR systems required to meet OEM cost targets.
Adoption of significant volumes of thermal cameras for SAE automation levels 2 and 3 are set to begin in 2022 or 2023, with yearly growth rates of 200% to 300% through 2030. Thermal cameras will be an affordable component for the ADAS and AV sensor suites, with the planned improvements and automotive manufacturing scale.
Figure 2: A grill-mounted FLIR thermal sensor integrated by Veoneer provides thermal vision for the BMW X5.
Leading a Developing Market
The AV and ADAS market is in the development phase and FLIR is leading the way toward thermal camera adoption. FLIR already produces full-rate production thermal sensors for automotive night vision. Furthermore, the FLIR Boson™-based FLIR ADK™ (Automotive Development Kit) is available and specifically designed for ADAS developers to utilize in test vehicles, which allows for new operational design domains.
Developers can employ the FLIR ADK and the free FLIR thermal starter dataset to assess how thermal cameras detect and classify pedestrians, cars, bicycles, dogs, and other vehicles quickly and more reliably than in the past. The free starter dataset encompasses over 14,000 annotated thermal images, and considerably more images are available for FLIR partners.
As adoption grows, and the value proposition of thermal imaging becomes well established, volumes will increase further and drive FLIR thermal imaging systems to price points and higher resolutions that manufacturers can utilize on all autonomous transportation vehicles.
Figure 3. Estimated global average end-user price and volume of thermal cameras. ©Maxtech International®, Inc.
1 Richard Retting and Sam Schwatz, Governors Highway Safety Association Pedestrian Traffic Fatalities by State (2017 Preliminary Data) www.ghsa.org
5 The World Market for Commercial & Dual-Use Infrared Imaging & Infrared Thermometry Equipment (Vol. IRW-C), published by Maxtech International®, Inc., 2018.
This information has been sourced, reviewed and adapted from materials provided by FLIR Cores and Components Group.
For more information on this source, please visit FLIR Cores and Components Group.