Reliable and robust perception is a critical factor for nighttime pedestrian visibility and safety. France-based Lynred and Umicore have joined forces to develop a thermal sensing system that improves the capabilities and performance of pedestrian autonomous emergency braking (PAEB) in adverse lighting conditions.
Autonomous vehicles (AVs) will need to be able to drive safely day and night and in all weather conditions. As the timeline for Level 5 AVs gets further and further away, it is critical to find ways to reduce road fatalities now.
This collaboration aims to extend the vehicle perception capabilities to the thermal infrared spectrum “not only for Level 4 and 5 AVs,” but “it will also improve a lot of PAEB in current Level 2/Level 2+ cars,” Sebastien Tinnes, global market leader at Lynred, told EE Times Europe.
“Current solutions like visible cameras and radars have limitations when it comes to difficult lighting conditions [e.g., night, sun glare, bad weather],” Tinnes added. “Level 4 and Level 5 will have to integrate additional sensing technologies to extend their use cases upon these difficult conditions. Thermal has the ability to detect heat from living beings. It will perfectly complement the current sensor suite by drastically improving the pedestrian detection in good lighting conditions but also extend pedestrian detection to low-visibility conditions.
“As the thermal sensor is not limited by the car’s headlamp range, it also improves detection distance, giving more time for the car to brake, thus covering higher-speed scenarios,” said Tinnes. “If thermal technology seems inescapable for Level 4 and 5 cars, it would also drastically improve PAEB in current Level 2 cars, which proves to be highly inefficient in low-visibility conditions, as demonstrated by several tests in the U.S. and in Europe.”
Vehicle-pedestrian collisions often result in the death of vulnerable road users. According to the World Health Organization, approximately 1.3 million people die each year as a result of road-traffic crashes, and more than half of all road-crash deaths involve vulnerable road users: pedestrians, cyclists, and motorcyclists.
In the European Union, 21% of road fatalities were pedestrians and 8% were cyclists, according to the 2020 report published by the European Safety Transport Council. The EU’s target is now to reduce the number of people seriously injured on the roads by 50% between 2015 and 2025.
The EU-funded Heliaus (tHErmaL vIsion AUgmented awarenesS) project aims to develop smart thermal perception systems that will detect long-wave infrared (LWIR) light both for in-cabin passenger monitoring and for the car’s surroundings. Ultimately, Heliaus aims to improve object classification of the automotive sensor suite in all light conditions, provide redundancy, and extend vehicle autonomy to Level 3 and beyond, day and night.
The project is using Lynred’s LWIR technology, said Tinnes. LWIR is based on cooled mercury cadmium telluride (MCT) technology for the high-end defense market, on the one hand, and bolometer technology for the defense and civilian applications, on the other hand.
As project leader and coordinator, Lynred is involved in the sensor technology development with CEA-Leti. Umicore is in charge of developing the optics technology and design. Lynred and Umicore have collaborated to optimize the performance of the system that results from the combination of the optics and bolometer designs. “The main difficulty is that the pixel pitch [8.5 µm] is equivalent to the smallest wavelength [8–12 µm], adding complexity to the optics design compared with previous bolometer generation,” said Tinnes. “All the development was done with the automotive application in mind regarding specifications, volumes, and costs.”
Tinnes said that Lynred is currently developing a product, based on its existing 12-µm technology, that could equip cars launched by 2024–2025. The Heliaus project, however, aims to develop the next generation of products, based on the 8.5-µm-pixel–pitch sensor technology for a post-2027 car launch. The outcome of the Heliaus project will be a prototype camera (A-sample) integrating the sensor and optics technologies developed during the project, Tinnes said.
“This technology is designed for the high-volume, low-cost automotive market. It will be ready for customers [B-sample] within two to three years, with production [C-sample] around 2027.”
The Heliaus project could be leveraged on other highly automated systems like industrial, robotic, IoT, or health. It is equivalent to €28.2 million of R&D effort and €8.2 million of EU funding.
Level 3 autonomous driving has been authorized in the European Union since July 1, 2022. In France, the measure came into effect, under certain conditions, on Sept. 1.
At this level, the driver is not required to hold the wheel. However, he or she must be ready to take control of the vehicle in case the automated system requests it.
Commenting on the announcement, Tinnes said, “It is an important date for French and European AV history. Reaching Level 3 required a lot of sensor technologies [visible, radar, and LiDAR] with still-important limitations of use, and the path to Level 3 democratization will be long and may require additional or other technologies like thermal cameras to extend their use cases or service rate.”
5 Trends Shaping the Automotive Perception Market
Look Ma, No Hands! The Road to Autonomous Driving
Anne-Françoise Pelé is editor-in-chief of EE Times Europe.