Solving the Challenges of Nighttime Driving

Nighttime driving has been recorded as the most hazardous time of the day to drive[1] and current ADAS cameras and near-future cameras for autonomous driving can’t operate satisfactorily at night. 

‍

Cameras powered by GatedVision technology, on the other hand, perform very well in dark conditions and provide obstacle detection. The basic automotive implementation of a GatedVision system is to have the GatedVision camera side by side with the visible (standard) camera for full visual operation in lighted conditions and darkness.

The Current State of Automotive Visual Sensors

Current automotive image sensors rely on CMOS technology. CMOS image sensors (CIS) have made huge progress in recent years but there is a lower limit of sensitivity where the input light cannot overcome the internal noise of the sensor. 

‍

In the image below, we see that at low signals, as the SNR drops below five, there is no usable image. So, SNR 5 is the imaging threshold.

In addition, the integration time of a CMOS image sensor is mostly controlled by the ambient light and thus is not optimal for dark scenes while driving, and short exposure is required to prevent the blurring of the image. Passive (regular) CIS does not sense the range information of objects in the scene. CIS can’t make any partition of obstacles and background, which might confuse detection algorithms. Looking at published data [3], we see that practically none of the OEMs equipped their cars with CIS that can perform automatic emergency braking in dark scenery. In addition, daytime performances are below the desired level and achieve accident mitigation only a third of the time. If one adds adverse weather conditions, things get worse. 

This is the current status, according to the IIHS: “Results showed that pedestrian AEB reduced the odds of a pedestrian crash by 32 percent (daytime)” and “Unfortunately, it also shows these systems are much less effective at night, when three-quarters of fatal pedestrian crashes happen.”

A supporting fact for the need for pedestrian safety in darkness is the percentage of pedestrian (77%) casualties in these conditions (see image below). The industry is pushing LiDAR as a safety sensor, but LiDARs will fail in adverse weather, low reflectivity, and in some cases when targeting a specular reflective surface. If one looks at the cost of casualties at night, it is obvious that it is more expensive than equipping new cars with nighttime ADAS. OEMs should not bear the full cost of these systems, which should provide close to 99% safety at all times.

GatedVision Is a Must in All Automotive Sensor Suites

GatedVision is an emerging imaging sensor technology for automotive applications. The technology relies on active laser illumination to allow for low-light imaging. The system flood pulse illuminates the whole field of view (FOV) and creates an image frame out of successive pulses of laser and short integration by the sensor. A GatedVision frame can be a portion of the whole range which is called a range slice (~30m wide) or a full range image (10m to 300m). The full range frame is constructed by multiple slices. 

Slicing enables the system to overcome backscatter, which is the dominant limitation in rain, fog, or snow.

Wavelength

There are several spectral regions to consider in active illumination, as used in GatedVision. The visible can be eliminated since you cannot blind other road users. There are NIR (0.78-1.2 µm near infrared), SWIR (1.2 -3µm short wave IR), and thermal sensors (3-5µm or 7-14µm). Thermal imagers are sensitive to temperature differences in the scene. The 3-5µm (MWIR) spectrum requires cooled detectors and costly systems, therefore it will not fit the automotive market. In the 7-14 µm (LWIR/FIR) band, there are moderate price systems that will detect hot targets (~300K). FIR and MWIR must be installed externally and suffer significant degradation in inclement weather. Thermal imaging is not suitable for GatedVision since the scene radiation is very strong and there is no need for active illumination. As for SWIR, it is still very expensive for reasonable performance. SWIR suffers strong attenuation by water molecules, so it is not useful for rain and fog. 

NIR is the best choice for automotive applications. Within the NIR spectrum, 0.8µm is the preferred choice due to the following:

• High QE
• Low-cost Si detectors
• Windshield transparency
• Does not interfere with LiDARs
• Low aerosol attenuation (rain and fog)

Dynamic Range

At night, GatedVision allows for each range slice to be optimized by itself. A range slice at 200m will have the same brightness as a range slice at 20m. Combining the slices linearly will provide a uniform illuminated scene with no need to adjust for a large intensity span. During the day, the dynamic range is the limiting factor for the available pulses at a certain slice width. Increasing the dynamic range will improve the SNR of a range slice. 

Backscatter

One of the major differences between active illumination systems and GatedVision (gated active system) is the ability of GatedVision to suppress the backscatter created by the suspended water particles in rain or fog. The width of the slice and its distance from the system as well as the distance between the illuminator and camera affect backscatter intensity.

Backscatter in snow: GatedVision (top) vs. a visible HDR camera (bottom)

‍

ADAS and AVs Must Be Able to See at Night

It is a fact that the highest number of casualties and fatal accidents occur while driving at night and in adverse weather conditions [2]. So many of these can be prevented if ADAS and AVs could see at night as clearly as in broad daylight. 

‍

GatedVision technology uses gated imaging working in the NIR wavelength and has the power and capability to fill the gap at night and in adverse weather to provide clear images with high contrast. The GatedVision system suppresses adverse weather interferences, such as backscatter, and uses its inherent ability to overlay slices and create an accurate depth map, as shown in recent studies [4]. 

‍

GatedVision technology is now mature and ready to provide high performance at a low cost to be adopted by the automotive industry and fill the gap that the current ADAS and AV sensor suites have at night and in adverse weather conditions.

‍

References:

[1]

N. N. C. f. S. a. Analysis, “Traffic Safety Facts 2020 Data (DOT HS 813 310),” National Highway Traffic Safety Administration, 1200 New Jersey Avenue SE, Washington, DC 20590, 2022.

[2]

I. I. f. H. S. (IIHS), “Pedestrian crash avoidance systems cut crashes — but not in the dark,” Insurance Institute for Highway Safety, Highway Loss Data Institute, 501(c)(3) organizations.

[3]

Joe Young, “Few vehicles excel in new nighttime test of pedestrian autobrake,” Insurance Institute for Highway Safety, Highway Loss Data Institute, 501(c)(3) organizations, Arlington, 2022.

[4]

Mario Bijelic, Tobias Gruber and Werner Ritter, “Benchmarking Image Sensors Under Adverse Weather Conditions for,” in 2018 IEEE Intelligent Vehicles Symposium (IV), Changshu, Suzhou, China, June 26-30, 2018, 2018.

‍