Today, cameras can be seen everywhere, in factories, vehicles, public buildings, streets… And their numbers are growing. Most cameras rely on image sensors to convert light in the scene into electronic images, driving up the need for image sensors. But there are many different types of image sensors with different functional characteristics, and designers need to be familiar with the different functional characteristics of different sensors in order to choose the right camera for a specific application.

The number of cameras installed in passenger cars has soared, and some luxury models even have more than a dozen cameras. Automakers need to add more sensors to improve safety, but also need to consider the economic cost and space taken up by each camera, which poses a challenge. As a result, automakers began looking for a solution, hoping to capture images optimized for both human and machine vision with a single camera. There are differences in the image quality applicable to human vision and machine vision, and trade-offs need to be made, so the implementation of this method is equally difficult.

Human vision

The perception of brightness difference between pixels in human vision system is different from that of machine vision algorithm. The human eye’s perception of brightness is nonlinear, that is, if the number of photons in the environment doubles, the brightness the eye perceives only doubles. This requires adjusting the camera image used for human vision to correspond to its dynamic range, thus fully amplifying the details of the brighter and darker areas perceived by the human eye. In addition, we are very sensitive to general color and the flicker of LED light sources (an increasingly common problem), so if the camera causes color distortion, even if the image is clear and otherwise of high quality, it will affect the human visual experience. For passive safety systems that assist driving, such as rear-view cameras, the driver also has an advantage over machine vision systems, because if the image is missing, the driver will automatically find it, and at this time there is no need to rely on the camera. Although this does not cause a major safety incident, the loss of usefulness of the camera is also inconvenient, so the driver does not rely on the camera image, but more active judgment.

PFEA112-65 3BSE050091R65

PFEA112-65 3BSE050091R65

Machine vision

Unlike human vision, an automated system using machine vision looks at the digital value of each pixel in the image, so it responds linearly to the number of photons. Unlike images used for human vision, they must therefore be adjusted to output an image corresponding to the measured pixel values. In addition, machine vision systems must be programmed or use special error detection hardware to detect image defects. Systems that lack this hardware may not function properly and will not inform the driver that their functionality is impaired or inoperable. For active safety systems such as automatic emergency braking, if there is a fault, the false alarm will cause the system to brake when there is no risk of collision, while the false alarm will cause the system to completely fail when there is a danger, resulting in serious consequences. If a driver uses such an assistance system, a message will need to be displayed to indicate that it is not functional, but may not be able to issue a warning of impaired function. Some systems alert drivers that their functionality is impaired or “unavailable,” and they often rely on special hardware features to detect errors or malfunctions in the sensors. Such features are subject to relevant industry standards such as the Automotive Safety Integrity Level (ASIL), and ASIL-enabled sensors will have the ability to detect and report faults, increasing safety. These are two reasons why sensors for machine vision and sensors for human vision need to be configured differently.

120 degree field of view sensor RYYCy image, color processing

Sensor solutions that use a single camera for observation and perception

The good news is that some sensors already have excellent capabilities for both human and machine vision, and can be optimized and output two simultaneous data streams, helping engineers design camera systems that can be used for both human and machine vision functions. This allows automakers to deploy only one camera at a specific location in the vehicle, minimizing footprint and reducing system costs, resulting in images that can be optimized for both workload applications.