With the widespread deployment of sensing and computing modules in edge infrastructure, high-speed perception, computation, and reconstruction of natural scenes are critical. Most of the existing end-to-side visual intelligence is a sense separation paradigm, that is, the sensor senses and collects the optical signal, converts it into an electrical signal, and then computs the intelligent task. The frequent conversion between light and electricity, as well as the slowdown in the development trend of electronic computing performance in the post-Moore era, restricts the speed and bandwidth of end-to-end intelligent processing.

PCIE-6321 Aiming at the sensing bottleneck of edge systems, Fang Lu’s research group in the Department of Electronic Engineering of Tsinghua University proposed an integrated all-optical intelligent computing architecture for natural scenes, and developed an optical parallel computational arraychip (OPCAchip). Breaking through the problem of incoherent light field matrix calculation, abandoning the sensory separation paradigm of “light perception-electrical calculation”, realizing all-optical machine vision with end-to-end calculation of “light in-light out”, increasing the speed of intelligent processing of incoherent light field to the order of nanoseconds, and supporting natural light field processing on the scale of 100 billion pixels per second.

The research group described the modulation relationship between the natural scene optical signal and the on-chip optical field, proposed the incoherent-coherent all-optical multiplier and addition operator, established the multi-dimensional all-optical neural network in the space spectrum domain, broke through the problem of intelligent sensing of incoherent optical field, proposed the all-optical on-chip resonant neuron model, and developed the all-optical sensing and computing integrated array chip OPCA. Support parallel perception and high-speed calculation of light field signals in natural scenes.

PCIE-6321

On this basis, the research group constructed a multi-wavelength all-optical neural network, integrating perception, calculation and reconstruction, achieving end-to-end all-optical codec and light field reconstruction tasks, and completing end-to-end reconstruction of natural scenes on a nanosecond time scale. In the visual intelligence classification task, the measured response time of OPCA chip is 6.0 nanoseconds, the visual processing bandwidth is THz, that is, it can process 100 billion pixels per second, compared with the traditional camera acquisition, storage, intelligent processing of the sensory separation system (response time is mostly milliseconds, processing 100 million pixels per second), the speed is increased by 6 orders of magnitude. The bandwidth is increased by 2 to 3 orders of magnitude.

PCIE-6321 The all-optical machine vision represented by OPCA chip marks an important step in the development of intelligent optical computing technology to the field of end-side application. The chip will be integrated with large-scale intelligent optical computing chips in the future, breaking through the speed and power consumption shackles brought by photoelectric/electro-optical conversion, and achieving end-to-end all-optical intelligent sensing from light perception to light processing. With its high speed and high bandwidth sensing characteristics, it is expected to bring subversive breakthroughs in performance for autonomous driving, industrial detection, intelligent robots, VR/AR and other fields, and has broad application prospects.

Recently, Parallel photonic chip for nano-second end-to end image processing, transmission, Parallel photonic chip for nano-second end-to end image processing, transmission, and reconstruction, published in the journal Optica.

The Department of Electronic Engineering of Tsinghua University is the first unit of the paper, Associate professor Fang Lu is the corresponding author of the paper, and Wu Wei, PhD student of the Department of Electronic Engineering and Zhou Tiankuang (Shuimu scholar), postdoctoral fellow of the department of Electronic Engineering, are the first authors of the paper. The research was supported by the 2030 Major Project of the Ministry of Science and Technology, the National Natural Science Foundation of China, the China Association for Science and Technology, the China Postdoctoral Foundation, and the Tsinghua University-Zhijiang Laboratory Joint Research Center.