Smart Digital Image Sensors to Enhance Visual Perception Capabilities

Tuesday - June 27 2023, 12:08 UTC - 1 year ago

KAUST researchers have used two-dimensional materials to create a charge-trapping 'in-memory' sensor that is sensitive to visible light and can be programmed optically and erased electrically, and that can perform visual perception capabilities, such as scene recognition. Experiments indicate photo-generated charge can be trapped or stored with an extremely long-lived retention time, and the team was able to perform successful binary image recognition with an accuracy of 91%. The ultimate aim of the research is to create a single optoelectronic device that can perform optical sensing and storage with computing capabilities.

Smart digital image sensors that can perform visual perception capabilities, such as scene recognition, are the result of recent research at KAUST. Exploiting the Nobel-Prize-winning technology of the charge-coupled device (CCD) image sensors found in early digital cameras, Dayanand Kumar, Nazek El-Atab and co-workers have adapted and enhanced the CCD's core structure to create light-sensitive memory devices that can be programmed by light. In particular, the research team embedded the two-dimensional material MoS2 into a semiconductor capacitor (MOSCAP) structure that underpins the charge-storing pixels of a CCD sensor. The resulting Al/Al2O3/MoS2/Al2O3/Si MOSCAP structures function as a charge-trapping "in-memory" sensor that is sensitive to visible light and can be programmed optically and erased electrically. The work was published in the journal Light: Science & Applications.

"The in-memory light sensors are smart multifunctional memory devices that can perform the roles of multiple—traditionally discrete—devices at once, including optical sensing, storage and computation," explained El-Atab. "Our long-term goal is to be able to demonstrate in-memory sensors that can detect different stimuli and compute," El-Atab explains. "This overcomes the memory wall and allows for faster and more real-time data analysis using reduced power consumption, which is a requirement in many futuristic and state-of-the-art applications such as Internet of Things, autonomous cars and artificial intelligence, among others." .

Experiments with light with a wavelength anywhere in the blue to the red spectral region indicate that a photo-generated charge can be trapped or stored with an extremely long-lived retention time. The resulting "memory window" voltage of >2V can be stored for up to 10 years prior to being electrically erased by applying a +/-6V signal. Moreover, it can be operated for many millions of cycles. The ultimate aim of the research is to create a single optoelectronic device that can perform optical sensing and storage with computing capabilities.

By combining their MoS2 MOSCAP structure with a neural network, the team showed that it was possible to perform simple binary image recognition, successfully distinguishing between images of either a dog or an automobile, with an accuracy of 91%. Each image was 32×32 pixels in size, and only the blue information from the images was extracted since that corresponds to the device's peak sensitivity.

The team would now like to further develop the level of optical control. "Current memory devices can be programmed optically but require erasing electrically," commented Kumar. "In the future, we would like to explore in-memory optical sensors that can be fully optically operated." .

In related work published in Advanced Materials, also on the topic of using optoelectronics to perform tasks for artificial visual perception, the team has explored the use of black phosphorus to create an optoelectronic memristive synapse that mimics the brain's neurons for neuromorphic computing applications. Their multilayer device consists of a thin layer of black phosphorus and hafnium oxidte sandwiched between two electrodes, which enables efficient charge transport and transitions between two electrical states, controllable by light.