Multi-object motion real-time perception enabled by all-in-one optoelectronic memory

Abstract

Motion perception is increasingly crucial in diverse artificial intelligence scenarios, but it is largely limited by the dual operation mode (light set and electrical reset) and the physical separation architecture. We propose an all-in-one optoelectronic memory that can provide positive and negative photoresponse to in situ execute image preprocessing, enabling the high efficiency frame difference for multiple moving object perception. This positive and negative photoresponse heavily relies on the light intensity that can alter structure phase of the fibroin protein. The developed optoelectronic memory exhibits short- and long-term synaptic plasticity under both positive and negative photoresponse, faithfully emulating human retina system to detect multiple motions in complex environment. Perception of the multi-object motion in real world is demonstrated, showing an accuracy of 95% and over 30 fps real-time processing speed. The all-in-one memory-enabled artificial retina system lays a significant in-sensor computing architecture for edge dynamic vision perception.