As any driver knows, accidents can happen in the blink of an eye—so when it comes to the camera system in autonomous vehicles, processing time is critical. The time that it takes for the system to snap an image and deliver the data to the microprocessor for image processing could mean the difference between avoiding an obstacle or getting into a major accident.
In-sensor image processing, in which important features are extracted from raw data by the image sensor itself instead of the separate microprocessor, can speed up the visual processing. To date, demonstrations of in-sensor processing have been limited to emerging research materials which are, at least for now, difficult to incorporate into commercial systems.
Now, researchers from the Harvard John A. Paulson School of Engineering and Applied Sciences (SEAS) have developed the first in-sensor processor that could be integrated into commercial silicon imaging sensor chips–known as complementary metal-oxide-semiconductor (CMOS) image sensors–that are used in nearly all commercial devices that need capture visual information, including smartphones.
