Focal-Plane Algorithmically-Multiplying CMOS Computational Image Sensor

The CMOS image sensor computes two-dimensional convolution of video frames with a programmable digital kernel of up to 8 times 8 pixels in parallel directly on the focal plane. Three operations, a temporal difference, a multiplication and an accumulation are performed for each pixel readout. A dual-memory pixel stores two video frames. Selective pixel output sampling controlled by binary kernel coefficients implements binary-analog multiplication. Cross-pixel column-parallel bit-level accumulation and frame differencing are implemented by switched-capacitor integrators. Binary-weighted summation and concurrent quantization is performed by a bank of column-parallel multiplying analog-to-digital converters (MADCs). A simple digital adder performs row-wise accumulation during ADC readout. A 128 times 128 active pixel array integrated with a bank of 128 MADCs was fabricated in a 0.35 mum standard CMOS technology. The 4.4 mm times 2.9 mm prototype is experimentally validated in discrete wavelet transform (DWT) video compression and frame differencing.