Advancing the digital camera pipeline for mobile multimedia: Key challenges from a signal processing perspective

The ubiquity of digital color image content continues to raise consumer technological awareness and expectations, and places a greater demand than ever on algorithms that support color image acquisition for mobile devices. In this paper we consider the key signal processing challenges to advancing the digital camera pipeline for mobile multimedia, with a particular focus on advances that have the potential to enhance image quality and reduce overall cost and power consumption. We first examine key technical challenges to pipeline design presented by demands such as shrinking device footprints, increasing throughput, and enhancing color fidelity. We then describe a recently introduced analytical framework based on spatio-spectral sampling for color image acquisition, and discuss its potential implications for quality and cost improvements. We then describe a number of resolution-distortion trade-offs, in particular noise processes and crosstalk, and show via simulation how a spatio-spectral acquisition framework helps to pinpoint aspects of pipeline design that can enhance computational efficiency and performance simultaneously.