New image processing challenges for jointly designed electro-optical imaging systems

Still-image processing algorithms are tailored to and depend crucially upon the properties of the class of images to which they are applied, for instance natural images in consumer digital cameras, medical images in fMRI machines, and binary text images in some photocopiers. We describe a new and possibly very important class of images and tasks for which traditional algorithms seem ill-suited, and for which new algorithms and general methods and concepts are required. This new class of images arises in imaging systems designed through new, joint optimization methods where the optics and the image processing are designed simultaneously in order to yield a high-quality digital image. These new design methods yield intermediate optical images that have unusual spatial, noise and chromatic properties ill-served by traditional image methods. Moreover, these new images present a number of novel challenges in image processing hardware implementations such as constrained space-variance. We describe these briefly new, joint methods for designing digital-optical imaging systems, characterize the intermediate optical images they yield, and some of the digital image processing challenges for producing high-quality still images from these sensed optical images.