The sampling and reconstruction of time-varying imagery with application in video systems

Sampling is a fundamental operation in all image communication systems. A time-varying image, which is a function of three independent variables, must be sampled in at least two dimensions for transmission over a one-dimensional analog communication channel, and in three dimensions for digital processing and transmission. At the receiver, the sampled image must be interpolated to reconstruct a continuous function of space and time. In imagery destined for human viewing, the visual system forms an integral part of the reconstruction process. This paper presents an overview of the theory of sampling and reconstruction of multidimensional signals. The concept of sampling structures based on lattices is introduced. The important problem of conversion between different sampling structures is also treated. This theory is then applied to the sampling of time-varying imagery, including the role of the camera and display apertures, and the human visual system. Finally, a class of nonlinear interpolation algorithms which adapt to the motion in the scene is presented.