Predictive vector quantizer design by deterministic annealing

Predictive vector quantizers (PVQs) are useful for the compression of sources with memory and other correlated data by incorporating memory into the quantization process. A new approach is proposed for the design of PVQs. By formulating both the prediction and quantization in probabilistic terms, a joint optimization procedure is made possible, which directly minimizes the expected distortion. The approach resolves two longstanding fundamental shortcomings of standard PVQ design. The first shortcoming is due to the piecewise constant nature of the quantizer function; it is difficult to optimize the predictor with respect to the overall reconstruction error. The second complication is due to the PVQ prediction loop, which has a detrimental impact on the convergence and the stability of the design procedure. We propose a new PVQ design approach, DA-ACL, that embeds the Khalil and Rose (2001) asymptotically closed-loop (ACL) approach within a deterministic annealing (DA) framework. The overall DA-ACL method profits from its two main components in a complementary way. DA eliminates the first design shortcoming by offering two benefits: its probabilistic framework replaces hard quantization with a differentiable expected cost function that can be jointly optimized for the predictor and quantizer parameters; and its annealing schedule allows avoiding many poor local optima. ACL is used to overcome the second difficulty and offers the means for stable quantizer design as it provides an open-loop design platform, yet allows the PVQ design algorithm to asymptotically converge to the objective closed-loop performance