Predictive vector quantizer design using deterministic annealing

A new approach is proposed for predictive vector quantizer (PVQ) design, which is inherently probabilistic, and is based on ideas from information theory and analogies to statistical physics. The approach effectively resolves three longstanding fundamental shortcomings of standard PVQ design. The first complication is due to the PVQ prediction loop, which has a detrimental impact on the convergence and the stability of the design procedure. The second shortcoming is due to the piecewise constant nature of the quantizer function, which makes it difficult to optimize the predictor with respect to the overall reconstruction error. Finally, a shortcoming inherited from standard VQ design is the tendency of the design algorithm to terminate at a locally, rather than the globally, optimal solution. We propose a new PVQ design approach that embeds our previous asymptotic closed-loop (ACL) approach within a deterministic annealing (DA) framework. The overall DA-ACL method profits from its two main components in a complementary way. ACL is used to overcome the first difficulty and offers the means for stable quantizer design as it provides an open-loop design platform, yet allows the PVQ design algorithm to asymptotically converge to optimization of the closed-loop performance objective. DA simultaneously mitigates or eliminates the remaining design shortcomings. Its probabilistic framework replaces hard quantization with a differentiable expected cost function that can be jointly optimized for the predictor and quantizer parameters, and its annealing schedule allows the avoidance of many poor local optima. Substantial performance gains over traditional methods have been achieved in the simulations.