Video quality and traffic QoS in learning-based subsampled and receiver-interpolated video sequences

Sources of real-time traffic are generally highly unpredictable with respect to the instantaneous and average load which they create. Yet such sources will provide a significant portion of traffic in future networks, and will significantly affect the overall performance of and quality of service. Clearly high levels of compression are desirable as long as video quality remains satisfactory, and our research addresses this key issue with a novel learning-based approach. We propose the use of neural networks (NNs) as post-processors for any existing video compression scheme. The approach is to interpolate video sequences and compensate for frames which may have been lost or deliberately dropped. We show that deliberately dropping frames will significantly reduce the amount of offered traffic in the network, and hence the cell loss probability and network congestion, while the NN post-processor will preserve most of the desired video quality. Dropping frames at the sender or in the network is also a fast way to react to network overload and reduce congestion. Our interpolation techniques at the receiver, including neural network-based algorithms, provide output frame rates which are identical to (or possibly higher than) the original video sequence´s frame rate. The resulting video quality is essentially equivalent to the sequence without frame drops, despite the loss of a significant fraction of the frames. Experimental evaluation using real video sequences is provided or interpolation with a connectionist NN using the backpropagation learning algorithm, the random NN (RNN) in a feed-forward configuration with its associated learning algorithm, and cubic spline interpolation. The experiments show that when more frames are being dropped or lost, the RNN performs generally better than the other techniques in terms of resulting video quality and overall performance. When the fraction of dropped frames is small, cubic splines offer better performance. Experimental data - hows that this receiver-reconstructed subsampling technique significantly reduces the cell loss rates in an asynchronous transfer mode switch for different buffer sizes and service rates.