Architectural decomposition of video decoders for many core architectures

The microprocessor industry trend towards many-core architectures introduced the necessity of devising appropriately scalable applications. In video decoding, the main challenges are the optimized partitioning of decoder operations, efficient tracking of dependencies and resource allocation/synchronization for multiple threads. In this paper, we propose a decoder architecture that replaces the conventional monolithic design with a pipelined structure. Bit stream decoding and image processing are separated from each other by means of a Meta Format Stream. The Meta Format is forward-oriented and self contained and multistandard capable, so that processing of Meta Streams is independent of the originating bit stream. Our approach does not require special coding settings and is applicable to accelerated decoding of any standards-compliant bit stream. A H.264 multiprocessing proposal is presented as a case study for the potential our our decoder architecture. The case study combines coarse grained frame-level parallel decoding of the bit stream with fine-grained macroblock level parallelism in the image processing stage. The proposed H.264 decoder achieved speedup factors of up to 7.6 on an 8 core machine with 2-way SMT. We are reporting actual decoding speeds of up to 150 frames per second in 2160p-resolution.