Novel resource regulator for media-processing algorithms on programmable components [video processing]

Media (video) processing on programmable components offers flexibility but may require too many resources for smooth real-time operation. Furthermore, the implementation of these algorithms must be efficient and cost-effective. This can be achieved with input data-dependent processing and optimal resource utilization. Since the content-dependent resource requirements are often unknown beforehand, the available processing resources per frame are insufficient in certain cases when load changes occur. This causes irregular overloads, e.g. after a shot change. We propose a novel resource regulator to keep the load close to a specified average. This regulator takes measurements within a frame to produce an accurate resource estimate for processing the frame. We have applied it in a 3D-recursive search motion estimator and show that, even at high resource utilization, overloads due to content-dependent processing can be avoided. In addition, object and segment based regulation within a frame become possible.