A 1 mJ/Frame Unified Media Application Processor With Dynamic Analog-Digital Mode Reconfiguration for Embedded 3D-Media Contents Processing

In this paper, a unified media application processor (UMAP) is presented for 2D/3D image analysis/synthesis applications on handheld devices. UMAP integrates parallel and sequential processing layers which consist of heterogeneous functional IPs for general media contents processing on today´s application processors (AP). Based on the heterogeneous many-core platform, UMAP supports not only graphics and vision processing for real-time augmented reality (AR) but also disparity estimation and 3D display synthesis for 3D-view AR acceleration. A new concept of 3D-view AR which synthesizes 3D display contents from two vertically aligned stereo images and a self-constructed disparity map is introduced to achieve true realism for next generation mobile devices. For low-cost 3D-view AR processing, a homography-based disparity estimation (HDE) algorithm is proposed to construct a disparity map between two stereo images with small implementation overhead. For real-time and energy-efficient system organization, workload-balanced 3-stage pipelined architecture and a mixed-mode feature extraction engine (FEE) are also implemented in UMAP. The 3-stage pipelined system which consists of graphics, vision, and display operation stages reduces per-frame execution latency, while dynamic analog/digital mode reconfiguration based on mixed-mode FEE reduces per-frame energy dissipation, so real-time energy-efficient 3D-view AR can be realized in UMAP. FEE performs high-speed corner detection for vision processing based on four pairs of analog current contention logics (CCLs). Especially, a diode-connected current sensing stabilizer (CSS) in each CCL reduces minimum sensing current for corner detection, so average power consumed in CCL is reduced by 44.9%. In 2D or 3D-view AR processing, FEE with four CCLs replaces the parallel processing core cluster which is the most power hungry IP in UMAP, so 96.7% of cluster power and 99.1% of target detection time are saved in real operation. Based- on the 3-stage pipelined architecture with the dynamic mode reconfiguration technique, the entire UMAP achieves up to 64.4% of energy reduction compared to the previous state-of-the-art media processors in full operation.