Abstract :
Augmented reality (AR) is being investigated in advanced displays for the augmentation of images in a real-world environment. Wearable systems, such as head-mounted display (HMD) systems, have attempted to support real-time AR as a next generation UI/UX [1-2], but have failed, due to their limited computing power. In a prior work, a chip with limited AR functionality was reported that could perform AR with the help of markers placed in the environment (usually 1D or 2D bar codes) [3]. However, for a seamless visual experience, 3D objects should be rendered directly on the natural video image without any markers. Unlike marker-based AR, markerless AR requires natural feature extraction, general object recognition, 3D reconstruction, and camera-pose estimation to be performed in parallel. For instance, markerless AR for a VGA input-test video consumes ~1.3W power at 0.2fps throughput, with TI´s OMAP4430, which exceeds power limits for wearable devices. Consequently, there is a need for a high-performance energy-efficient markerless AR processor to realize a real-time AR system, especially for HMD applications.
Keywords :
augmented reality; electronic engineering computing; helmet mounted displays; multiprocessing systems; network-on-chip; neural nets; 1.22TOPS; 1D bar code; 2D bar code; 3D object; 3D reconstruction; HMD system; NoC; augmented reality multicore processor; camera-pose estimation; general object recognition; head-mounted display; image augmentation; limited AR functionality; natural feature extraction; natural video image; neural network; seamless visual experience; wearable device; wearable system; Augmented reality; Engines; Multicore processing; Neural networks; Real-time systems; Three-dimensional displays; Throughput;
