A Novel Interactive Streaming Protocol for Image-Based 3D Virtual Environment Navigation

Image-based rendering (IBR) has attracted recent attention mainly due to its low requirements for generating new scenes based on a sequence of reference images. IBR has paved the way for a new class of application, the free viewpoint television (FTV), which allows the viewer to interactively control the camera and move freely within a scene. IBR can also be used to render complex 3D virtual environments on graphics-constrained devices, such as cellphones and PDAs. In this paper, we propose a new protocol that offers the user a richer virtual experience by pre-streaming the necessary imagery data to generate new views as the user wanders within a 3D environment. We introduce the idea of key partial panoramas, Le., panorama segments that cover movements in any direction by simply strafing from an appropriate key partial panorama and streaming the amount of lost pixels. We have implemented our protocols and evaluated them against two well-known approaches. Experimental results show that our solution outperforms the selected approaches, by minimizing the delay between image updates and by allowing for a more complex navigation scheme.