Multi-viewpoint images for multi-user interaction

In this paper, we introduce the idea of projecting different image elements from multiple viewpoints and combining them in a single stereoscopic image. We use this technique to enable multi-user interaction and co-located collaboration in large projection-based immersive display systems. Viewing stereoscopic images from a viewpoint outside of the projection viewpoint introduces parallax, a skew distortion of the spatial image, resulting in a misalignment between real and virtual object positions for multiple viewers sharing a single-view stereoscopic display system. With multi-viewpoint images, we can project different image elements from multiple viewpoints, corresponding to the viewing positions of multiple users, and combine them in a single image. We use this technique to project interaction elements for each user in the correct position and depth, matching, from the user´s point of view, the tracked real positions of interaction devices with the virtual position of visual and functional interaction elements such as pointers, menus or picking rays. We introduce a rendering method to combine projections for different viewpoints in a single, consistent stereoscopic image. We have used multi-viewpoint images for interaction in applications, where a large audience is looking at a non-head-tracked immersive presentation and a guide user is controlling the application with direct interaction techniques. Furthermore we have developed complex interaction scenarios, where multiple users share a conventional single-view projection-based display environment for co-located collaboration.