From Finger Gesture to Finger Choreography: Enabling 3D Live Performances on Smartphones

With the burgeoning availability of touchscreen mobile devices, display monitors and TVs, gestures manipulation on 2D has become a norm: a one or two-finger user interaction with text or 2D images as part of our daily activities. However, interaction with 3D objects and applications remains little explored. Apart from typing, map navigations, most of the 3D object gesture manipulation has been confined to three points of view for rotations, scaling and translations (RST). Currently, commonly seen applications are 3D video games where the user interacts with virtual game pads displayed on screen. 3D graphics interaction are still in its infancy, largely due to the limitation of the 3D surface constructs and the difficulty of interacting in 3D environments from 2D inputs. This paper offers a novel perspective on several classes of 3D structures devised by the author, whereby it enables a one or two-finger gestures to achieve as many degree of freedom (DoF) ad lib. Consequently, it empowers finger choreography on touchscreen of a one-handed handheld device such as a smartphone for 3D performances in real-time. And hence the constraint-free, easy-to-use, rich natural finger gesture manipulation enables extensive 3D object movements along any arbitrary axis at ease. As such, finger trajectories can be composed, synchronized with music to choreograph a live 3D performance. This paradigm shift approach presents a palette of personalized finger gestures and the results of the finger choreography on several classes of equation-based 3D objects. On mobile devices.