Framework for Script Based Virtual Directing and Multimedia Authoring in Live Video Streaming

We propose a novel framework that facilitates automatic editing and authoring of multimedia using static and moving cameras in a dynamic scene. The framework incorporates several video techniques such as object tracking using mean shift and object recognition using Scaled Invariant Feature Transform (SIFT). These techniques are linked together by a comprehensive yet simple-to-program script authoring mechanism based on video event detection. These combined features empower the system to play a virtual director role in live video stream editing and multimedia integration. The system requires minimum human intervention and can leverage production efficiency for both novice and professional users. The experimental results from our prototype system demonstrate that this framework is achievable using inexpensive hardware and standard video cameras. Our system provides comprehensive pre-production authoring capabilities that lend towards integration of video and heterogonous multimedia elements in realtime. We have found this framework to be useful in many applications such as live video streaming, distance education, live entertainment, sports coverage and personal video broadcasting.