A multi-agent operator interface for unmanned aerial vehicles

This paper described a scalable, expandable and platform-independent operator interface system for controlling future long range, high endurance unmanned aerial vehicles. MIIIRO (multi-modal immersive and intelligent interface for remote operation), developed based on open Internet technologies including the Java 2 platform, Java 3D, VRML, RMI, and CORBA, enables collaborative planning and coordinated control of UAVs over a network for achieving a common mission goal. Our work now is focused on the Interface Agent which is envisioned to be a personal assistant cooperating with the operator in planning and control the UAVs. The Interface Agent tries to understand the goal and intention of the operator based on the head motion and other inputs, the task context and current situation. When the need of the operator is identified, the Interface Agent invokes the services of other agents to satisfy the operator´s desire. When the operator needs to watch for a number of concurrent activities, the Interface Agent coordinates the visualization agents to determine where on the display and what information should be presented to the operator