FlightTracker: a novel optical/inertial tracker for cockpit enhanced vision

One of the earliest fielded augmented reality applications was enhanced vision for pilots, in which a display projected on the pilot´s visor provides geo-spatially registered information to help the pilot navigate, avoid obstacles, maintain situational awareness in reduced visibility, and interact with avionics instruments without looking down. This requires exceptionally robust and accurate head-tracking, for which there is not a sufficient solution yet available. In this paper, we apply miniature MEMS sensors to cockpit helmet-tracking for enhanced/synthetic vision by implementing algorithms for differential inertial tracking between helmet-mounted and aircraft-mounted inertial sensors, and novel optical drift correction techniques. By fusing low-rate inside-out and outside-in optical measurements with high-rate inertial data, we achieve millimeter position accuracy and milliradian angular accuracy, low-latency and high robustness using small and inexpensive sensors.