Financialized methods for market-based multi-sensor fusion

Autonomous systems rely on an increasing number of input sensors of various modalities, and the problem of sensor fusion has received attention for many years. Autonomous system architectures are becoming more complex with time, and the number and placement of sensors will be modified regularly, sensors will fail for many reasons, information will arrive asynchronously, and the system will need to adjust to rapidly changing environments. To address these issues we propose a new paradigm for fusing information from multiple sources that draws from the rich of field pertaining to financial markets, particularly recent research on prediction market design. Among the many benefits of this financialized approach is that, both in theory and in practice, markets are well-equipped to robustly synthesize information from diverse sources in a decentralized fashion. Our framework poses sensor processing algorithms as profit-seeking market participants, data is incorporated via financial transactions, and the joint estimation is represented as a price equilibrium. We use pedestrian detection as a motivating application. Pedestrian detection is a well studied field and essential to autonomous driving. Real world fusion results are presented on RGB and LIDAR data from the KITTI Vision Benchmark Suite. We demonstrate we can achieve comparable performance to state-of-the-art hand designed fusion techniques using the proposed approach.