Multi-armed bandit formulation for autonomous mobile acoustic relay adaptive positioning

We apply the stationary multi-armed bandit (MAB) formalism to the problem of maximizing underwater acoustic data transmission through adaptive positioning of a mobile relay. Shallow-water environments, in particular, show complex and variable performance across physical space. This is a classic exploration vs. exploitation scenario, since choosing to visit new sites to learn their properties may be disadvantageous for cumulative data transmission if a better-performing site is already known with high confidence. Normal Gittins indices define a rule by which an optimal relay position can be chosen at each decision epoch, without assuming prior information and while pursuing the overall goal of maximizing total data transmission. We present results from shallow-water field experiments conducted with autonomous surface vehicles and acoustic modems transmitting data through a one-way, two-hop network in the Charles River Basin, Boston.