Tuning of reinforcement learning parameters applied to OLSR using a cognitive network design tool

In wireless mesh networks, with the standard Optimized Link State Routing (OLSR) metric (i.e. hop count), traffic is routed on the shortest path without considering factors such as traffic distribution and link capacities. Consequently, some nodes may get overloaded from the uneven utilization of network resources. OLSR can be modified to use other link cost metrics, with route selection based on lowest cost path. With delay as the metric, OLSR reduces average round trip time but the load-aware routes may cause wide variance in delay and packet reordering due to route oscillations. We describe a new hybrid routing approach that combines the strength of a) link state routing (e.g. fast convergence), b) load-aware routing (e.g., avoiding congested paths) and c) cognitive routing (e.g. learning to avoid path oscillations). In particular, we investigate the use of Q-learning with OLSR to increase network capacity and reduce congestion delay. We present simulation results for a 36 node dynamic mobile ad hoc network, with standard OLSR, a non-cognitive load-aware OLSR (OLSR-D) and our new hybrid cognitive load-aware OLSR (OLSR-Q). We show that OLSR-Q >; OLSR-D >; OLSR in terms of reducing delay and increasing network capacity. Furthermore, we show that, unlike conventional cognitive Q-routing protocols, our hybrid approach does not reduce performance at low load. Although OLSR-Q can significantly reduce delay and improve capacity, the learning time can reduce connectivity and the distribution of more link state information can reduce raw link capacity. We show how adding OLSR, OLSR-D and OLSR-Q as routing options into the Cognitive Network Engineering Design Analytic Toolset (C-NEDAT), we can select the best routing protocol and parameters (e.g., learning rate) for a given network and its mission. We verify simulation performance improvements by implementing the OLSR-Q in on a 9 node wireless testbed.