Combining learning and evolution to develop high DOF robot control

This paper describes a method for developing control of high degree-of-freedom (DOF) mobile robots using the seventh generation (7G) system, a software system that incorporates learning, genetic algorithms, and scripting. The control agent is based on a neural network implementing a reinforcement learning process. The network accepts sensor data as input and learns to output control actions. A novel feature of the learning system allows the developer to insert a script as an alternative action output from the neural network learning system. The script significantly reduces the search space for the learning system even if it is sometimes wrong, thereby enabling the learning network to bootstrap toward more effective solutions. An integrated genetic algorithm system modifies parameters of the control agent to evolve the best control agent based on fitness. Fitness is measured by the success of a control agent in learning to control behavior of a simulated model of the robot in selected simulated terrains. An iterative process is described in which the control software is integrated with a simulation model of the robot running in a 3D physics-based simulation system. The method was used to develop control of the OmniTread OT-4, a high DOF serpentine robot.