Population coding for a reward-modulated Hebbian learning of vergence control

We show how a cortical model of early disparity detectors is able to autonomously learn effective control signals in order to drive the vergence eye movements of a binocular active vision system. The proposed approach employs early binocular mechanisms of vision and basic learning processes such as synaptic plasticity and reward modulation. The computational substrate consists of a population of modeled V1 complex cells, that provides a distributed representation of binocular disparity information. The population response also provides a global signal to describe the state of the system and thus its deviation from the desired vergence position. The proposed network, by taking into account the modification of its internal state as a consequence of the action performed, evolves following a differential Hebbian rule. Furthermore, the weights update is driven by an intrinsic signal derived by the overall activity of the population. Exploiting this signal implies a maximization of the population activity itself, thus providing an highly effective reward for the developing of a stable and accurate vergence behaviour. The efficacy of the proposed intrinsic reward signal is comparatively assessed against the ground-truth signal (the actual disparity) providing equivalent results, and thus validating the approach. Experimental tests in a simulated environment demonstrate that the proposed network is able to cope with vergent geometry and thus to learn effective vergence movements for static and moving visual targets in realistic situations.