Emergence of Memory in Reactive Agents Equipped With Environmental Markers

In the neuronal circuits of natural and artificial agents, memory is usually implemented with recurrent connections, since recurrence allows past agent state to affect the present, on-going behavior. Here, an interesting question arises in the context of evolution: how reactive agents could have evolved into cognitive ones with internalized memory? Our idea is that reactive agents with simple feedforward circuits could have achieved behavior comparable to internal memory if they can drop and detect external markers (e.g., pheromones or excretions) in the environment. We tested this idea in two tasks (ball-catching and food-foraging task) where agents needed memory to be successful. We evolved feedforward neural network controllers with a dropper and a detector, and compared their performance with recurrent neural network controllers. The results show that feedforward controllers with external material interaction show adequate performance compared to recurrent controllers in both tasks. This means that even memoryless feedforward networks can evolve behavior that can solve tasks requiring memory, when material interaction is allowed. These results are expected to help us better understand the possible evolutionary route from reactive to cognitive agents.