A note on backpropagation, projection learning, and feedback in neural systems

We show instances where parts of algorithms similar to backpropagation respectively projection learning algorithms have been implemented via feedback in neural systems. The corresponding algorithms, with the same or a similar mathematical expression, do not minimize an error in the output space of the network, but rather in the input space of the network, via a comparison between the function to be approximated and the current approximation executed by the network, which is fed back to the input space: We argue that numerous interlayer resp. intracortical feedback connections, e.g. in the visual primary system of mammals, could serve exactly this purpose. We introduce the paradigm with linear operators for illustration purposes, show the extension to nonlinear operators in function space, introduce projection learning, and discuss future work