The manifold backpropagation algorithm

This paper introduces a generalization of the concept of neural network by allowing the activation functions to be defined from a C^k-manifold to a C^k-manifold. Real-valued neural networks (RNN), complex-valued neural networks (CNN) as well as the recently introduced vector neural networks (VNN) can be seen as special cases of manifold neural networks (MNN). The classical back-propagation algorithm can be extended to manifolds, assuming some hypothesis on the order of differentiability and on some global properties of the manifold, The case of analytic manifolds are briefly discussed as well as the case of manifolds with boundaries