Symmetry of backpropagation and chain rule

Gradient backpropagation, as a method of computing derivatives of composite functions, is commonly understood as a version of the chain rule. We show that this is not true, and both methods are in a sense opposite. As for the chain rule one needs derivatives with respect to all variables that influence a given intermediate variable, the backpropagation calls for derivatives of all variables that are influenced by the present variable. Knowing this, the derivation of the gradient even for complicated neural networks is almost trivial. In a matrix form, both methods differ in the order of matrix multiplication. The use of the chain rule is almost automatic, while the use of the backpropagation can be automatic as an equivalent alternative version of the derivative calculation for composite functions