Enhancement of a gradient descent or ascent with a function weight for the minimum α-divergence

The natural gradient method with Riemannian metric shows good performance for a parameter estimation problem based on the minimization of a divergence between an observed data and the estimated functions or models. However, it requires much computational times for iterative calculations of the inverse matrix of the metric. In this paper, we consider such estimation problem on an α-affine manifold in a function space over finite discrete variables. By using the auto-parallelism, we derive efficient linear calculation methods for the (-α)-geodesic on the manifold, and for the α-projection onto it from an observed data. The methods can be applied to any initial, terminal, and the interpolated points on the geodesic. Further, we discuss an enhancement of gradient descent or ascent with a function weight by an extension of log, which is related to the α-divergence. The results suggests an adaptive selection of the value of α ∈ R for the family of parametric functions or models (as α-affine manifold) according to the data