Universal property of learning curves under entropy loss

A learning curve shows how fast a learning machine improves it behaviour as the number of training examples increases. A study of the universal asymptotic behaviour of learning curves for general dichotomy machines is presented. It is proved rigorously that the average predictive entropy <e*(t)> converges to zero as <e*(t)>~d/t as the number of t of training examples increases, where d is the number of modifiable parameters of a machine, irrespectively of the architecture of the machine