Estimation of dependencies based on small number of observations

Summary form only given. The main conceptual problem with estimating the dependencies from empirical data arises when the number of given observations is small. To do well in this situation, one has to use an induction principle that takes into account, along with the performance on the training set, the VC-dimension of the set of functions from which the decision function is chosen. It is therefore possible to construct methods that generalize well even in a very high dimensional input space using a small number of observations. To do the best, given a small sample size, one must only try to solve the problem one really needs to solve, rather than some more general problem. Often, however, this is not easy. For many applications (including financing) it is important to estimate the values of the function at the given points of interest, rather than to estimate the function itself. It is possible to construct algorithms for direct decision making. I describe the problem of learning to make actions, whose solution is not based on the model estimating technique. The problem of learning to make actions is a generalization of the decision making problem. It appears applicable to the many financing problems