Combining Uncertainty Sampling Methods for Active Meta-Learning

Meta-learning has been applied to acquire useful knowledge to predict learning performance. Each training example in meta-learning (i.e. each meta-example) is related to a learning problem and stores features of the problem plus the performance obtained by a set of candidate algorithms when evaluated on the problem. Based on a set of such meta-examples, a meta-learner will be used to predict algorithm performance for new problems. The generation of a set of meta-examples can be expensive, since for each problem it is necessary to perform an empirical evaluation of the candidate algorithms. In a previous work, we proposed the active meta-learning, in which active learning was used to reduce the set of meta-examples by selecting only the most relevant problems for meta-example generation. In the current work, we proposed the combination of different uncertainty sampling methods for active meta-learning, considering that each individual method will provide useful information that can be combined in order to have a better assessment of problem relevance for meta-example generation. In our experiments, we observed a gain in meta-learning performance when the proposed method was compared to the individual active methods being combined.