Complex problem solving with reinforcement learning

We previously measured human performance on a complex problem-solving task that involves finding which ball in a set is lighter or heavier than the others with a limited number of weightings. None of the participants found a correct solution within 30 minutes without help of demonstrations or instructions. In this paper, we model human performance on this task using a biologically plausible computational model based on reinforcement learning. We use a SARSA-based Softmax learning algorithm where the reward function is learned using cascade-correlation neural networks. First, we find that the task can be learned by reinforcement alone with substantial training. Second, we study the number of alternative actions available to Softmax and find that 5 works well for this problem which is compatible with estimates of human working memory size. Third, we find that simulations are less accurate than humans given equivalent amount of training We suggest that humans use means-ends analysis to self-generate rewards in non-terminal states. Implementing such self-generated rewards might improve model accuracy. Finally, we pretrain models to prefer simple actions, like humans. We partially capture a simplicity bias, and find that it had little impact on accuracy.