D-entropy controller for interpretation and generalization

In this paper, we propose a method to control D-entropy for better generalization and explicit interpretation of internal representations. By controlling D-entropy, a few hidden units are detected as important units without saturation. In addition, a small number of important input-hidden connections are detected and the majority of the connections are eliminated. Thus, we can obtain much simplified internal representations with better interpretation and generalization. The D-entropy control method was applied to the inference of well-formedness of an artificial language. Experimental results confirmed that by maximizing and minimizing D-entropy, generalization performance can significantly be improved