Entropy and negentropy (Edtl.)

Certainly there is a relationship between the mathematical theory of communication and statistical thermodynamics. It should be made clear without any doubt or hesitation that the term entropy is used in information theory by mathematical analogy. The expression X, which we call the entropy of a set of probabilities, is identical in form and in sign with the expression for entropy in physics. The units may differ, and the probabilities stand for different things, but neither of these can change the sign of X. The pure information theorist, of course, is left unmoved by arguments about mere names. In a mathematical theory, definitions are judged solely in the light of the theorems to which they lead, and word-names are arbitrary, even irrelevant. But when we apply the theory, or compare two different theories, it is surely important to avoid using one name for two different things or two opposite names for one and and the same thing.