DocumentCode :
286748
Title :
A Hebbian/anti-Hebbian network which optimizes information capacity by orthonormalizing the principal subspace
Author :
Plumbley, M.D.
Author_Institution :
King´´s Coll., London, UK
fYear :
1993
fDate :
25-27 May 1993
Firstpage :
86
Lastpage :
90
Abstract :
A number of recent papers have used the approach of maximising information capacity or mutual information (MI) to examine unsupervised neural networks. In particular, for a linear `compressing´ N-input M-output network (N>M), with noise on the input only, the author (1991) maximised MI when the output represents the principal subspace (or top M principal components) of the input. On the other hand, for a linear `straight-through´ M-input M-output network with noise on the output only, he maximised MI (for a fixed output power) when the outputs are orthonormalised, i.e. decorrelated and of equal variance. A number of algorithms exist to achieve both of these optimal arrangements. In this paper, the author extends this work to develop an algorithm for the case of both input and output noise, with an output power constraint. He finds that it is possible to simplify the obvious algorithm obtained by concatenating the two previous solutions
Keywords :
information theory; neural nets; optimisation; Hebbian network; M-input M-output network; N-input M-output network; anti-Hebbian network; noise; output power constraint; principal subspace orthonormalisation; unsupervised neural networks;
fLanguage :
English
Publisher :
iet
Conference_Titel :
Artificial Neural Networks, 1993., Third International Conference on
Conference_Location :
Brighton
Print_ISBN :
0-85296-573-7
Type :
conf
Filename :
263251
Link To Document :
بازگشت