Linear independence of internal representations in multilayer perceptrons

Author

Shah, Jagesh V. ; Poon, Chi-Sang

Author_Institution

Div. of Health Sci. & Technol., MIT, Cambridge, MA, USA

Volume

10

Issue

1

fYear

1999

fDate

1/1/1999 12:00:00 AM

Firstpage

10

Lastpage

18

Abstract

Identifies the linear independence of the internal representation of the multilayer perceptron as an essential property for exact learning. The sigmoidal hidden unit activation function has the ability to produce linearly independent outputs. As a result, the minimum number of hidden units for a set of specified input is the number of patterns less the rank of the input patterns. In addition, the basis of many training algorithms is shown to inherently increase the number of linearly independent vectors in the internal representations, thereby increasing the likelihood of exact learning

Keywords

learning (artificial intelligence); multilayer perceptrons; transfer functions; exact learning; internal representations; linear independence; multilayer perceptrons; sigmoidal hidden unit activation function; Adaptive optics; Artificial neural networks; Biomedical optical imaging; Character recognition; Feature extraction; Multilayer perceptrons; Optical character recognition software; Optical computing; Optical network units; Pattern recognition;

fLanguage

English

Journal_Title

Neural Networks, IEEE Transactions on

Publisher

ieee

ISSN

1045-9227

Type

jour

DOI

10.1109/72.737489

Filename

737489