Title :
Deep and Shallow Architecture of Multilayer Neural Networks
Author_Institution :
Dept. of Appl. Math., Nat. Univ. of Kaohsiung, Kaohsiung, Taiwan
Abstract :
This paper focuses on the deep and shallow architecture of multilayer neural networks (MNNs). The demonstration of whether or not an MNN can be replaced by another MNN with fewer layers is equivalent to studying the topological conjugacy of its hidden layers. This paper provides a systematic methodology to indicate when two hidden spaces are topologically conjugated. Furthermore, some criteria are presented for some specific cases.
Keywords :
multilayer perceptrons; neural net architecture; topology; MNN; deep-shallow architecture; hidden layers; multilayer neural networks; topologically conjugated hidden spaces; Artificial neural networks; Biological neural networks; Entropy; Mathematical model; Matrix decomposition; Multi-layer neural network; Nonhomogeneous media; Deep architecture; factor-like matrix; multilayer neural networks (MNNs); sofic shift; topological entropy; topological entropy.;
Journal_Title :
Neural Networks and Learning Systems, IEEE Transactions on
DOI :
10.1109/TNNLS.2014.2387439
