Learning in deep architectures with folding transformations

We propose a folding transformation paradigm for supervised layer-wise learning in deep neural networks by introducing concepts of internal decision making, mapping and shatter complexity. These concepts aid in the analysis of an individual hidden transformation in a deep architecture and help to map the capabilities of the proposed folding transformations. We justify the increase of VC-dimension due to depth by showing that the extra model complexity is needed to resolve large variability in the input data for complex problems. We provide an implementation and test the architecture´s performance on a classification task.