A gradient descending solution to the LASSO criteria

In this paper, we propose a new perspective to achieve sparseness via the winner-take-all principle for the linear kernel regression and classification tasks. We form the duality of the LASSO criteria, and transfer an ℓ₁ norm minimization to an ℓ_∞ norm maximization problem. We introduce a novel winner-take-all neural network solution derived from gradient descending, which links the sparse representation and the competitive learning scheme. This scheme is a form of unsupervised learning in which each input pattern comes through learning, to be associated with the activity of one or at most a few neurons. However, the lateral interaction between neurons in the same layer is strictly preemptive in this model. This framework is applicable to a variety of problems, such as independent component analysis (ICA), feature selection, and data clustering.