DocumentCode :
2772288
Title :
Accelerated Gradient Method for Multi-task Sparse Learning Problem
Author :
Chen, Xi ; Pan, Weike ; Kwok, James T. ; Carbonell, Jaime G.
Author_Institution :
Sch. of Comput. Sci., Carnegie Mellon Univ. Pittsburgh, Pittsburgh, PA, USA
fYear :
2009
fDate :
6-9 Dec. 2009
Firstpage :
746
Lastpage :
751
Abstract :
Many real world learning problems can be recast as multi-task learning problems which utilize correlations among different tasks to obtain better generalization performance than learning each task individually. The feature selection problem in multi-task setting has many applications in fields of computer vision, text classification and bio-informatics. Generally, it can be realized by solving a L-1-infinity regularized optimization problem. And the solution automatically yields the joint sparsity among different tasks. However, due to the nonsmooth nature of the L-1-infinity norm, there lacks an efficient training algorithm for solving such problem with general convex loss functions. In this paper, we propose an accelerated gradient method based on an ``optimal´´ first order black-box method named after Nesterov and provide the convergence rate for smooth convex loss functions. For nonsmooth convex loss functions, such as hinge loss, our method still has fast convergence rate empirically. Moreover, by exploiting the structure of the L-1-infinity ball, we solve the black-box oracle in Nesterov´s method by a simple sorting scheme. Our method is suitable for large-scale multi-task learning problem since it only utilizes the first order information and is very easy to implement. Experimental results show that our method significantly outperforms the most state-of-the-art methods in both convergence speed and learning accuracy.
Keywords :
gradient methods; learning (artificial intelligence); L-1-infinity regularized optimization problem; Nesterov method; accelerated gradient method; bioinformatics; computer vision; feature selection problem; first order black-box method; multitask sparse learning problem; optimal first order black-box method; text classification; Acceleration; Application software; Computer science; Convergence; Data engineering; Data mining; Gradient methods; Large-scale systems; Machine learning; Optimization methods; L-1-infinity regularization; gradient descend; multi-task learning; optimal method;
fLanguage :
English
Publisher :
ieee
Conference_Titel :
Data Mining, 2009. ICDM '09. Ninth IEEE International Conference on
Conference_Location :
Miami, FL
ISSN :
1550-4786
Print_ISBN :
978-1-4244-5242-2
Electronic_ISBN :
1550-4786
Type :
conf
DOI :
10.1109/ICDM.2009.128
Filename :
5360305
Link To Document :
بازگشت