DocumentCode :
3165711
Title :
Efficient Learning for Models with DAG-Structured Parameter Constraints
Author :
Zhong, Leon Wenliang ; Kwok, James T.
Author_Institution :
Dept. of Comput. Sci. & Eng., Hongkong Univ. of Sci. & Technol., Hong Kong, China
fYear :
2013
fDate :
7-10 Dec. 2013
Firstpage :
897
Lastpage :
906
Abstract :
In high-dimensional models, hierarchical and structural relationships among features are often used to constrain the search for the more important interactions. These relationships may come from prior knowledge or traditional design principles, such as that low-order effects should have larger contributions than higher-order ones and should be included into the model earlier. However, these structural constraints also make the optimization problem more challenging. In this paper, we propose the use of the alternating direction method of multipliers (ADMM) and accelerated gradient methods. In particular, we show that ADMM can be used to either directly solve the problem or serve as a key building block. Experimental results on a number of synthetic and real-world data sets demonstrate that the proposed algorithm is efficient and flexible. Moreover, the use of the hierarchical relationships consistently improves generalization performance and parameter estimation.
Keywords :
directed graphs; generalisation (artificial intelligence); gradient methods; learning (artificial intelligence); optimisation; parameter estimation; ADMM; DAG-structured parameter constraints; accelerated gradient methods; alternating direction method of multipliers; design principles; feature hierarchical relationships; feature structural relationships; generalization performance; high-dimensional models; key building block; learning; low-order effects; optimization problem; parameter estimation; structural constraints; Conferences; Data mining; Accelerated gradient methods; Alternating direction method of multipliers; Heredity; Structural sparsity;
fLanguage :
English
Publisher :
ieee
Conference_Titel :
Data Mining (ICDM), 2013 IEEE 13th International Conference on
Conference_Location :
Dallas, TX
ISSN :
1550-4786
Type :
conf
DOI :
10.1109/ICDM.2013.123
Filename :
6729574
Link To Document :
بازگشت