Robust nonparametric regression by controlling sparsity*

Nonparametric methods are widely applicable to statistical learning problems, since they rely on a few modeling assumptions. In this context, the fresh look advocated here permeates benefits from variable selection and compressive sampling, to robustify nonparametric regression against outliers. A variational counterpart to least-trimmed squares regression is shown closely related to an ℓ0-(pseudo)norm-regularized estimator, that encourages sparsity in a vector explicitly modeling the outliers. This connection suggests efficient (approximate) solvers based on convex relaxation, which lead naturally to a variational M-type estimator equivalent to Lasso. Outliers are identified by judiciously tuning regularization parameters, which amounts to controlling the sparsity of the outlier vector along the whole robustification path of Lasso solutions. An improved estimator with reduced bias is obtained after replacing the ℓ0-(pseudo)norm with a nonconvex surrogate, as corroborated via simulated tests on robust thin-plate smoothing splines.