Statistics and Its Interface

Volume 12 (2019)

Number 4

Nonnegative hierarchical lasso with a mixed $(1, \frac{1}{2})$-penalty and a fast solver

Pages: 599 – 615

DOI: https://dx.doi.org/10.4310/SII.2019.v12.n4.a9

Authors

Wanling Xie (College of Mathematics and Statistics, Chongqing University, Chongqing, China)

Hu Yang (College of Mathematics and Statistics, Chongqing University, Chongqing, China)

Abstract

Grouping structures arise naturally in many high dimensional statistical problems. Incorporation of grouping information can efficiently improve the statistical accuracy and model interpretability. In addition, nonnegative constraints are essential to cope with index tracking problems. This paper proposes the nonnegative hierarchical lasso with nonnegative constraints on the coefficients both in low dimensional setting and ultra high dimensional setting, which is capable of simultaneous selection at both the group and withingroup levels with overlap, namely the bi-level selection.

In theoretical analysis, we show the nonnegative hierarchical lasso enjoys the oracle properties in group selection when the number of covariates diverges with the sample size under certain regularity conditions. Since there are less works devoted to the theoretical properties of bi-level selection methods in cases where the number of variables or groups is much larger than the sample size, we also derive the oracle inequalities for the prediction and $l_1$ estimation errors of the estimator under the restricted eigenvalue conditions on the design matrix. It is shown to have group selection consistency and estimation consistency in ultra high-dimensional sparse linear regression models. To get the solution of the nonnegative hierarchical lasso, we propose a fast and efficient iterative half thresholding-based local linear approximation algorithm (IHT-LLA) for solving. Simulations indicate that the nonnegative hierarchical lasso outperforms other nonnegative regularization methods and is robust against possible mis-specified grouping structure. Besides, we further apply our method to the index tracking problems.

Keywords

nonnegative hierarchical lasso, Oracle property, bi-level selection, index tracking, Oracle inequality

Full Text (PDF format)

Received 14 January 2019

Published 18 July 2019