论文标题
$ l_q $正则化的相变和高阶分析在依赖
Phase transition and higher order analysis of $L_q$ regularization under dependence
论文作者
论文摘要
我们研究了估计$ k $ -sparse信号$ {\ mbox {\ mbox {$β$}} _ 0 \ in {\ bf r} x} {\ mbox {$β$}}}+{\ bf w} $，其中$ {\ bf x} \ in {\ bf r}^{n \ times p} $是测量矩阵，这些行是从分配中绘制的，是$ n（0，{\ mbox {\ mbox {$ bbox {$} $}）。我们考虑$ l_q $ regularized最小二乘（lqls）由配方$ \ hat {\ mbox {\ mbox {$β$}}}（λ，q）= \ text {argmin} _ {\ mbox {\ mbox {\ mbox {$β$} \ in {\ bf r}^p} \ frac {1} {2} \ | {\ bf y} - {\ bf x} {\ mbox {\ mbox {$β$}}} \ |^2_2+λ\ \ | $（0 \ le q \ le 2）$表示$ l_q $ -norm。在设置$ p，n，k \ rightarrow \ Infty $，带固定$ k/p =ε$和$ n/p =δ$，我们得出了$ \ hat {\ mbox {\ mbox {$β$}}（λ，q）的渐近风险，用于任意共汇率$ {$ {$ {$ {$ {$ {即$ x_ {ij} \ Overset {i.i.d} {\ sim} n（0,1）$。我们对LQL进行了高阶分析，在小问题方面，可以使用第一个主要项来确定LQL的相变行为。我们的结果表明，在情况下，第一个主要项不取决于$ {\ mbox {$σ$}} $的协方差结构。为了研究$ {\ mbox {$σ$}} $的协方差结构对LQLS性能的影响，在此情况下，$ 0 \ le q <1 $和$ 1 <q \ le 2 $，我们在不误差的差异方面得出了第二个主导期限的明确公式。广泛的计算实验证实，我们的分析预测与数值结果一致。
We study the problem of estimating a $k$-sparse signal ${\mbox{$β$}}_0\in{\bf R}^p$ from a set of noisy observations ${\bf y}\in{\bf R}^n$ under the model ${\bf y}={\bf X}{\mbox{$β$}}+{\bf w}$, where ${\bf X}\in{\bf R}^{n\times p}$ is the measurement matrix the row of which is drawn from distribution $N(0,{\mbox{$Σ$}})$. We consider the class of $L_q$-regularized least squares (LQLS) given by the formulation $\hat{\mbox{$β$}}(λ,q)=\text{argmin}_{\mbox{$β$}\in{\bf R}^p}\frac{1}{2}\|{\bf y}-{\bf X}{\mbox{$β$}}\|^2_2+λ\|{\mbox{$β$}}\|_q^q$, where $\|\cdot\|_q$ $(0\le q\le 2)$ denotes the $L_q$-norm. In the setting $p,n,k\rightarrow\infty$ with fixed $k/p=ε$ and $n/p=δ$, we derive the asymptotic risk of $\hat{\mbox{$β$}}(λ,q)$ for arbitrary covariance matrix ${\mbox{$Σ$}}$ which generalizes the existing results for standard Gaussian design, i.e. $X_{ij}\overset{i.i.d}{\sim}N(0,1)$. We perform a higher-order analysis for LQLS in the small-error regime in which the first dominant term can be used to determine the phase transition behavior of LQLS. Our results show that the first dominant term does not depend on the covariance structure of ${\mbox{$Σ$}}$ in the cases $0\le q< 1$ and $1< q\le 2$ which indicates that the correlations among predictors only affect the phase transition curve in the case $q=1$ a.k.a. LASSO. To study the influence of the covariance structure of ${\mbox{$Σ$}}$ on the performance of LQLS in the cases $0\le q< 1$ and $1<q\le 2$, we derive the explicit formulas for the second dominant term in the expansion of the asymptotic risk in terms of small error. Extensive computational experiments confirm that our analytical predictions are consistent with numerical results.
