论文标题
高斯矩阵以外的凸面惩罚线性回归的渐近错误
Asymptotic errors for convex penalized linear regression beyond Gaussian matrices
论文作者
论文摘要
我们考虑了从噪声线性观测值$ r^{n} $中学习系数向量$ x_ {0} $的问题我们为通过受较大的凸回归估计值（例如Lasso或Elastic Net）获得的渐近平均平方误差（例如使用统计物理学的启发式方法）提供了一个明确的公式（首先是使用统计物理学的启发式方法）的严格推导。该证明基于对向量近似消息通话（Oracle-vamp）的甲骨文版本的收敛分析以及其状态演化方程的属性。我们的方法利用并突出显示了向量近似消息，道格拉斯 - 拉赫福德分裂和近端血统算法之间的联系，从而扩大了使用i.i.d.的先前结果。大量问题的矩阵。我们在一些具体的例子上说明了我们的结果，并表明即使它们是渐进的，我们的预测即使对于非常适中的数字也非常吻合。
We consider the problem of learning a coefficient vector $x_{0}$ in $R^{N}$ from noisy linear observations $y=Fx_{0}+w$ in $R^{M}$ in the high dimensional limit $M,N$ to infinity with $α=M/N$ fixed. We provide a rigorous derivation of an explicit formula -- first conjectured using heuristic methods from statistical physics -- for the asymptotic mean squared error obtained by penalized convex regression estimators such as the LASSO or the elastic net, for a class of very generic random matrices corresponding to rotationally invariant data matrices with arbitrary spectrum. The proof is based on a convergence analysis of an oracle version of vector approximate message-passing (oracle-VAMP) and on the properties of its state evolution equations. Our method leverages on and highlights the link between vector approximate message-passing, Douglas-Rachford splitting and proximal descent algorithms, extending previous results obtained with i.i.d. matrices for a large class of problems. We illustrate our results on some concrete examples and show that even though they are asymptotic, our predictions agree remarkably well with numerics even for very moderate sizes.
