论文标题
随机特征回归模型的最佳激活函数
Optimal Activation Functions for the Random Features Regression Model
论文作者
论文摘要
最近已经研究了随机特征回归模型（RFR）的渐近平方平方测试误差和灵敏度。我们以这项工作为基础，并在封闭形式中识别激活函数家族（AFS），从而最大程度地减少了在功能性parsimony的不同概念下，RFR的测试误差和灵敏度的组合。我们发现最佳AFS是线性，饱和线性函数或以Hermite多项式表达的情况。最后，我们展示了使用最佳AFS如何影响RFR模型的良好属性，例如其双重下降曲线，以及其最佳正则化参数对观测噪声水平的依赖性。
The asymptotic mean squared test error and sensitivity of the Random Features Regression model (RFR) have been recently studied. We build on this work and identify in closed-form the family of Activation Functions (AFs) that minimize a combination of the test error and sensitivity of the RFR under different notions of functional parsimony. We find scenarios under which the optimal AFs are linear, saturated linear functions, or expressible in terms of Hermite polynomials. Finally, we show how using optimal AFs impacts well-established properties of the RFR model, such as its double descent curve, and the dependency of its optimal regularization parameter on the observation noise level.
