论文标题
一种随机亚级别方法，用于分布鲁棒的非凸学习
A Stochastic Subgradient Method for Distributionally Robust Non-Convex Learning
论文作者
论文摘要
我们考虑了在统计学习中出现的随机优化问题的分布鲁棒公式，其中鲁棒性与基础数据分布的不确定性相关。我们的配方基于规避风险优化技术和相干风险措施的理论。它使用半差异风险来量化不确定性，从而使我们能够计算对人口数据分布中扰动的强大解决方案。我们考虑了一个可以是非凸面和非平滑的损失函数，并开发出有效的随机亚级别方法。我们证明它会收敛到满足最佳条件的点。据我们所知，这是第一种具有严格合并的方法，可以在非凸线非平滑分布的背景下保证了稳健的随机优化。与人口风险最小化相比，我们的方法可以达到任何理想的鲁棒性，几乎没有额外的计算成本。我们还说明了在凸和非凸的监督学习问题中产生的真实数据集上的算法的性能。
We consider a distributionally robust formulation of stochastic optimization problems arising in statistical learning, where robustness is with respect to uncertainty in the underlying data distribution. Our formulation builds on risk-averse optimization techniques and the theory of coherent risk measures. It uses semi-deviation risk for quantifying uncertainty, allowing us to compute solutions that are robust against perturbations in the population data distribution. We consider a large family of loss functions that can be non-convex and non-smooth and develop an efficient stochastic subgradient method. We prove that it converges to a point satisfying the optimality conditions. To our knowledge, this is the first method with rigorous convergence guarantees in the context of non-convex non-smooth distributionally robust stochastic optimization. Our method can achieve any desired level of robustness with little extra computational cost compared to population risk minimization. We also illustrate the performance of our algorithm on real datasets arising in convex and non-convex supervised learning problems.
