论文标题
通过复合优化学习分布稳健的模型
Learning Distributionally Robust Models at Scale via Composite Optimization
论文作者
论文摘要
训练机器学习模型可在数据中进行分配变化，因此，分布强劲的优化（DRO）已被证明非常有效。但是，现有的学习分布鲁棒模型的方法需要解决复杂的优化问题，例如半决赛编程或一阶方法，其收敛性与数据样本的数量线性缩放，这阻碍了它们对大型数据集的可扩展性。在本文中，我们展示了DRO的不同变体如何仅仅是有限和复合优化的实例，我们为其提供可扩展的方法。我们还提供了经验结果，以证明我们提出的算法在先前的艺术方面的有效性，以便从非常大的数据集中学习强大的模型。
To train machine learning models that are robust to distribution shifts in the data, distributionally robust optimization (DRO) has been proven very effective. However, the existing approaches to learning a distributionally robust model either require solving complex optimization problems such as semidefinite programming or a first-order method whose convergence scales linearly with the number of data samples -- which hinders their scalability to large datasets. In this paper, we show how different variants of DRO are simply instances of a finite-sum composite optimization for which we provide scalable methods. We also provide empirical results that demonstrate the effectiveness of our proposed algorithm with respect to the prior art in order to learn robust models from very large datasets.
