论文标题
平滑参数化对非凸优化景观的影响
The effect of smooth parametrizations on nonconvex optimization landscapes
论文作者
论文摘要
我们开发了研究非凸优化的景观的新工具。给定一个优化问题，我们通过平滑参数域将其与另一个优化问题配对。这是出于实际目的（例如，使用具有良好保证的平滑优化算法），或用于理论目的（例如，揭示景观满足严格的鞍属性）。在这两种情况下，主要问题是：这两个问题的景观如何相关？更准确地说：一个问题中的局部最小值和关键点之类的理想点与另一个问题中的问题有何关系？本文的一个关键发现是这些关系通常取决于参数化本身，几乎完全独立于成本函数。因此，我们引入了一个通用框架来研究参数对景观的影响。该框架使我们能够为一系列问题获得新的保证，其中一些以前是在文献中逐案对待的。应用包括：通过因素化优化低级矩阵和张量；通过Burer-Monteiro方法解决半决赛计划；通过优化其体重和偏见来训练神经网络；并商定对称性。
We develop new tools to study landscapes in nonconvex optimization. Given one optimization problem, we pair it with another by smoothly parametrizing the domain. This is either for practical purposes (e.g., to use smooth optimization algorithms with good guarantees) or for theoretical purposes (e.g., to reveal that the landscape satisfies a strict saddle property). In both cases, the central question is: how do the landscapes of the two problems relate? More precisely: how do desirable points such as local minima and critical points in one problem relate to those in the other problem? A key finding in this paper is that these relations are often determined by the parametrization itself, and are almost entirely independent of the cost function. Accordingly, we introduce a general framework to study parametrizations by their effect on landscapes. The framework enables us to obtain new guarantees for an array of problems, some of which were previously treated on a case-by-case basis in the literature. Applications include: optimizing low-rank matrices and tensors through factorizations; solving semidefinite programs via the Burer-Monteiro approach; training neural networks by optimizing their weights and biases; and quotienting out symmetries.
