论文标题
对流形的几何优化，并应用深度学习
Geometric Optimisation on Manifolds with Applications to Deep Learning
论文作者
论文摘要
我们设计并实施了一个Python库，以使用所有这些强大的工具来帮助非专家，以高效，可扩展且易于融合到数据科学家，从业者和应用研究人员的工作流程中。该库中实现的算法是设计了可用性和GPU效率的设计，并且可以将它们添加到只有一条额外代码的任何Pytorch模型中。 我们在时间序列分析的设置中展示了这些工具对歧管的应用的有效性。在这种情况下，正交和统一优化用于限制和正规化经常性模型，并避免消失和爆炸梯度问题。专为Geotorch设计的算法使我们能够在该模型家族的标准测试中实现最先进的状态。 我们使用比较几何形状中的工具来赋予优化问题中有意义的数量的界限。特别是，我们建立在（Kaul 1976）的工作基础上，以根据Riemannian指数的第二个衍生物的规范给出明确的界限。
We design and implement a Python library to help the non-expert using all these powerful tools in a way that is efficient, extensible, and simple to incorporate into the workflow of the data scientist, practitioner, and applied researcher. The algorithms implemented in this library have been designed with usability and GPU efficiency in mind, and they can be added to any PyTorch model with just one extra line of code. We showcase the effectiveness of these tools on an application of optimisation on manifolds in the setting of time series analysis. In this setting, orthogonal and unitary optimisation is used to constraint and regularise recurrent models and avoid vanishing and exploding gradient problems. The algorithms designed for GeoTorch allow us to achieve state of the art results in the standard tests for this family of models. We use tools from comparison geometry to give bounds on quantities that are of interest in optimisation problems. In particular, we build on the work of (Kaul 1976) to give explicit bounds on the norm of the second derivative of the Riemannian exponential.
