论文标题
Funnscope：用于交互式探索完全连接神经网络的损失格局的视觉显微镜
FuNNscope: Visual microscope for interactively exploring the loss landscape of fully connected neural networks
论文作者
论文摘要
尽管它们在各个领域有效地使用，但神经网络的许多方面知之甚少。研究神经网络特征的一种重要方法是探索损失格局。但是，大多数模型都会产生高维的非凸面景观，这很难可视化。我们通过使用具有可解释轴的图表来讨论并扩展基于1D和2D切片的现有可视化方法，该方法近似于实际损失景观几何形状。基于这样的假设，即对小神经网络的观察可以推广到更复杂的系统并为我们提供有用的见解，我们专注于几十个重量范围内的小型模型，这可以实现计算廉价的实验和交互式仪表板的使用。我们观察到零矢量周围的对称性，不同层对全球景观的影响，最小化器周围的不同重量敏感性以及梯度下降如何导航高损失障碍。用户研究导致平均SUS（系统可用性量表）得分，并提出了改进的建议，并打开了许多可能的应用程序方案，例如自动编码器和集合网络。
Despite their effective use in various fields, many aspects of neural networks are poorly understood. One important way to investigate the characteristics of neural networks is to explore the loss landscape. However, most models produce a high-dimensional non-convex landscape which is difficult to visualize. We discuss and extend existing visualization methods based on 1D- and 2D slicing with a novel method that approximates the actual loss landscape geometry by using charts with interpretable axes. Based on the assumption that observations on small neural networks can generalize to more complex systems and provide us with helpful insights, we focus on small models in the range of a few dozen weights, which enables computationally cheap experiments and the use of an interactive dashboard. We observe symmetries around the zero vector, the influence of different layers on the global landscape, the different weight sensitivities around a minimizer, and how gradient descent navigates high-loss obstacles. The user study resulted in an average SUS (System Usability Scale) score with suggestions for improvement and opened up a number of possible application scenarios, such as autoencoders and ensemble networks.
