论文标题
作为分类的回归：任务制定对神经网络特征的影响
Regression as Classification: Influence of Task Formulation on Neural Network Features
论文作者
论文摘要
可以通过使用基于梯度的方法最小化正方形损失来训练神经网络来解决回归问题。但是，从业人员通常希望将回归重新定义为分类问题，从而观察到跨熵损失的培训会导致更好的性能。通过专注于两层Relu网络，可以完全以其特征空间的措施来表征，我们探讨了如何通过基于梯度的优化引起的隐式偏见如何部分解释上述现象。我们提供了理论上的证据，表明回归公式产生的措施在一维数据的情况下，其支持与分类可能有很大差异。我们提出的最佳支持直接与网络输入层所学的功能相对应。这些支持的不同性质阐明了正方形损失在训练过程中可能遇到的可能优化困难，我们提出了证明这种现象的经验结果。
Neural networks can be trained to solve regression problems by using gradient-based methods to minimize the square loss. However, practitioners often prefer to reformulate regression as a classification problem, observing that training on the cross entropy loss results in better performance. By focusing on two-layer ReLU networks, which can be fully characterized by measures over their feature space, we explore how the implicit bias induced by gradient-based optimization could partly explain the above phenomenon. We provide theoretical evidence that the regression formulation yields a measure whose support can differ greatly from that for classification, in the case of one-dimensional data. Our proposed optimal supports correspond directly to the features learned by the input layer of the network. The different nature of these supports sheds light on possible optimization difficulties the square loss could encounter during training, and we present empirical results illustrating this phenomenon.
