论文标题
深入研究标签平滑
Delving Deep into Label Smoothing
论文作者
论文摘要
标签平滑是一种有效的深神经网络（DNN）的有效正则化工具，它通过在均匀分布和硬标签之间应用加权平均值来生成软标签。它通常用于减少培训DNN的过度拟合问题，并进一步提高分类性能。在本文中，我们旨在调查如何生成更可靠的软标签。我们提出了一种在线标签平滑（OLS）策略，该策略根据目标类别的模型预测的统计数据生成软标签。拟议的OLS构建了目标类别和非目标类别之间的更合理的概率分布，以监督DNN。实验表明，基于相同的分类模型，所提出的方法可以有效地改善CIFAR-100，ImageNet和细粒数据集的分类性能。此外，与当前标签平滑方法相比，提出的方法可以显着提高DNN模型对嘈杂标签的鲁棒性。
Label smoothing is an effective regularization tool for deep neural networks (DNNs), which generates soft labels by applying a weighted average between the uniform distribution and the hard label. It is often used to reduce the overfitting problem of training DNNs and further improve classification performance. In this paper, we aim to investigate how to generate more reliable soft labels. We present an Online Label Smoothing (OLS) strategy, which generates soft labels based on the statistics of the model prediction for the target category. The proposed OLS constructs a more reasonable probability distribution between the target categories and non-target categories to supervise DNNs. Experiments demonstrate that based on the same classification models, the proposed approach can effectively improve the classification performance on CIFAR-100, ImageNet, and fine-grained datasets. Additionally, the proposed method can significantly improve the robustness of DNN models to noisy labels compared to current label smoothing approaches.
