论文标题
盲目的知识蒸馏可用于强大的图像分类
Blind Knowledge Distillation for Robust Image Classification
论文作者
论文摘要
用嘈杂的标签优化神经网络是一项具有挑战性的任务，尤其是如果标签集包含现实世界噪声。网络倾向于在早期训练阶段概括为合理的模式，并过度融合了后者样本中嘈杂样本的特定细节。我们介绍了盲目的知识蒸馏 - 一种新颖的教师研究方法，通过掩盖了与地面真理相关的教师的产出，以滤除潜在的损坏的知识，并估算从概括到过度拟合的临界点，从而用嘈杂的标签进行学习。基于此，我们可以通过OTSUS算法对训练数据中的噪声进行估计。通过此估计，我们通过修改的加权跨透明损失函数来训练网络。我们在实验中表明，盲目的知识蒸馏在训练过程中有效地检测过度拟合，并改善了最近发表的CIFAR-N数据集上清洁和嘈杂标签的检测。代码可在GitHub上找到。
Optimizing neural networks with noisy labels is a challenging task, especially if the label set contains real-world noise. Networks tend to generalize to reasonable patterns in the early training stages and overfit to specific details of noisy samples in the latter ones. We introduce Blind Knowledge Distillation - a novel teacher-student approach for learning with noisy labels by masking the ground truth related teacher output to filter out potentially corrupted knowledge and to estimate the tipping point from generalizing to overfitting. Based on this, we enable the estimation of noise in the training data with Otsus algorithm. With this estimation, we train the network with a modified weighted cross-entropy loss function. We show in our experiments that Blind Knowledge Distillation detects overfitting effectively during training and improves the detection of clean and noisy labels on the recently published CIFAR-N dataset. Code is available at GitHub.
