论文标题
改善卷积神经网络的输入掩蔽
Towards Improved Input Masking for Convolutional Neural Networks
论文作者
论文摘要
从机器学习模型输入中删除功能的能力对于理解和解释模型预测非常重要。但是，这对于视觉模型来说是不平凡的，因为掩盖了输入图像的一部分通常会导致较大的分布变化。这是因为用于掩盖的基线颜色（通常为灰色或黑色）是不可分割的。此外，掩模本身的形状可能包含不必要的信号，该信号可以被模型用于其预测。最近，在视觉变压器的图像掩盖中缓解此问题（称为丢失偏见）方面取得了一些进展。在这项工作中，我们为CNN提出了一种新的掩蔽方法，我们称之为层掩蔽，其中掩盖引起的偏见在很大程度上减少了。直观地，层掩蔽将掩码应用于中间激活图，以便模型仅处理未掩盖的输入。我们表明，我们的方法（i）能够消除或最小化掩模形状或颜色对模型输出的影响，并且（ii）要比用黑色或灰色代替掩盖的区域，以获取基于输入扰动的可解释性技术，例如石灰。因此，与其他掩盖策略相比，层掩蔽受到遗失偏差的影响要小得多。我们还展示了掩码的形状如何泄漏有关类的信息，从而影响了模型依赖于输入蒙版的相关特征的模型依赖。此外，我们讨论了数据增强技术在解决此问题方面的作用，并认为它们不足以防止模型依赖掩盖形状。该项目的代码可在https://github.com/sriramb-98/layer_masking上公开获取
The ability to remove features from the input of machine learning models is very important to understand and interpret model predictions. However, this is non-trivial for vision models since masking out parts of the input image typically causes large distribution shifts. This is because the baseline color used for masking (typically grey or black) is out of distribution. Furthermore, the shape of the mask itself can contain unwanted signals which can be used by the model for its predictions. Recently, there has been some progress in mitigating this issue (called missingness bias) in image masking for vision transformers. In this work, we propose a new masking method for CNNs we call layer masking in which the missingness bias caused by masking is reduced to a large extent. Intuitively, layer masking applies a mask to intermediate activation maps so that the model only processes the unmasked input. We show that our method (i) is able to eliminate or minimize the influence of the mask shape or color on the output of the model, and (ii) is much better than replacing the masked region by black or grey for input perturbation based interpretability techniques like LIME. Thus, layer masking is much less affected by missingness bias than other masking strategies. We also demonstrate how the shape of the mask may leak information about the class, thus affecting estimates of model reliance on class-relevant features derived from input masking. Furthermore, we discuss the role of data augmentation techniques for tackling this problem, and argue that they are not sufficient for preventing model reliance on mask shape. The code for this project is publicly available at https://github.com/SriramB-98/layer_masking
