论文标题
使用汇总输入显着性了解文本分类数据和模型
Understanding Text Classification Data and Models Using Aggregated Input Salience
论文作者
论文摘要
意识到何时出于错误的理由是正确的，这并不是微不足道的，并且需要模型开发人员的巨大努力。在某些情况下，一种输入显着性方法突出了输入中最重要的部分，可能会揭示出有问题的推理。但是，在许多数据实例上仔细审查亮点是乏味的，而且通常是不可行的。此外，分析示例隔离并不能揭示数据或模型行为中的一般模式。在本文中，我们旨在解决这些问题，并从了解单个示例到了解整个数据集和模型。我们建议的方法基于汇总显着图，我们应用了聚类，最近的邻居搜索和可视化。使用此方法，我们通过显示如何确定和解释有问题的数据和模型行为来解决多个不同但常见的模型开发人员的需求，这是改进模型的必要第一步。
Realizing when a model is right for a wrong reason is not trivial and requires a significant effort by model developers. In some cases an input salience method, which highlights the most important parts of the input, may reveal problematic reasoning. But scrutinizing highlights over many data instances is tedious and often infeasible. Furthermore, analyzing examples in isolation does not reveal general patterns in the data or in the model's behavior. In this paper we aim to address these issues and go from understanding single examples to understanding entire datasets and models. The methodology we propose is based on aggregated salience maps, to which we apply clustering, nearest neighbor search and visualizations. Using this methodology we address multiple distinct but common model developer needs by showing how problematic data and model behavior can be identified and explained -- a necessary first step for improving the model.
