论文标题
探路者：发现深神经网络中的决策途径
PathFinder: Discovering Decision Pathways in Deep Neural Networks
论文作者
论文摘要
解释性正在成为深层神经网络的越来越重要的主题。尽管卷积层中的操作更容易理解，但在完全连接的层中处理变得不透明。我们工作中的基本思想是，每个实例都流过各层时，都会在隐藏的图层中以及我们的路径方法中引起不同的激活模式，我们将这些激活向量聚集到每个隐藏层的这些激活向量，然后查看连续图层中的簇如何相互连接，因为激活从输入层流向输出。 We find that instances of the same class follow a small number of cluster sequences over the layers, which we name ``decision paths." Such paths explain how classification decisions are typically made, and also help us determine outliers that follow unusual paths. We also propose using the Sankey diagram to visualize such pathways. We validate our method with experiments on two feed-forward networks trained on MNIST and CELEB data sets, and one recurrent network接受了吊坠训练。
Explainability is becoming an increasingly important topic for deep neural networks. Though the operation in convolutional layers is easier to understand, processing becomes opaque in fully-connected layers. The basic idea in our work is that each instance, as it flows through the layers, causes a different activation pattern in the hidden layers and in our Paths methodology, we cluster these activation vectors for each hidden layer and then see how the clusters in successive layers connect to one another as activation flows from the input layer to the output. We find that instances of the same class follow a small number of cluster sequences over the layers, which we name ``decision paths." Such paths explain how classification decisions are typically made, and also help us determine outliers that follow unusual paths. We also propose using the Sankey diagram to visualize such pathways. We validate our method with experiments on two feed-forward networks trained on MNIST and CELEB data sets, and one recurrent network trained on PenDigits.
