论文标题
迷宫的强化学习中没有准确的裂缝性权衡
There is no Accuracy-Interpretability Tradeoff in Reinforcement Learning for Mazes
论文作者
论文摘要
可解释性是强化学习系统中可信度的重要组成部分。但是，可解释性可能以绩效恶化的成本为代价，导致许多研究人员建立复杂的模型。我们的目标是分析可解释性的成本。我们表明，在某些情况下，人们可以在保持其最优性的同时实现政策可解释性。我们专注于从增强学习中的经典问题：$ \ mathbb {r}^d $中的$ k $障碍物的迷宫。我们证明了一个小型决策树的存在，在每个内部节点和深度$ o（\ log k + 2^d）$上具有线性函数，代表最佳策略。请注意，对于不断$ d $的有趣情况，我们有$ o（\ log k）$ depth。因此，在这种情况下，没有准确的截止性权衡。为了证明这一结果，我们使用了一种新的“压缩”技术，该技术可能在其他设置中有用。
Interpretability is an essential building block for trustworthiness in reinforcement learning systems. However, interpretability might come at the cost of deteriorated performance, leading many researchers to build complex models. Our goal is to analyze the cost of interpretability. We show that in certain cases, one can achieve policy interpretability while maintaining its optimality. We focus on a classical problem from reinforcement learning: mazes with $k$ obstacles in $\mathbb{R}^d$. We prove the existence of a small decision tree with a linear function at each inner node and depth $O(\log k + 2^d)$ that represents an optimal policy. Note that for the interesting case of a constant $d$, we have $O(\log k)$ depth. Thus, in this setting, there is no accuracy-interpretability tradeoff. To prove this result, we use a new "compressing" technique that might be useful in additional settings.
