论文标题
自动诱导的分销转移的隐藏激励措施
Hidden Incentives for Auto-Induced Distributional Shift
论文作者
论文摘要
机器学习系统做出的决定对世界的影响越来越大，但是机器学习算法通常不存在这种影响是很常见的。一个例子是使用I.I.D。内容建议中的假设。实际上，所显示的内容的（选择）可以改变用户的看法和偏好，甚至可以将其驱逐出去，从而导致用户分布的变化。我们介绍了术语自动诱导的分布移位（AD），以描述算法的现象，从而导致其自身输入的分布变化。我们的目标是确保机器学习系统不会利用广告来提高性能，这是不可取的。我们证明，学习算法的变化，例如引入元学习，可能会揭示自动诱导的分布转移（HI-ADS）的隐藏激励措施。为了解决这个问题，我们介绍了HI-ADS的“单元测试”和缓解策略，以及在内容建议中使用HI-ADS建模现实世界中问题的玩具环境，我们证明，强大的元学习者通过广告实现了绩效的收益。我们显示元学习和Q学习有时是失败的单位测试，但是在使用缓解策略时通过。
Decisions made by machine learning systems have increasing influence on the world, yet it is common for machine learning algorithms to assume that no such influence exists. An example is the use of the i.i.d. assumption in content recommendation. In fact, the (choice of) content displayed can change users' perceptions and preferences, or even drive them away, causing a shift in the distribution of users. We introduce the term auto-induced distributional shift (ADS) to describe the phenomenon of an algorithm causing a change in the distribution of its own inputs. Our goal is to ensure that machine learning systems do not leverage ADS to increase performance when doing so could be undesirable. We demonstrate that changes to the learning algorithm, such as the introduction of meta-learning, can cause hidden incentives for auto-induced distributional shift (HI-ADS) to be revealed. To address this issue, we introduce `unit tests' and a mitigation strategy for HI-ADS, as well as a toy environment for modelling real-world issues with HI-ADS in content recommendation, where we demonstrate that strong meta-learners achieve gains in performance via ADS. We show meta-learning and Q-learning both sometimes fail unit tests, but pass when using our mitigation strategy.
