论文标题
一个用于统一主动学习问题的信息理论框架
An Information-Theoretic Framework for Unifying Active Learning Problems
论文作者
论文摘要
本文提出了一个信息理论框架，用于统一主动学习问题：水平集估计（LSE），贝叶斯优化（BO）及其广义变体。我们首先引入了一个新颖的主动学习标准，该标准涵盖了现有的LSE算法，并在连续输入域中实现了LSE问题中最先进的性能。然后，通过利用LSE和BO之间的关系，我们为BO设计了一个竞争性信息理论获取函数，该功能与上限置信度绑定和最大值熵搜索（MES）具有有趣的连接。后一种连接揭示了MES的缺点，这不仅对ME，而且对其他基于MES的采集功能具有重要意义。最后，我们的统一信息理论框架可以应用于以数据有效的方式涉及多个级别集的LSE和BO的广义问题。我们通过综合基准功能，现实世界数据集以及机器学习模型的高参数调整来评估提出的算法的性能。
This paper presents an information-theoretic framework for unifying active learning problems: level set estimation (LSE), Bayesian optimization (BO), and their generalized variant. We first introduce a novel active learning criterion that subsumes an existing LSE algorithm and achieves state-of-the-art performance in LSE problems with a continuous input domain. Then, by exploiting the relationship between LSE and BO, we design a competitive information-theoretic acquisition function for BO that has interesting connections to upper confidence bound and max-value entropy search (MES). The latter connection reveals a drawback of MES which has important implications on not only MES but also on other MES-based acquisition functions. Finally, our unifying information-theoretic framework can be applied to solve a generalized problem of LSE and BO involving multiple level sets in a data-efficient manner. We empirically evaluate the performance of our proposed algorithms using synthetic benchmark functions, a real-world dataset, and in hyperparameter tuning of machine learning models.
