论文标题
代表学习的自由能原则
A Free-Energy Principle for Representation Learning
论文作者
论文摘要
本文采用机器学习与热力学的正式联系来表征传递学习的学说质量。我们讨论信息理论功能（例如速率，失真和分类损失）如何位于凸，所谓的平衡表面上。我们规定了在约束下遍历该表面的动力学过程，例如，ISO分类过程，一个较低的速度和失真，以使分类损失损失无链。我们演示了如何将此过程用于将表示图从源数据集传输到目标数据集，同时保持分类损失恒定。在标准图像分类数据集上提供了理论结果的实验验证。
This paper employs a formal connection of machine learning with thermodynamics to characterize the quality of learnt representations for transfer learning. We discuss how information-theoretic functional such as rate, distortion and classification loss of a model lie on a convex, so-called equilibrium surface.We prescribe dynamical processes to traverse this surface under constraints, e.g., an iso-classification process that trades off rate and distortion to keep the classification loss unchanged. We demonstrate how this process can be used for transferring representations from a source dataset to a target dataset while keeping the classification loss constant. Experimental validation of the theoretical results is provided on standard image-classification datasets.
