论文标题
Einsum Networks：快速，可扩展的概率电路学习
Einsum Networks: Fast and Scalable Learning of Tractable Probabilistic Circuits
论文作者
论文摘要
概率电路（PC）是概率建模的有前途的途径，因为它们允许多种精确有效的推理程序。 PC的最新``深度学习风格''实现了更好的可扩展性，但由于其稀疏连接的计算图，仍然很难在现实世界中进行训练。在本文中，我们提出了Einsum Networks（EINETS），这是一种针对PC的新型实施设计，在几个方面改善了先前的艺术。 Einets在其核心中将大量的算术操作结合在单个整体式einsum-operation中，从而与先前的实现相比，导致了最多两个数量级的加速和内存节省。作为算法贡献，我们表明，可以通过利用自动分化来简化预期最大化（EM）的实现。此外，我们证明EINET可以很好地扩展到以前无法触及的数据集，例如SVHN和Celeba，并且可以用作忠实的生成图像模型。
Probabilistic circuits (PCs) are a promising avenue for probabilistic modeling, as they permit a wide range of exact and efficient inference routines. Recent ``deep-learning-style'' implementations of PCs strive for a better scalability, but are still difficult to train on real-world data, due to their sparsely connected computational graphs. In this paper, we propose Einsum Networks (EiNets), a novel implementation design for PCs, improving prior art in several regards. At their core, EiNets combine a large number of arithmetic operations in a single monolithic einsum-operation, leading to speedups and memory savings of up to two orders of magnitude, in comparison to previous implementations. As an algorithmic contribution, we show that the implementation of Expectation-Maximization (EM) can be simplified for PCs, by leveraging automatic differentiation. Furthermore, we demonstrate that EiNets scale well to datasets which were previously out of reach, such as SVHN and CelebA, and that they can be used as faithful generative image models.
