论文标题
使用二元神经网络降低深层生成模型的计算成本
Reducing the Computational Cost of Deep Generative Models with Binary Neural Networks
论文作者
论文摘要
深层生成模型提供了一组强大的工具来了解真实的数据。但是随着这些模型的改善，它们的大小和复杂性会增加，因此它们在记忆和执行时间的计算成本会增加。在神经网络中使用二进制重量是一种在降低这一成本方面显示出希望的方法。但是，是否可以在生成模型中使用二进制神经网络是一个开放的问题。在这项工作中，我们首次表明我们可以成功培训利用二进制神经网络的生成模型。这大大降低了模型的计算成本。我们开发了一类新的二进制重量标准化，并为这些二进制生成模型的架构设计提供了见解。我们证明，可以使用这些技术有效地将两个最先进的深层生成模型（Resnet VAE和Flow ++模型）有效地进行二进制。我们训练二进制模型，以实现接近常规模型的损失值，但尺寸较小90％-94％，并且还可以在执行时间内进行大幅加速。
Deep generative models provide a powerful set of tools to understand real-world data. But as these models improve, they increase in size and complexity, so their computational cost in memory and execution time grows. Using binary weights in neural networks is one method which has shown promise in reducing this cost. However, whether binary neural networks can be used in generative models is an open problem. In this work we show, for the first time, that we can successfully train generative models which utilize binary neural networks. This reduces the computational cost of the models massively. We develop a new class of binary weight normalization, and provide insights for architecture designs of these binarized generative models. We demonstrate that two state-of-the-art deep generative models, the ResNet VAE and Flow++ models, can be binarized effectively using these techniques. We train binary models that achieve loss values close to those of the regular models but are 90%-94% smaller in size, and also allow significant speed-ups in execution time.
