论文标题
同源性媒介量熵正常
Homology-constrained vector quantization entropy regularizer
论文作者
论文摘要
本文根据对VQ嵌入的持续同源性分析，描述了向量量化（VQ）的熵正则项。较高的嵌入熵与更高的代码簿利用率正相关，从而使基于VQ的自动编码器中的身份和代码书崩溃过度拟合[1]。我们表明，同源性约束的正则化是增加VQ过程熵（近似于输入熵）的有效方法，同时保留了量化的潜在空间中近似拓扑，平均在迷你批处理上。这项工作进一步探讨了由向量量化形成的潜在潜在图表的一些模式。我们将所提出的算法作为集成到样品VQ-VAE的模块实现和测试。链接的代码存储库提供了所提出的体系结构的功能实现，该架构在这项工作中进一步称为同源性受限的向量量化（HC-VQ）。
This paper describes an entropy regularization term for vector quantization (VQ) based on the analysis of persistent homology of the VQ embeddings. Higher embedding entropy positively correlates with higher codebook utilization, mitigating overfit towards the identity and codebook collapse in VQ-based autoencoders [1]. We show that homology-constrained regularization is an effective way to increase entropy of the VQ process (approximated to input entropy) while preserving the approximated topology in the quantized latent space, averaged over mini batches. This work further explores some patterns of persistent homology diagrams of latents formed by vector quantization. We implement and test the proposed algorithm as a module integrated into a sample VQ-VAE. Linked code repository provides a functioning implementation of the proposed architecture, referred to as homology-constrained vector quantization (HC-VQ) further in this work.
