论文标题
知识库指数通过维度和精确降低的压缩
Knowledge Base Index Compression via Dimensionality and Precision Reduction
论文作者
论文摘要
最近，基于神经网络的知识密集型NLP任务（例如问答）的方法开始严重依赖神经夺回者和读者的组合。检索通常是通过大型文本知识库（KB）进行的，该知识库需要大量的内存和计算资源，尤其是在扩展时。在HOTPOTQA上，我们通过维度（稀疏的随机投影，PCA，自动编码器）和数值精度降低系统地研究了KB指数的大小。 我们的结果表明，PCA是一个简单的解决方案，它需要很少的数据，并且仅比自动编码器稍差，而自动编码器稳定不那么稳定。所有方法对预处理和后处理都敏感，并且在降低尺寸之前和之后，应始终将数据始终居中和归一化。最后，我们证明可以将PCA与每个维度使用1位使用1位相结合。总体而言，我们以75％的速度实现（1）100 $ \ times $压缩，（2）24 $ \ times $压缩，以92％的原始检索性能。
Recently neural network based approaches to knowledge-intensive NLP tasks, such as question answering, started to rely heavily on the combination of neural retrievers and readers. Retrieval is typically performed over a large textual knowledge base (KB) which requires significant memory and compute resources, especially when scaled up. On HotpotQA we systematically investigate reducing the size of the KB index by means of dimensionality (sparse random projections, PCA, autoencoders) and numerical precision reduction. Our results show that PCA is an easy solution that requires very little data and is only slightly worse than autoencoders, which are less stable. All methods are sensitive to pre- and post-processing and data should always be centered and normalized both before and after dimension reduction. Finally, we show that it is possible to combine PCA with using 1bit per dimension. Overall we achieve (1) 100$\times$ compression with 75%, and (2) 24$\times$ compression with 92% original retrieval performance.
