论文标题
低维流形中的比较器与比较器的外推性关系推理
Extrapolatable Relational Reasoning With Comparators in Low-Dimensional Manifolds
论文作者
论文摘要
尽管现代深度神经体系结构从与培训数据相同的分布中采样时，概括了良好的概括，但对于测试数据分布与培训分布的不同，它们甚至沿着几个维度差异。当任务变得更加抽象和复杂，例如关系推理时，缺乏分布概括的概括越来越大。在本文中，我们提出了一个由神经科学启发的诱导偏置模块，可以与当前的神经网络体系结构相融化，以改善有关关系推理任务上的分布（O.O.D）通用性能。该模块学会了将高维对象表示形式投影到低维流形的，以进行更有效，更普遍的关系比较。我们表明，具有这种归纳偏见的神经网络在一系列关系推理任务方面取得了相当大的O.O.D概括性能。我们最终分析了提出的归纳偏置模块，以了解较低维投影的重要性，并提出对算法对齐理论的增强，以更好地测量算法对准与概括。
While modern deep neural architectures generalise well when test data is sampled from the same distribution as training data, they fail badly for cases when the test data distribution differs from the training distribution even along a few dimensions. This lack of out-of-distribution generalisation is increasingly manifested when the tasks become more abstract and complex, such as in relational reasoning. In this paper we propose a neuroscience-inspired inductive-biased module that can be readily amalgamated with current neural network architectures to improve out-of-distribution (o.o.d) generalisation performance on relational reasoning tasks. This module learns to project high-dimensional object representations to low-dimensional manifolds for more efficient and generalisable relational comparisons. We show that neural nets with this inductive bias achieve considerably better o.o.d generalisation performance for a range of relational reasoning tasks. We finally analyse the proposed inductive bias module to understand the importance of lower dimension projection, and propose an augmentation to the algorithmic alignment theory to better measure algorithmic alignment with generalisation.
