论文标题
学习视觉接地的记忆助手
Learning a Visually Grounded Memory Assistant
论文作者
论文摘要
我们介绍了一个新颖的界面，以大规模收集人类记忆和援助。使用3D Matterport模拟器，我们创建了一个现实的室内环境，在该环境中，我们让人们执行模仿家庭日常活动的特定体现的记忆任务。然后将该界面部署在亚马逊机械土耳其人上，使我们能够测试和记录人类记忆，导航和对以前不可能的大规模帮助的需求。使用该界面，我们收集旨在发展我们对（1）人们在3D环境导航期间的信息的理解以及（2）人们要求内存帮助的条件的信息。此外，我们尝试预测人们何时使用经过手工选择的视觉和语义特征训练的模型寻求帮助。这提供了一个机会，通过学习的人类感知，记忆和认知模型在机器学习和认知科学社区之间建立更牢固的联系。
We introduce a novel interface for large scale collection of human memory and assistance. Using the 3D Matterport simulator we create a realistic indoor environments in which we have people perform specific embodied memory tasks that mimic household daily activities. This interface was then deployed on Amazon Mechanical Turk allowing us to test and record human memory, navigation and needs for assistance at a large scale that was previously impossible. Using the interface we collect the `The Visually Grounded Memory Assistant Dataset' which is aimed at developing our understanding of (1) the information people encode during navigation of 3D environments and (2) conditions under which people ask for memory assistance. Additionally we experiment with with predicting when people will ask for assistance using models trained on hand-selected visual and semantic features. This provides an opportunity to build stronger ties between the machine-learning and cognitive-science communities through learned models of human perception, memory, and cognition.
