论文标题
用合成代理导航人类语言模型
Navigating Human Language Models with Synthetic Agents
论文作者
论文摘要
诸如GPT-2/GPT-3之类的现代自然语言模型包含有关人类信念的大量信息，以一致的可测试形式。如果可以证明这些模型可以准确反映人类的基本信念，这些人类产生了用于训练这些模型的数据，那么这种模型以与传统方法（例如访谈和调查）不同的方式成为强大的社会学工具。在这项研究中，我们使用文本字符串来创建上下文和方向，在历史象棋游戏中训练GPT-2版本，然后将合成代理的簇“启动”到模型中。我们比较了代理/模型产生的文本中包含的轨迹，并将其与国际象棋板的已知地面真理，移动合法性和历史模式进行比较。我们发现使用模型的移动百分比与人类模式基本相似。我们进一步发现，该模型会创建棋盘的准确潜在表示，并且可以使用此知识绘制整个法律移动的轨迹。
Modern natural language models such as the GPT-2/GPT-3 contain tremendous amounts of information about human belief in a consistently testable form. If these models could be shown to accurately reflect the underlying beliefs of the human beings that produced the data used to train these models, then such models become a powerful sociological tool in ways that are distinct from traditional methods, such as interviews and surveys. In this study, We train a version of the GPT-2 on a corpora of historical chess games, and then "launch" clusters of synthetic agents into the model, using text strings to create context and orientation. We compare the trajectories contained in the text generated by the agents/model and compare that to the known ground truth of the chess board, move legality, and historical patterns of play. We find that the percentages of moves by piece using the model are substantially similar from human patterns. We further find that the model creates an accurate latent representation of the chessboard, and that it is possible to plot trajectories of legal moves across the board using this knowledge.
