论文标题
低数据环境的基于变压器的程序合成
Transformer-based Program Synthesis for Low-Data Environments
论文作者
论文摘要
大型预训练的变压器模型（GPT2/3，T5）的最新进展已在程序合成中使用来生成满足一组输入/输出示例的程序。但是，这些模型在长马和低数据任务上的表现较差，而且通常似乎不了解它们产生的语言的语义。我们通过使用编程语言的属性无上下文环境来生成程序，然后分析生成的程序，以便可以用编译和运行时属性（例如类型）注释它们，从而可以通过类型来注释它们，以便可以通过编程和运行时属性进行注释，以便在长期生成期间记住有关程序的信息。我们首先发现可以有效地制作合成的数据集，并可以为变压器模型提供足够的数据，以便在某些综合任务上执行良好的数据。我们还发现，在低数据环境中，让模型访问程序属性尤其有效，并且趋向于改善质量并减少变形金刚生成的程序的错误。
Recent advancements in large pre-trained transformer models (GPT2/3, T5) have found use in program synthesis to generate programs that satisfy a set of input/output examples. However, these models perform poorly on long-horizon and low-data tasks, and often don't seem to understand the semantics of the languages they generate. We investigate an approach that tackles both of these issues, by using attributed context-free-grammars of programming languages to generate programs, and then analyzing generated programs so that they can be annotated with compile and runtime attributes, such as types, so that information about the program can be remembered during long-horizon generation. We firstly find that synthesized datasets can be made efficiently and can provide transformer models with enough data in order to perform well on some synthesis tasks. We also find that giving models access to program attributes is especially effective in low-data environments, and tends improve the quality and reduce errors of transformer-generated programs.
