论文标题
SGPT：GPT句子嵌入式语义搜索
SGPT: GPT Sentence Embeddings for Semantic Search
论文作者
论文摘要
解码器变压器的规模继续增加，达到了数千亿个参数。由于它们的比例，相同的解码器通过提示或微调来设置各种语言任务的最新结果。但是，这些大型基础模型对于语义搜索和句子嵌入的相关领域仍然无法使用。这样可以防止新的最新结果，并迫使组织训练和维护单独的模型。为此，我们建议SGPT通过提示或微调使用解码器将解码器用于句子嵌入和语义搜索。在贝尔搜索基准中测得的，在58亿个参数的参数sgpt上的边距为7％，略高于7％，并胜过1750亿参数的并发方法。代码，模型和结果文件可在https://github.com/muennighoff/sgpt上免费获得。
Decoder transformers have continued increasing in scale reaching hundreds of billions of parameters. Due to their scale the same decoder sets state-of-the-art results on various language tasks via prompting or fine-tuning. Yet, these large foundation models remain unusable for the related fields of semantic search and sentence embeddings. This prevents possibly new state-of-the-art results and forces organizations to train and maintain separate models. To this end, we propose SGPT to use decoders for sentence embeddings and semantic search via prompting or fine-tuning. At 5.8 billion parameters SGPT improves on the previously best sentence embeddings by a margin of 7% and outperforms a concurrent method with 175 billion parameters as measured on the BEIR search benchmark. Code, models and result files are freely available at https://github.com/Muennighoff/sgpt.
