论文标题
人类判断作为指南针，以导航自动指标以进行形式转移
Human Judgement as a Compass to Navigate Automatic Metrics for Formality Transfer
论文作者
论文摘要
尽管近年来文本样式转移已经见证了快速发展，但尚无既定标准的评估标准，该标准是使用多个自动指标进行的，因此缺乏始终诉诸于人类判断的可能性。我们专注于形式转移的任务，以及通常评估的三个方面：风格强度，内容保存和流利性。为了阐明如何通过共同和新指标评估此类方面，我们进行了基于人类的评估并进行丰富的相关分析。然后，我们能够就形式转移中使用此类指标的使用提出一些建议，还要关注它们对相关任务的普遍性（或不使用）。
Although text style transfer has witnessed rapid development in recent years, there is as yet no established standard for evaluation, which is performed using several automatic metrics, lacking the possibility of always resorting to human judgement. We focus on the task of formality transfer, and on the three aspects that are usually evaluated: style strength, content preservation, and fluency. To cast light on how such aspects are assessed by common and new metrics, we run a human-based evaluation and perform a rich correlation analysis. We are then able to offer some recommendations on the use of such metrics in formality transfer, also with an eye to their generalisability (or not) to related tasks.
