论文标题
在对话型AI中处理医疗查询的风险级别安全
Risk-graded Safety for Handling Medical Queries in Conversational AI
论文作者
论文摘要
会话AI系统可以在处理用户的医疗查询时可能会产生严重后果甚至可能导致死亡的情况。因此，系统需要能够认识到医疗投入的严重性和具有适当风险水平的反应。我们创建了人类书面的英语医学查询和不同类型系统的响应的语料库。我们将它们标记为众包和专家注释。尽管个别人群工人可能在评分提示的严重性方面可能不可靠，但他们的汇总标签倾向于在更大程度上同意专业意见，即确定医疗查询并认识到响应所带来的风险类型。分类实验的结果表明，尽管这些任务可以自动化，但应谨慎行事，因为错误可能非常严重。
Conversational AI systems can engage in unsafe behaviour when handling users' medical queries that can have severe consequences and could even lead to deaths. Systems therefore need to be capable of both recognising the seriousness of medical inputs and producing responses with appropriate levels of risk. We create a corpus of human written English language medical queries and the responses of different types of systems. We label these with both crowdsourced and expert annotations. While individual crowdworkers may be unreliable at grading the seriousness of the prompts, their aggregated labels tend to agree with professional opinion to a greater extent on identifying the medical queries and recognising the risk types posed by the responses. Results of classification experiments suggest that, while these tasks can be automated, caution should be exercised, as errors can potentially be very serious.
