论文标题
Threshy：支持安全使用智能网络服务
Threshy: Supporting Safe Usage of Intelligent Web Services
论文作者
论文摘要
“智能” Web服务的受欢迎程度越来越高，为开发人员提供了很少的努力，为最终用户提供了机器学习功能。但是，这些服务需要设置一个取决于特定问题数据的决策阈值。开发人员缺乏评估智能服务的系统方法，现有的评估工具主要针对数据科学家进行开发前评估。本文介绍了一个工作流和支持工具Threshy，以帮助软件开发人员选择适合其问题域的决策门槛。与现有工具不同，Threshy旨在在多个工作流程中运行，包括前开发，预发行和支持。 Threshy旨在调整智能Web服务返回的置信分数，并且不涉及ML模型中使用的超参数优化。此外，它考虑了误报的财务影响。通过阈值导出的阈值配置文件可以集成到客户端应用程序和监视基础架构中。演示：https：//bit.ly/2ykeyhe。
Increased popularity of `intelligent' web services provides end-users with machine-learnt functionality at little effort to developers. However, these services require a decision threshold to be set which is dependent on problem-specific data. Developers lack a systematic approach for evaluating intelligent services and existing evaluation tools are predominantly targeted at data scientists for pre-development evaluation. This paper presents a workflow and supporting tool, Threshy, to help software developers select a decision threshold suited to their problem domain. Unlike existing tools, Threshy is designed to operate in multiple workflows including pre-development, pre-release, and support. Threshy is designed for tuning the confidence scores returned by intelligent web services and does not deal with hyper-parameter optimisation used in ML models. Additionally, it considers the financial impacts of false positives. Threshold configuration files exported by Threshy can be integrated into client applications and monitoring infrastructure. Demo: https://bit.ly/2YKeYhE.
