论文标题
二进制分类器技术评估的审计框架
An Audit Framework for Technical Assessment of Binary Classifiers
论文作者
论文摘要
出于二进制分类的目的，使用逻辑回归（MLOGRM）和随机森林模型（RFM）的多级模型越来越多地部署在行业中。在某些条件下，欧洲委员会提出的人工智能法（AIA）必须使这种模型的应用是公平，透明和道德的，因此，这意味着对这些模型的技术评估。本文提出并展示了通过重点关注模型，歧视和透明度和与解释性相关的方面的审计框架，用于对RFM和MLOGRM进行技术评估。为了衡量这些方面，提出了20个kpis，与交通灯风险评估方法配对。开源数据集用于训练RFM和MLOGRM模型，并将这些KPI与交通信号灯进行了比较。评估了流行的解释性方法（例如内核和树状）的性能。预计该框架将有助于监管机构对二进制分类器进行一致性评估，并使提供者和用户受益于部署此类AI系统以符合AIA。
Multilevel models using logistic regression (MLogRM) and random forest models (RFM) are increasingly deployed in industry for the purpose of binary classification. The European Commission's proposed Artificial Intelligence Act (AIA) necessitates, under certain conditions, that application of such models is fair, transparent, and ethical, which consequently implies technical assessment of these models. This paper proposes and demonstrates an audit framework for technical assessment of RFMs and MLogRMs by focussing on model-, discrimination-, and transparency & explainability-related aspects. To measure these aspects 20 KPIs are proposed, which are paired to a traffic light risk assessment method. An open-source dataset is used to train a RFM and a MLogRM model and these KPIs are computed and compared with the traffic lights. The performance of popular explainability methods such as kernel- and tree-SHAP are assessed. The framework is expected to assist regulatory bodies in performing conformity assessments of binary classifiers and also benefits providers and users deploying such AI-systems to comply with the AIA.
