论文标题
LAF:自动化深神经网络重复使用的无标签模型选择
LaF: Labeling-Free Model Selection for Automated Deep Neural Network Reusing
论文作者
论文摘要
近年来,将深度学习应用于科学是一种新趋势,它导致DL工程成为一个重要的问题。尽管培训数据准备,模型架构设计和模型培训是建立DL模型的正常过程,但它们都是复杂且昂贵的。因此,重复开源的预培训模型是绕过开发人员这一障碍的实用方法。鉴于一项特定的任务,开发人员可以从公共来源收集大量的预训练的深度神经网络以重新利用。但是,测试多个DNN的性能(例如准确性和鲁棒性)并建议应使用哪种模型在标记数据的稀缺性和对域专业知识的需求方面具有挑战性。在本文中,我们提出了一种无标签(LAF)模型选择方法,以克服自动化模型重复使用的标签工作的局限性。主要思想是从统计学上学习贝叶斯模型,以仅根据预测标签推断模型的专业。我们使用9个基准数据集评估LAF,包括图像,文本和源代码以及165个DNN,考虑了模型的准确性和鲁棒性。实验结果表明,在Spearman的相关性和Kendall的$τ$上,LAF的表现分别优于基线方法高达0.74和0.53。
Applying deep learning to science is a new trend in recent years which leads DL engineering to become an important problem. Although training data preparation, model architecture design, and model training are the normal processes to build DL models, all of them are complex and costly. Therefore, reusing the open-sourced pre-trained model is a practical way to bypass this hurdle for developers. Given a specific task, developers can collect massive pre-trained deep neural networks from public sources for re-using. However, testing the performance (e.g., accuracy and robustness) of multiple DNNs and recommending which model should be used is challenging regarding the scarcity of labeled data and the demand for domain expertise. In this paper, we propose a labeling-free (LaF) model selection approach to overcome the limitations of labeling efforts for automated model reusing. The main idea is to statistically learn a Bayesian model to infer the models' specialty only based on predicted labels. We evaluate LaF using 9 benchmark datasets including image, text, and source code, and 165 DNNs, considering both the accuracy and robustness of models. The experimental results demonstrate that LaF outperforms the baseline methods by up to 0.74 and 0.53 on Spearman's correlation and Kendall's $τ$, respectively.