论文标题
NAS基础 - 套件零：对零成本代理的加速研究
NAS-Bench-Suite-Zero: Accelerating Research on Zero Cost Proxies
论文作者
论文摘要
零成本代理（ZC代理）是一种近期的体系结构性能预测技术，旨在显着加快神经体系结构搜索算法（NAS）。最近的工作表明，这些技术表现出巨大的希望，但是某些方面（例如评估和利用它们的互补优势）并没有研究。在这项工作中，我们创建了NAS Bench-Suite：我们评估了28个任务的13个ZC代理，创建了ZC代理的最大数据集（和统一的代码库），从而使ZC Proxies上的数量级实验更快，同时避免避免从不同实现的混淆因素。为了证明NAS基础 - 套件的有用性，我们对ZC代理进行大规模分析，包括偏见分析，以及第一个信息理论分析，得出的结论是ZC Proxies捕获了实质性的互补信息。在这些发现的激励下，我们提出了一种程序，通过减少诸如细胞大小之类的偏差来提高ZC代理的性能，并且我们还表明，将所有13个ZC代理纳入NAS算法使用的替代模型可以将其预测性能提高其预测性能高达42％。我们的代码和数据集可从https://github.com/automl/naslib/tree/zerocost获得。
Zero-cost proxies (ZC proxies) are a recent architecture performance prediction technique aiming to significantly speed up algorithms for neural architecture search (NAS). Recent work has shown that these techniques show great promise, but certain aspects, such as evaluating and exploiting their complementary strengths, are under-studied. In this work, we create NAS-Bench-Suite: we evaluate 13 ZC proxies across 28 tasks, creating by far the largest dataset (and unified codebase) for ZC proxies, enabling orders-of-magnitude faster experiments on ZC proxies, while avoiding confounding factors stemming from different implementations. To demonstrate the usefulness of NAS-Bench-Suite, we run a large-scale analysis of ZC proxies, including a bias analysis, and the first information-theoretic analysis which concludes that ZC proxies capture substantial complementary information. Motivated by these findings, we present a procedure to improve the performance of ZC proxies by reducing biases such as cell size, and we also show that incorporating all 13 ZC proxies into the surrogate models used by NAS algorithms can improve their predictive performance by up to 42%. Our code and datasets are available at https://github.com/automl/naslib/tree/zerocost.
