论文标题
SVD-NAS：耦合低级别近似和神经体系结构搜索
SVD-NAS: Coupling Low-Rank Approximation and Neural Architecture Search
论文作者
论文摘要
压缩预训练的深度神经网络的任务吸引了研究社区的广泛兴趣，因为它可以使从业人员摆脱数据访问要求。在这个域中，低级别的近似是一种有前途的方法，但是现有的解决方案被认为是限制的设计选择，并且未能有效地探索设计空间，从而导致严重的准确性降解和有限的压缩比达到了限制。为了解决上述局限性，这项工作提出了SVD-NAS框架，该框架将低级近似和神经体系结构搜索的域结合在一起。 SVD-NAS通用并扩展了以前作品的设计选择，它通过引入低级架构空间LR空间，这是一个更细粒度的低级别近似设计空间。之后，这项工作提出了基于梯度的搜索，以有效地穿越LR空间。对可能的设计选择的更精细，更彻底的探索会提高准确性，并减少CNN模型的参数，失败和潜伏期。结果表明，在数据限制的问题设置下，SVD-NAS的成像网上的精度比最新方法高2.06-12.85pp。 SVD-NAS在https://github.com/yu-zhewen/svd-nas上开源。
The task of compressing pre-trained Deep Neural Networks has attracted wide interest of the research community due to its great benefits in freeing practitioners from data access requirements. In this domain, low-rank approximation is a promising method, but existing solutions considered a restricted number of design choices and failed to efficiently explore the design space, which lead to severe accuracy degradation and limited compression ratio achieved. To address the above limitations, this work proposes the SVD-NAS framework that couples the domains of low-rank approximation and neural architecture search. SVD-NAS generalises and expands the design choices of previous works by introducing the Low-Rank architecture space, LR-space, which is a more fine-grained design space of low-rank approximation. Afterwards, this work proposes a gradient-descent-based search for efficiently traversing the LR-space. This finer and more thorough exploration of the possible design choices results in improved accuracy as well as reduction in parameters, FLOPS, and latency of a CNN model. Results demonstrate that the SVD-NAS achieves 2.06-12.85pp higher accuracy on ImageNet than state-of-the-art methods under the data-limited problem setting. SVD-NAS is open-sourced at https://github.com/Yu-Zhewen/SVD-NAS.
