论文标题
对数修剪是您所需要的
Logarithmic Pruning is All You Need
论文作者
论文摘要
彩票票证假设是一个猜想，每个大型神经网络都包含一个子网，在孤立地训练时，可以实现与大型网络相当的性能。最近已经证明了一个更强的猜想：每个足够的过度参数化网络都包含一个子网，该子网随机初始化，但没有训练，就可以达到与受过训练的大型网络相当的准确性。然而，后一个结果依赖于许多强大的假设，并保证了与目标函数相比，大型网络大小的多项式因素。在这项工作中，我们删除了先前工作的最局限性假设，同时提供了更紧密的界限：过度参数化的网络仅需要一个对数因子（在所有变量中，但深度）每个重量的神经元数量）每个重量的神经元数量。
The Lottery Ticket Hypothesis is a conjecture that every large neural network contains a subnetwork that, when trained in isolation, achieves comparable performance to the large network. An even stronger conjecture has been proven recently: Every sufficiently overparameterized network contains a subnetwork that, at random initialization, but without training, achieves comparable accuracy to the trained large network. This latter result, however, relies on a number of strong assumptions and guarantees a polynomial factor on the size of the large network compared to the target function. In this work, we remove the most limiting assumptions of this previous work while providing significantly tighter bounds:the overparameterized network only needs a logarithmic factor (in all variables but depth) number of neurons per weight of the target subnetwork.
