论文标题

证明彩票票证假设:修剪是您所需要的

Proving the Lottery Ticket Hypothesis: Pruning is All You Need

论文作者

Malach, Eran, Yehudai, Gilad, Shalev-Shwartz, Shai, Shamir, Ohad

论文摘要

彩票假设(Frankle and Carbin,2018年)指出,一个随机限制的网络包含一个小的子网络,因此,在隔离培训时,可以与原始网络的性能竞争。我们证明了一个更强的假设(正如Ramanujan等人,2019年的猜想),表明,对于每个有界分布的分布和每个具有有界权重的目标网络,一个具有随机权重的过度参数化的神经网络包含一个子网,具有与目标网络相同的准确性,而无需进行任何进一步的训练。

The lottery ticket hypothesis (Frankle and Carbin, 2018), states that a randomly-initialized network contains a small subnetwork such that, when trained in isolation, can compete with the performance of the original network. We prove an even stronger hypothesis (as was also conjectured in Ramanujan et al., 2019), showing that for every bounded distribution and every target network with bounded weights, a sufficiently over-parameterized neural network with random weights contains a subnetwork with roughly the same accuracy as the target network, without any further training.

扫码加入交流群

加入微信交流群

微信交流群二维码

扫码加入学术交流群,获取更多资源