论文标题
在范围内的深层神经网络中降低
Subaging in underparametrized Deep Neural Networks
论文作者
论文摘要
我们考虑一个简单的分类问题，以表明隔离式状态中有限宽度的深神经网络的动力学产生了类似于与玻璃系统相似的效果，即损失函数和老化的缓慢演变。值得注意的是，在等待时间（下属）中，衰老是均匀的，并且在恒定总参数总数的约束下，对不同架构的幂律指数是可靠的。我们的结果保持在MNIST数据库的更复杂方案中。我们发现，对于此数据库，有一个唯一的指数来统治整个阶段的求职行为。
We consider a simple classification problem to show that the dynamics of finite-width Deep Neural Networks in the underparametrized regime gives rise to effects similar to those associated with glassy systems, namely a slow evolution of the loss function and aging. Remarkably, the aging is sublinear in the waiting time (subaging) and the power-law exponent characterizing it is robust to different architectures under the constraint of a constant total number of parameters. Our results are maintained in the more complex scenario of the MNIST database. We find that for this database there is a unique exponent ruling the subaging behavior in the whole phase.
