论文标题
MMA正则化：通过最大化最小角度来使神经网络的权重置
MMA Regularization: Decorrelating Weights of Neural Networks by Maximizing the Minimal Angles
论文作者
论文摘要
神经元或过滤器之间的强相关性可以显着削弱神经网络的概括能力。受到众所周知的毛茸茸问题的启发，我们提出了一种新型的多样性正则化方法来解决此问题，这使神经元或过滤器的归一化重量向量通过最大程度地提高最小成对角度（MMA），使神经元或过滤器尽可能均匀地分布在超晶体上。通过将MMA正则项插入损失函数中，该方法可以轻松地发挥其效果，并用可忽略的计算开销。 MMA正则化简单，高效且有效。因此，它可以用作神经网络训练中的基本正则方法。广泛的实验表明，MMA正则化能够增强各种现代模型的概括能力，并在CIFAR100和Tinyimagenet数据集上取得了可观的性能改进。此外，面部验证的实验表明，MMA正则化对特征学习也有效。代码可在以下网址提供：https：//github.com/wznpub/mma_regularization。
The strong correlation between neurons or filters can significantly weaken the generalization ability of neural networks. Inspired by the well-known Tammes problem, we propose a novel diversity regularization method to address this issue, which makes the normalized weight vectors of neurons or filters distributed on a hypersphere as uniformly as possible, through maximizing the minimal pairwise angles (MMA). This method can easily exert its effect by plugging the MMA regularization term into the loss function with negligible computational overhead. The MMA regularization is simple, efficient, and effective. Therefore, it can be used as a basic regularization method in neural network training. Extensive experiments demonstrate that MMA regularization is able to enhance the generalization ability of various modern models and achieves considerable performance improvements on CIFAR100 and TinyImageNet datasets. In addition, experiments on face verification show that MMA regularization is also effective for feature learning. Code is available at: https://github.com/wznpub/MMA_Regularization.
