论文标题
MaxDropout：基于最大输出值的深神经网络正则化
MaxDropout: Deep Neural Network Regularization Based on Maximum Output Values
论文作者
论文摘要
在深度学习方案中已经出现了不同的技术，例如卷积神经网络，深度信念网络和长期的短期记忆网络，以引用一些。在Lockstep中，旨在通过惩罚重量连接或关闭某些单元来防止过度拟合的正规化方法已被广泛研究。在本文中，我们提出了一种名为MaxDropout的新方法，该方法是深层神经网络模型的常规化合物，该方法通过在每个隐藏层中删除（关闭）著名的神经元（即最活跃），以监督的方式工作。该模型迫使更少的激活单元学习更多代表性信息，从而提供稀疏性。关于实验，我们表明可以改善现有的神经网络，并在MaxDropout取代辍学时在神经网络中提供更好的结果。提出的方法在图像分类中进行了评估，与现有正规化器（例如切割和随机性化）相当的结果，也提高了通过MaxDropout替换现有层来使用辍学的神经网络的准确性。
Different techniques have emerged in the deep learning scenario, such as Convolutional Neural Networks, Deep Belief Networks, and Long Short-Term Memory Networks, to cite a few. In lockstep, regularization methods, which aim to prevent overfitting by penalizing the weight connections, or turning off some units, have been widely studied either. In this paper, we present a novel approach called MaxDropout, a regularizer for deep neural network models that works in a supervised fashion by removing (shutting off) the prominent neurons (i.e., most active) in each hidden layer. The model forces fewer activated units to learn more representative information, thus providing sparsity. Regarding the experiments, we show that it is possible to improve existing neural networks and provide better results in neural networks when Dropout is replaced by MaxDropout. The proposed method was evaluated in image classification, achieving comparable results to existing regularizers, such as Cutout and RandomErasing, also improving the accuracy of neural networks that uses Dropout by replacing the existing layer by MaxDropout.
