论文标题
神经元特异性辍学：一种确定性的正则化技术，可防止神经网络过度拟合并减少对大型训练样本的依赖
Neuron-Specific Dropout: A Deterministic Regularization Technique to Prevent Neural Networks from Overfitting & Reduce Dependence on Large Training Samples
论文作者
论文摘要
为了在其输入和输出之间建立复杂的关系，深度神经网络训练并调整大量参数。为了使这些网络精确地工作，需要大量数据。但是，有时，所需的数据数量是不存在或用于培训的。神经元特异性辍学（NSDROPOUT）是解决此问题的工具。 NSDROPOUT查看模型中一层的训练通行证和验证通行证。通过比较数据集中每个类别为每个类别产生的平均值，该网络能够删除目标单元。该层能够预测模型在测试过程中正在查看哪些功能或噪声，而这些功能或噪声在查看验证中的样本时不存在。与辍学不同，“稀释”网络不能“不固定”进行测试。事实证明，特定于神经元的辍学能够实现相似的数据，即使不是更好的数据，比辍学方法和其他正则化方法的数据要少得多。实验表明，特定于神经元的辍学减少了网络过度拟合的机会，并减少了对图像识别中监督学习任务的大型培训样本的需求，同时又产生了最佳的结果。
In order to develop complex relationships between their inputs and outputs, deep neural networks train and adjust large number of parameters. To make these networks work at high accuracy, vast amounts of data are needed. Sometimes, however, the quantity of data needed is not present or obtainable for training. Neuron-specific dropout (NSDropout) is a tool to address this problem. NSDropout looks at both the training pass, and validation pass, of a layer in a model. By comparing the average values produced by each neuron for each class in a data set, the network is able to drop targeted units. The layer is able to predict what features, or noise, the model is looking at during testing that isn't present when looking at samples from validation. Unlike dropout, the "thinned" networks cannot be "unthinned" for testing. Neuron-specific dropout has proved to achieve similar, if not better, testing accuracy with far less data than traditional methods including dropout and other regularization methods. Experimentation has shown that neuron-specific dropout reduces the chance of a network overfitting and reduces the need for large training samples on supervised learning tasks in image recognition, all while producing best-in-class results.
