论文标题
具有快速再培训的深层网络
Deep Networks with Fast Retraining
论文作者
论文摘要
最近的工作[1]利用了深卷积神经网络（DCNN）学习中的摩尔 - 苯甲酸（MP）逆，该学习通过随机梯度下降（SGD）管道实现了比DCNN更好的概括性能。但是，杨的工作在实践中并没有在实践中获得太大的知名度，因为它对超参数的敏感性很高以及对计算资源的严格要求。为了增强其适用性，本文提出了一种新型的MP基于基于逆的快速再培训策略。在每个训练时期，首先使用了一个随机学习策略，该策略控制在向后通过的卷积层数量。然后，开发了一个基于MP逆批量的学习策略，该策略可以在不访问工业规模的计算资源的情况下实现网络，以完善密集的层参数。实验结果从经验上表明，快速撤回是可用于所有DCNN的统一策略。与其他学习策略相比，提议的学习管道对超参数具有鲁棒性，并且计算资源的要求大大减少了。 [1] Y. Yang，J。Wu，X。Feng和A. Thangarajah，“ dcnn的Perfor-238mance改进的致密层的重新成分”，IEEE Trans。模式肛门。马赫。 Intell。，2019。
Recent work [1] has utilized Moore-Penrose (MP) inverse in deep convolutional neural network (DCNN) learning, which achieves better generalization performance over the DCNN with a stochastic gradient descent (SGD) pipeline. However, Yang's work has not gained much popularity in practice due to its high sensitivity of hyper-parameters and stringent demands of computational resources. To enhance its applicability, this paper proposes a novel MP inverse-based fast retraining strategy. In each training epoch, a random learning strategy that controls the number of convolutional layers trained in the backward pass is first utilized. Then, an MP inverse-based batch-by-batch learning strategy, which enables the network to be implemented without access to industrial-scale computational resources, is developed to refine the dense layer parameters. Experimental results empirically demonstrate that fast retraining is a unified strategy that can be used for all DCNNs. Compared to other learning strategies, the proposed learning pipeline has robustness against the hyper-parameters, and the requirement of computational resources is significantly reduced. [1] Y. Yang, J. Wu, X. Feng, and A. Thangarajah, "Recomputation of dense layers for the perfor-238mance improvement of dcnn," IEEE Trans. Pattern Anal. Mach. Intell., 2019.
