论文标题
脊柱网络：具有逐渐输入的深神经网络
SpinalNet: Deep Neural Network with Gradual Input
论文作者
论文摘要
深度神经网络（DNNS）在许多领域都达到了最先进的表现。但是，DNN需要较高的计算时间，人们总是期望在较低的计算中表现更好。因此，我们研究人类体感系统并设计神经网络（Spinnet），以更少的计算获得更高的精度。传统NN中的隐藏层在上一层中接收输入，应用激活函数，然后将结果传输到下一层。在提出的脊柱网络中，将每一层分为三个拆分：1）输入拆分，2）中间拆分和3）输出拆分。每一层的输入拆分接收一部分输入。每一层的中间拆分接收了上一层的中间拆分的输出以及当前层的输入拆分的输出。输入权重的数量显着低于传统DNN。脊柱网络也可以用作DNN的完全连接或分类层，并支持传统学习和转移学习。我们观察到大多数DNN的计算成本较低的误差降低。带有SpinalNet分类层的VGG-5网络上的传统学习提供了QMNIST，KUZUSHIJI-MNIST，EMNIST（字母，数字和平衡）数据集的最新性能（SOTA）。具有Imagenet预先训练的初始权重和脊柱分类层的传统学习提供了STL-10，Fruits 360，Bird225和Caltech-101数据集的SOTA性能。提议的Spinalnet的脚本可在以下链接上找到：https：//github.com/dipuk0506/spinalnet
Deep neural networks (DNNs) have achieved the state of the art performance in numerous fields. However, DNNs need high computation times, and people always expect better performance in a lower computation. Therefore, we study the human somatosensory system and design a neural network (SpinalNet) to achieve higher accuracy with fewer computations. Hidden layers in traditional NNs receive inputs in the previous layer, apply activation function, and then transfer the outcomes to the next layer. In the proposed SpinalNet, each layer is split into three splits: 1) input split, 2) intermediate split, and 3) output split. Input split of each layer receives a part of the inputs. The intermediate split of each layer receives outputs of the intermediate split of the previous layer and outputs of the input split of the current layer. The number of incoming weights becomes significantly lower than traditional DNNs. The SpinalNet can also be used as the fully connected or classification layer of DNN and supports both traditional learning and transfer learning. We observe significant error reductions with lower computational costs in most of the DNNs. Traditional learning on the VGG-5 network with SpinalNet classification layers provided the state-of-the-art (SOTA) performance on QMNIST, Kuzushiji-MNIST, EMNIST (Letters, Digits, and Balanced) datasets. Traditional learning with ImageNet pre-trained initial weights and SpinalNet classification layers provided the SOTA performance on STL-10, Fruits 360, Bird225, and Caltech-101 datasets. The scripts of the proposed SpinalNet are available at the following link: https://github.com/dipuk0506/SpinalNet
