论文标题
U（1）在通用CNN瓶颈层中观察到的对称性破坏
U(1) Symmetry-breaking Observed in Generic CNN Bottleneck Layers
论文作者
论文摘要
我们报告了一个新的模型，该模型将深度卷积神经网络（CNN）与生物视觉和基本粒子物理学联系起来。 CNN中的信息传播是通过与光学系统的类比建模的，该信息集中在瓶颈附近，其中2D空间分辨率折叠了大约焦点$ 1 \ times 1 = 1 $。 A 3D space $(x,y,t)$ is defined by $(x,y)$ coordinates in the image plane and CNN layer $t$, where a principal ray $(0,0,t)$ runs in the direction of information propagation through both the optical axis and the image center pixel located at $(x,y)=(0,0)$, about which the sharpest possible spatial focus is limited to a circle of confusion in the image plane.我们的新颖见解是将主要光射线$（0,0，t）$建模为几何等同于内侧向量，以$ n $ channel激活空间的r^{n+} $ in r^{n+} $ in r^{n+} $中的内侧向量等同。沿着$ rgb $ color Space中的灰度（或亮度）矢量$（t，t，t）$。因此，信息集中在能源潜力$ e（x，y，t）= \ | i（x，x，y，t）\ |^2 $中，特别是对于瓶颈$ t $ t $ t $ t $ t $ t $ t $ t $ t $ the sombrero solys the boson粒子。这种对称性在分类中被打破，其中通用预训练的CNN模型的瓶颈层表现出对u（1）$在图像平面和激活特征空间中同时定义的u（1）$中的角度$θ\的一致性偏差。最初的观察结果验证了我们的假设，可以从通用的预训练的CNN激活图和基于微小的存储器分类方案（没有训练或调整）中进行验证。使用合并的一热$+ U（1）$损失从头开始培训可改善包括Imagenet在内的所有测试的任务的分类。
We report on a novel model linking deep convolutional neural networks (CNN) to biological vision and fundamental particle physics. Information propagation in a CNN is modeled via an analogy to an optical system, where information is concentrated near a bottleneck where the 2D spatial resolution collapses about a focal point $1\times 1=1$. A 3D space $(x,y,t)$ is defined by $(x,y)$ coordinates in the image plane and CNN layer $t$, where a principal ray $(0,0,t)$ runs in the direction of information propagation through both the optical axis and the image center pixel located at $(x,y)=(0,0)$, about which the sharpest possible spatial focus is limited to a circle of confusion in the image plane. Our novel insight is to model the principal optical ray $(0,0,t)$ as geometrically equivalent to the medial vector in the positive orthant $I(x,y) \in R^{N+}$ of a $N$-channel activation space, e.g. along the greyscale (or luminance) vector $(t,t,t)$ in $RGB$ colour space. Information is thus concentrated into an energy potential $E(x,y,t)=\|I(x,y,t)\|^2$, which, particularly for bottleneck layers $t$ of generic CNNs, is highly concentrated and symmetric about the spatial origin $(0,0,t)$ and exhibits the well-known "Sombrero" potential of the boson particle. This symmetry is broken in classification, where bottleneck layers of generic pre-trained CNN models exhibit a consistent class-specific bias towards an angle $θ\in U(1)$ defined simultaneously in the image plane and in activation feature space. Initial observations validate our hypothesis from generic pre-trained CNN activation maps and a bare-bones memory-based classification scheme, with no training or tuning. Training from scratch using combined one-hot $+ U(1)$ loss improves classification for all tasks tested including ImageNet.
