论文标题
通过双通道卷积网络具有卷积LSTM的层次分层空间光谱特征融合的多光谱板折叠
Multispectral Pan-sharpening via Dual-Channel Convolutional Network with Convolutional LSTM Based Hierarchical Spatial-Spectral Feature Fusion
论文作者
论文摘要
多光谱板肖型旨在通过融合Panchromatic（PAN）图像和相应的MS图像来在空间和光谱域中产生高分辨率（HR）多光谱（MS）图像。在本文中，我们提出了一个新型的双通道网络（DCNET）框架，用于MS Pan-Sharpening。在我们的DCNET中，双通道主链涉及使用2D CNN捕获空间信息的空间通道，以及使用3D CNN提取光谱信息的光谱通道。这种异质的2D/3D CNN体系结构可以最大程度地减少引起光谱信息失真，这通常发生在常规的2D CNN模型中。为了完全整合从不同级别捕获的空间和光谱特征，我们引入了多级融合策略。具体而言，提出了一个空间 - 光谱CLSTM（S $^2 $ -CLSTM）模块，用于融合层次的空间和频谱特征，该特征可以有效地捕获多级特征之间的相关性。 S $^2 $ -CLSTM模块附加了两种融合方法：通过双向横向连接进行内部融合，并通过S $^2 $ -CLSTM中的单元格状态进行层间融合。最后，理想的HR-MS图像通过重建模块恢复。已经在模拟的较低尺度和现实世界数据集的原始规模上进行了广泛的实验。与最先进的方法相比，所提出的DCNET取得了卓越或竞争性能。
Multispectral pan-sharpening aims at producing a high resolution (HR) multispectral (MS) image in both spatial and spectral domains by fusing a panchromatic (PAN) image and a corresponding MS image. In this paper, we propose a novel dual-channel network (DCNet) framework for MS pan-sharpening. In our DCNet, the dual-channel backbone involves a spatial channel to capture spatial information with a 2D CNN, and a spectral channel to extract spectral information with a 3D CNN. This heterogeneous 2D/3D CNN architecture can minimize causing spectral information distortion, which typically happens in conventional 2D CNN models. In order to fully integrate the spatial and spectral features captured from different levels, we introduce a multi-level fusion strategy. Specifically, a spatial-spectral CLSTM (S$^2$-CLSTM) module is proposed for fusing the hierarchical spatial and spectral features, which can effectively capture correlations among multi-level features. The S$^2$-CLSTM module attaches two fusion ways: the intra-level fusion via bi-directional lateral connections and inter-level fusion via the cell state in the S$^2$-CLSTM. Finally, the ideal HR-MS image is recovered by a reconstruction module. Extensive experiments have been conducted at both simulated lower scale and the original scale of real-world datasets. Compared with the state-of-the-art methods, the proposed DCNet achieves superior or competitive performance.
