Abstract:
The pan sharpening method combines low resolution hyperspectral and panchromatic images to obtain images with high spatial and spectral resolution. To fully explore the spatial texture features of panchromatic images, octave convolution is introduced into the original VGG19 feature extractor, while paying attention to the high and low frequency features of panchromatic images at different spatial scales. To enhance the local spatial features of spectral images, a feature modulation module is added to the original VGG19 feature extractor to inject high-frequency features of panchromatic images into the spatial details of spectral images. In order to enhance the global dependency between spectral image features and low-frequency features of panchromatic images, and reduce network complexity, the network optimizes multi head attention deeply, so that spectral images only focus on the most relevant image block regions of panchromatic images. In the reconstruction stage, Sobel filters, GCN graph neural networks, and ECA attention aggregation spectral image edge details and significant band features are used. At the same time, VGG perception loss and edge loss are introduced to increase the proportion of edge features in the fused image. Verified with other pan sharpening methods on the Pavia Center, Botswana, and Chikusei datasets, the Peak Signal-to-Noise Ratio(PSNR) values reached 40.80%, 45.90% and 43.26%, respectively, while the SAM values decreased to 4.15%, 1.38% and 2.70%, respectively. The proposed pan sharpening method can effectively restore spatial texture details of spectral images while avoiding spectral distortion.