천리안위성 2A호 위성영상을 위한 영상융합기법의 비교평가 = A Comparison of Pan-sharpening Algorithms for GK-2A Satellite Imagery

국문 초록 (Abstract)

기후변화 감시에 위성 자료 활용을 위해 GCOS (Global Climate Observing System)는 시공간 해상도, 시간 변화에 따른 안정성, 불확실도 등의 요구사항을 제시하고 있다. 천리안위성 2A호의 경우, 센서의 한계로 인해 산출물들이 공간해상도 조건에 충족하지 못하는 경우가 많다. 따라서 본 연구에서는 영상융합 기법들을 천리안위성 2A호 영상에 적용하여 산출물 생성 시 활용될 수 있는 최적의 기법을 찾고자 한다. 이를 위해 CS (Component Substitution), MRA (Multiresolution Analysis), VO (Variational Optimization), DL (Deep Learning)에 포함되는 총 6가지 영상융합 기법을 활용하였다. DL의 경우 합성적(Synthesis) 특성 기반 방법을 훈련자료 구축에 사용하였다. 합성적 특성 기반 방법의 과정은 PAN (Panchromatic)과 MS (Multispectral) 영상의 공간해상도 차이만큼 두 영상의 해상도를 낮춰 융합 영상을 생성한 후 원본 MS 영상과 비교한다. 합성적 특성 기반 방법은 공간해상도를 저하시킨 PAN 영상과 MS 영상 간 기하 특성이 같아야 사용자가 원하는 수준의 융합 영상을 제작할 수 있다. 하지만, 훈련자료 구축 시 비유사성이 존재하기에 이를 최소화하는 방법으로 무작위 비율을 활용한 PSGAN 모델(PSGAN_RD)을 추가로 활용하였다. 융합 영상의 검증은 일관성(consistency) 및 합성적 특성 기반 정성적, 정량적 분석을 수행하였다. 분석 결과, 영상융합 알고리즘 중 GSA가 공간 유사도를 나타내는 평가지수에서 가장 높은 수치를 보였으며, 분광 유사도를 나타내는 지수들은 PSGAN_RD 모델의 정확도가 가장 높았다. 융합 영상의 공간 및 분광 특성을 모두 고려한다면 PSGAN_RD 모델이 천리안위성 2A호 산출물 제작에 가장 최적일 것으로 판단하였다.

다국어 초록 (Multilingual Abstract)

In order to detect climate changes using satellite imagery, the GCOS (Global Climate Observing System) defines requirements such as spatio-temporal resolution, stability by the time change, and uncertainty. Due to limitation of GK-2A sensor performance, the level-2 products can not satisfy the requirement, especially for spatial resolution. In this paper, we found the optimal pan-sharpening algorithm for GK-2A products. The six pan-sharpening methods included in CS (Component Substitution), MRA (Multi-Resolution Analysis), VO (Variational Optimization), and DL (Deep Learning) were used. In the case of DL, the synthesis property based method was used to generate training dataset. The process of synthesis property is that pan-sharpening model is applied with Pan (Panchromatic) and MS (Multispectral) images with reduced spatial resolution, and fused image is compared with the original MS image. In the synthesis property based method, fused image with desire level for user can be produced only when the geometric characteristics between the PAN with reduced spatial resolution and MS image are similar. However, since the dissimilarity exists, RD (Random Down-sampling) was additionally used as a way to minimize it. Among the pan-sharpening methods, PSGAN was applied with RD (PSGAN_RD). The fused images are qualitatively and quantitatively validated with consistency property and the synthesis property. As validation result, the GSA algorithm performs well in the evaluation index representing spatial characteristics. In the case of spectral characteristics, the PSGAN_RD has the best accuracy with the original MS image. Therefore, in consideration of spatial and spectral characteristics of fused image, we found that PSGAN_RD is suitable for GK-2A products.

참고문헌 (Reference)

1 Xin Tian, "Variational Pansharpening by Exploiting Cartoon-Texture Similarities" Institute of Electrical and Electronics Engineers (IEEE) 60 : 1-16, 2022

2 Jong-Seong Kug, "Two distinct influences of Arctic warming on cold winters over North America and East Asia" Springer Science and Business Media LLC 8 (8): 759-762, 2015

3 F.A. Kruse, "The spectral image processing system (SIPS)—interactive visualization and analysis of imaging spectrometer data" Elsevier BV 44 (44): 145-163, 1993

4 Werner Balogh, "The United Nations Programme on Space Applications: Status and direction for 2010" Elsevier BV 26 (26): 185-188, 2010

5 GCOS, "The Status of the Global Climate Observing System 2021: The GCOS Status Report" WMO 1-384, 2021

6 WMO, "Space and Climate Change: Use of Spacebased Technologies in the United Nations System"

7 Furkan Ozcelik, "Rethinking CNN-Based Pansharpening: Guided Colorization of Panchromatic Images via GANs" Institute of Electrical and Electronics Engineers (IEEE) 59 (59): 3486-3501, 2021

8 Judah Cohen, "Recent Arctic amplification and extreme mid-latitude weather" Springer Science and Business Media LLC 7 (7): 627-637, 2014

9 Frosti Palsson, "Quantitative Quality Evaluation of Pansharpened Imagery: Consistency Versus Synthesis" Institute of Electrical and Electronics Engineers (IEEE) 54 (54): 1247-1259, 2016

10 Giuseppe Masi, "Pansharpening by Convolutional Neural Networks" MDPI AG 8 (8): 594-, 2016

11 Arash Golibagh Mahyari, "Panchromatic and Multispectral Image Fusion Based on Maximization of Both Spectral and Spatial Similarities" Institute of Electrical and Electronics Engineers (IEEE) 49 (49): 1976-1985, 2011

12 Liu, Q., "PSGAN: A generative adversarial network for remote sensing image pan-sharpening" 59 (59): 10227-10242, 2020

13 Sajjad Eghbalian, "Multi spectral image fusion by deep convolutional neural network and new spectral loss function" Informa UK Limited 39 (39): 3983-4002, 2018

14 B. Aiazzi, "MTF-tailored Multiscale Fusion of High-resolution MS and Pan Imagery" American Society for Photogrammetry and Remote Sensing 72 (72): 591-596, 2006

15 정남기 ; 정형섭 ; 오관영 ; 박숭환 ; 이승찬, "KOMPSAT-2/3/3A호의 영상융합에 대한 품질평가 프로토콜의 비교분석" 대한원격탐사학회 32 (32): 453-469, 2016

16 X. Otazu, "Introduction of sensor spectral response into image fusion methods. Application to wavelet-based methods" Institute of Electrical and Electronics Engineers (IEEE) 43 (43): 2376-2385, 2005

17 B. Aiazzi, "Improving Component Substitution Pansharpening Through Multivariate Regression of MS $+$Pan Data" Institute of Electrical and Electronics Engineers (IEEE) 45 (45): 3230-3239, 2007

18 Z. Wang, "Image Quality Assessment: From Error Visibility to Structural Similarity" Institute of Electrical and Electronics Engineers (IEEE) 13 (13): 600-612, 2004

19 Wen Dou, "Image Degradation for Quality Assessment of Pan-Sharpening Methods" MDPI AG 10 (10): 154-, 2018

20 Wald, L., "Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images" 63 (63): 691-699, 1997

21 Ranchin, T., "Fusion of high spatial and spectral resolution images: The ARSIS concept and its implementation" 66 (66): 49-61, 2000

22 Yuhas, R. H., "Discrimination among semi-arid landscape endmembers using the spectral angle mapper (SAM) algorithm" 147-149, 1992

23 Xu, S., "Deep gradient projection networks for pansharpening" 1366-1375, 2021

24 Vivone, G., "Contrast and error-based fusion schemes for multispectral image pansharpening" 11 (11): 930-934, 2013

25 L.. Alparone, "Comparison of Pansharpening Algorithms: Outcome of the 2006 GRS-S Data-Fusion Contest" Institute of Electrical and Electronics Engineers (IEEE) 45 (45): 3012-3021, 2007

26 Rebecca, L., "Climate change: Arctic sea ice summer minimum"

27 IPCC, "Climate Change 2021: The Physical Science Basis. Contribution of Working Group I to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change" Cambridge University Press 3-32, 2021

28 D. Fasbender, "Bayesian Data Fusion for Adaptable Image Pansharpening" Institute of Electrical and Electronics Engineers (IEEE) 46 (46): 1847-1857, 2008

29 Chao Dong, "Accelerating the Super-Resolution Convolutional Neural Network" Springer International Publishing 391-407, 2016

30 Vicinanza, M. R., "A pansharpening method based on the sparse representation of injected details" 12 (12): 180-184, 2014

31 Coloma Ballester, "A Variational Model for P+XS Image Fusion" Springer Science and Business Media LLC 69 (69): 43-58, 2006

32 Frosti Palsson, "A New Pansharpening Algorithm Based on Total Variation" Institute of Electrical and Electronics Engineers (IEEE) 11 (11): 318-322, 2014

33 Shutao Li, "A New Pan-Sharpening Method Using a Compressed Sensing Technique" Institute of Electrical and Electronics Engineers (IEEE) 49 (49): 738-746, 2011

34 Gemine Vivone, "A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting Pansharpening With Classical and Emerging Pansharpening Methods" Institute of Electrical and Electronics Engineers (IEEE) 9 (9): 53-81, 2021

35 Jaewan Choi, "A New Adaptive Component-Substitution-Based Satellite Image Fusion by Using Partial Replacement" Institute of Electrical and Electronics Engineers (IEEE) 49 (49): 295-309, 2011

36 Qiangqiang Yuan, "A Multiscale and Multidepth Convolutional Neural Network for Remote Sensing Imagery Pan-Sharpening" Institute of Electrical and Electronics Engineers (IEEE) 11 (11): 978-989, 2018