Two-stage Deep Learning Model with LSTM-based Autoencoder and CNN for Crop Classification Using Multi-temporal Remote Sensing Images = Two-stage Deep Learning Model with LSTM-based Autoencoder and CNN for Crop Classification Using Multi-temporal Remote Sensing Images

다국어 초록 (Multilingual Abstract)

This study proposes a two-stage hybrid classification model for crop classification using multi-temporal remote sensing images; the model combines feature embedding by using an autoencoder (AE) with a convolutional neural network (CNN) classifier to fully utilize features including informative temporal and spatial signatures. Long short-term memory (LSTM)-based AE (LAE) is fine-tuned using class label information to extract latent features that contain less noise and useful temporal signatures. The CNN classifier is then applied to effectively account for the spatial characteristics of the extracted latent features. A crop classification experiment with multi-temporal unmanned aerial vehicle images is conducted to illustrate the potential application of the proposed hybrid model. The classification performance of the proposed model is compared with various combinations of conventional deep learning models (CNN, LSTM, and convolutional LSTM) and different inputs (original multi-temporal images and features from stacked AE). From the crop classification experiment, the best classification accuracy was achieved by the proposed model that utilized the latent features by fine-tuned LAE as input for the CNN classifier. The latent features that contain useful temporal signatures and are less noisy could increase the class separability between crops with similar spectral signatures, thereby leading to superior classification accuracy. The experimental results demonstrate the importance of effective feature extraction and the potential of the proposed classification model for crop classification using multi-temporal remote sensing images.

참고문헌 (Reference)

1 곽근호, "작물 분류에서 시공간 특징을 고려하기 위한 2D CNN과 양방향 LSTM의 결합" 대한원격탐사학회 35 (35): 681-692, 2019

2 나상일, "국내 작황 모니터링을 위한 무인항공기 적용방안" 대한원격탐사학회 34 (34): 829-846, 2018

3 나상일, "계층분류 기법을 이용한 위성영상 기반의 동계작물 구분도 작성" 대한원격탐사학회 33 (33): 677-687, 2017

4 Kalinicheva, E., "Unsupervised satellite image time series clustering using object-based approaches and 3D convolutional autoencoder" 12 (12): 1816-, 2020

5 De Maesschalck, R., "The Mahalanobis distance" 50 (50): 1-18, 2000

6 Weiss, M., "Remote sensing for agricultural applications : A metareview" 236 : 111402-, 2020

7 Hinton, G. E., "Reducing the dimensionality of data with neural networks" 313 (313): 504-507, 2006

8 Kwak, G. -H., "Potential of hybrid CNN-RF model for early crop mapping with limited input data" 13 (13): 1629-, 2021

9 Geun-Ho Kwak, "Potential of Bidirectional Long Short-Term Memory Networks for Crop Classification with Multitemporal Remote Sensing Images" 대한원격탐사학회 36 (36): 515-525, 2020

10 Rußwurm, M., "Multi-temporal land cover classification with sequential recurrent encoders" 7 (7): 129-, 2018

11 이지혜, "MODIS 전천후 기상자료 기반의 생물리학적 벼 수량 모형 개발" 대한원격탐사학회 33 (33): 721-732, 2017

12 Zhou, Y., "Long-short-term-memory-based crop classification using high-resolution optical images and multi-temporal SAR data" 56 (56): 1170-1191, 2019

13 Hochreiter, S., "Long shortterm memory" 9 (9): 1735-1780, 1997

14 Hao, P., "Feature selection of time series MODIS data for early crop classification using random forest : A case study in Kansas, USA" 7 (7): 5347-5369, 2015

15 Guo, J., "Feature dimension reduction using stacked sparse auto-encoders for crop classification with multi-temporal, quad-pol SAR data" 12 (12): 321-, 2020

16 Zhong, L., "Deep learning based multi-temporal crop classification" 221 : 430-443, 2019

17 LeCun, Y., "Deep learning" 521 (521): 436-444, 2015

18 Li, K., "Cropland data fusion and correction using spatial analysis techniques and the Google Earth Engine" 57 (57): 1026-1045, 2020

19 Shi, X., "Convolutional LSTM network: A machine learning approach for precipitation nowcasting" 802-810, 2015

20 Bruzzone, L., "An extension of the Jeffreys-Matusita distance to multiclass cases for feature selection" 33 (33): 1318-1321, 1995

21 Demir, B., "An effective strategy to reduce the labeling cost in the definition of training sets by active learning" 11 (11): 79-83, 2014

22 Hamidi, M., "An auto-encoder based classifier for crop mapping from multitemporal multispectral imagery" 42 (42): 986-1016, 2020