Fusion of Deep Learning Models for Improving Classification Accuracy of Remote Sensing Images


P. Deepan,L.R. Sudha,




Image classification,Remote sensing,Feature fusion,Convolutional neural network,Deep CNN and Ensemble classifier,


Over the recent years we have witnessed an increasing number of applications using deep learning techniques such as Convolutional Neural networks (CNNs), Recurrent Neural Networks (RNN) and Deep Neural Networks (DNN) for remote sensing image classification. But, we found that these models suffer for characterizing complex patterns in remote sensing imagery because of small inter class variations and large intra class variations. The intent of this paper is to study the effect of ensemble classifier constructed by combining three Deep Convolutional Neural Networks (DCNN) namely; CNN, VGG-16 and Res Inception models by using average feature fusion techniques. The proposed approach is validated with 7,000 remote sensing images from Northern Western Polytechnical University – Remote Sensing Image Scene Classification (NWPU- RESISC) 45 class dataset and confirmed as an effective technique to improve the robustness over a single deep learning model.


I. Ayhan E, and Kansu O, “Analysis of Image Classification methods for Remote
Sensing experimental Techniques”, Society for Experimental Mechanics, pp.18-
25, 2012.
II. Chan T H, Jia K, Gao S, Lu J, Zeng Z, and Ma Y, “PCANet: A simple deep
learning baseline for image classification?” IEEE Transaction on Image
Processing, pp.5017–5032, 2015.
III. Cheng G, Han J, and Lu X, “Remote Sensing Image Scene Classification:
Benchmark and State of the Art”, Proceedings of the IEEE, pp.1-19, 2017.
IV. Cheng C, and Han J, “A survey on object detection in optical remote sensing
images”, ISPRS Journal of Photogrammetry and Remote Sensing, pp.11-28,
V. Cheng J, Aurélien B, and Mark van der L, “The relative performance of
ensemble methods with deep convolutional neural networks for image
classification”, Journal of Applied Statistics, pp.1-19, 2018.
VI. Deepan P, and Sudha L R, “Object Detection in Remote Sensing Images: A
Review”, International Journal of Scientific Research in Computer Science
Applications and Management Studies, pp.1-10, 2018.
VII. Grant J, Richard A, Curt H, and Tyler W, “Fusion of Deep Convolutional Neural
Networks for Land Cover Classification of High-Resolution Imagery”, IEEE
Geoscience and Remote Sensing Letters, pp.1-5, 2017.
VIII. Krizhevsky A, Sutskever I, and Geoffrey E, “ImageNet Classification with Deep
Convolutional Neural Networks”, Proceedings, pp.1-9, 2015.
IX. Maarten C, Henri B, Noëlle M, and Klamer S, “Object recognition using deep
convolutional neural networks with complete transfer and partial frozen layers”,
Proceedings of Society of Photo-Optical Instrumentation Engineers (SPIE), pp.1-
7, 2016.
X. Maher Ibrahim S, Biswajeet P, and Omar Saud A, “Classification of Very High
Resolution Aerial Photos Using Spectral-Spatial Convolutional Neural
Networks”, Journal of Sensors, pp.1-12, 2018.
XI. Marmanis D, Datcu M, Esch T, and Stilla U, “Deep Learning Earth Observation
Classification Using Image Net Pre trained Networks”, IEEE Geoscience and
Remote Sensing Letters, pp.105-109, 2016.

XII. McInerney D O, and Nieuwenhuis M, “A comparative analysis of kNN and
decision tree methods for the irish national forest inventory”, International
Journal of Remote Sensing, pp.4937–4955, 2009.
XIII. Mountrakis G, Im J, and Ogole C, “Support vector machines in remote sensing: a
review”, ISPRS Journal of Photogrammetry Remote Sensing, pp.247–259, 2011.
XIV. Muhammet Ali D, “Deep Network Ensembles for Aerial Scene Classification”,
IEEE Geoscience and Remote Sensing Letters, pp.1-4, 2018.
XV. Qi K, Wu H, Shen C, and Gong J, “Land use scene classification in highresolution
remote sensing images using improved correlations,” IEEE Geosci.
Remote Sens. Lett., pp. 2403–2407, 2015.
XVI. Shuying L and Weihong D, “Very Deep Convolutional Neural Network Based
Image Classification Using Small Training Sample Size”, Third IAPR Asian
Conference on Pattern Recognition, pp1-5, 2015.
XVII. Souleyman C, Huan L, Yanfeng G, and Hongxun Y, “Deep Feature Fusion for
VHR Remote Sensing Scene Classification”, IEEE Transaction on Geoscience
and Remote Sensing, pp.1-10, 2017.
XVIII. Taylor L, and Nitschke G, “Improving Deep Learning using Generic Data
Augmentation”, Proceedings, pp.1-6, 2015.
XIX. Thanh Noi P, and Kappas M, “Comparison of Random Forest, k-Nearest
Neighbor, and Support Vector Machine Classifiers for Land Cover Classification
Using Sentinel-2 Imagery”, Journal on the science and technology of sensors,
pp.1-20, 2018.
XX. Williams T, and Li R, “An Ensemble of Convolutional Neural Networks Using
Wavelets for Image Classification”, Journal of Software Engineering and
Applications, pp.69-88, 2018.
XXI. Xingrui Y, Xiaomin W, Chunbo L and Peng R, “Deep learning in remote sensing
scene classification: a data augmentation enhanced convolutional neural network
framework”, Journal of GIScience and Remote Sensing, pp.1-18, 2017.
XXII. Yang L, Yiping G, Zhifeng X, and Qing L, “Accurate Object Localization in
Remote Sensing Images Based on Convolutional Neural Networks”, IEEE
Transactions On Geoscience and Remote Sensing, pp.2486-2498, 2017.
XXIII. Yansheng L, Chao T, Yihua T, Ke S, and Jinwen T, “Unsupervised Multilayer
Feature Learning for Satellite Image Scene Classification” IEEE Geoscience and
Remote Sensing Letters, pp.1-12, 2016.
XXIV. Zhao L, Tang P, and Huo L, “Feature significance- based multi bag-of-visualwords
model for remote sensing image scene classification,” Journal of Appl.
Remote Sens., pp.035-040, 2016.
XXV. Zhiling G, Qi C, Guangming W, Yongwei X, Ryosuke S and Xiaowei S,
“Village Building Identification Based on Ensemble Convolutional Neural
Networks”, Journal of Sensors, pp.1-22, 2017.

View Download