Urban Land Cover Classification of High-Resolution Aerial Imagery Using a Relation-Enhanced Multiscale Convolutional Network
Abstract
:1. Introduction
- The proposed network uses a dense connectivity pattern to build the lightweight model. Meanwhile, parallel multi-kernel convolution and deconvolution modules are designed in the encoding and decoding stages, respectively, to increase the variety of receptive field sizes that can capture different scales objects more effectively.
- We introduce spatial and channel relation-enhanced blocks to explicitly model spatial and channel context relationships. They significantly improve the classification results by capturing global context information in the spatial and channel domains.
- We propose a relation-enhanced multiscale network for urban land cover classification in aerial imagery. Case studies with the ISPRS Vaihingen dataset and an area of Shanghai of about 143 km2 demonstrate that the proposed REMSNet can generate a better land cover classification accuracy and can produce more practical land cover maps compared with several state-of-the-art deep learning models.
2. Related Work
2.1. Semantic Segmentation for High-Resolution Aerial Images
2.2. Attention Mechanism
3. The Proposed Network
3.1. Proposed Network Architecture
3.2. Spatial Relation-Enhanced Block
3.3. Channel Relation Enhanced Block
3.4. Multiscale Fusion Module
4. Experimental Results and Analysis
4.1. Study Area and Preprocessing
4.1.1. Study area and data description
4.1.2. Data Preprocessing and Augmentation
4.2. Training Detail and Evaluation Metrics
4.3. Results and Analysis
4.3.1. Results of Vaihingen
4.3.2. Results of Shanghai
5. Discussion
5.1. Influence of the Different Modules on Classification
5.2. Model Size and Efficiency Analysis
6. Conclusions
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
References
- Patino, J.E.; Duque, J.C. A review of regional science applications of satellite remote sensing in urban settings. Comput. Environ. Urban Syst. 2013, 37, 1–17. [Google Scholar] [CrossRef]
- Qiu, C.; Mou, L.; Schmitt, M.; Zhu, X.X. Local climate zone-based urban land cover classification from multi-seasonal Sentinel-2 images with a recurrent residual network. ISPRS J. Photogramm. Remote Sens. 2019, 154, 151–162. [Google Scholar] [CrossRef] [PubMed]
- Yuan, F.; Sawaya, K.E.; Loeffelholz, B.C.; Bauer, M.E. Land cover classification and change analysis of the Twin Cities (Minnesota) Metropolitan Area by multitemporal Landsat remote sensing. Remote Sens. Environ. 2005, 98, 317–328. [Google Scholar] [CrossRef]
- Zhu, X.X.; Tuia, D.; Mou, L.; ** using a deep convolutional neural network with high spatial resolution multispectral remote sensing imagery. Remote Sens. Environ. 2018, 214, 73–86. [Google Scholar] [CrossRef]
- Belward, A.S.; Skøien, J.O. Who launched what, when and why; trends in global land-cover observation capacity from civilian earth observation satellites. ISPRS J. Photogramm. Remote Sens. 2015, 103, 115–128. [Google Scholar] [CrossRef]
- Pesaresi, M.; Huadong, G.; Blaes, X.; Ehrlich, D.; Ferri, S.; Gueguen, L.; Halkia, M.; Kauffmann, M.; Kemper, T.; Lu, L.; et al. A Global Human Settlement Layer from Optical HR/VHR RS Data: Concept and First Results. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2013, 6, 2102–2131. [Google Scholar] [CrossRef]
- Cheng, G.; Zhu, F.; ** a multi-filter convolutional neural network for semantic segmentation using high-resolution aerial imagery and LiDAR data. ISPRS J. Photogramm. Remote Sens. 2018, 143, 3–14. [Google Scholar] [CrossRef] [Green Version]
- Abadi, M.; Barham, P.; Chen, J.; Chen, Z.; Davis, A.; Dean, J.; Devin, M.; Ghemawat, S.; Irving, G.; Isard, M.; et al. TensorFlow: A system for large-scale machine learning. In Proceedings of the 12th USENIX conference on Operating Systems Design and Implementation, Savannah, GA, USA, 2–4 November 2016; pp. 265–283. [Google Scholar]
- Hinton, G.; Tieleman, T. Lecture 6.5-Rmsprop, Coursera: Neural Networks for Machine Learning; Technical Report; University of Toronto: Toronto, ON, Canada, 2012. [Google Scholar]
- Howard, A.G.; Zhu, M.; Chen, B.; Kalenichenko, D.; Wang, W.; Weyand, T.; Andreetto, M.; Adam, H. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. ar**v 2017, ar**v:1704.04861. [Google Scholar]
Method | Imp. Surf. | Buildings | Low Veg. | Tree | Car | mIoU | OA | |
---|---|---|---|---|---|---|---|---|
SVL-boosting + CRF [54] | 86.10 | 90.90 | 77.60 | 84.90 | 59.90 | 79.90 | - | 84.70 |
RF + dCRF [55] | 86.90 | 92.00 | 78.3 | 86.90 | 29.00 | 74.60 | - | 85.90 |
SegNet [31] | 92.27 | 93.73 | 77.35 | 86.65 | 62.76 | 81.27 | 70.40 | 87.42 |
MobileNet [59] | 90.29 | 94.13 | 83.13 | 84.07 | 77.19 | 85.56 | 75.56 | 87.76 |
FC-DenseNet [50] | 92.40 | 94.46 | 72.95 | 90.93 | 78.49 | 86.73 | 77.20 | 87.99 |
RotEqNet [56] | 89.50 | 94.80 | 77.50 | 86.50 | 72.60 | 84.18 | - | 87.50 |
HSNet [19] | 90.89 | 94.51 | 78.83 | 87.84 | 81.87 | 86.79 | - | 88.32 |
ENR [34] | 91.48 | 95.11 | 79.42 | 88.18 | 89.00 | 88.64 | - | 88.88 |
REMSNet | 92.07 | 95.95 | 83.53 | 90.13 | 81.48 | 89.07 | 80.73 | 90.46 |
Method | Farmland | Forestland | Building | Road | Arti. Stru. | Bare Soil | Water | mIoU | OA | |
---|---|---|---|---|---|---|---|---|---|---|
FC-DenseNet [50] | 94.92 | 62.52 | 91.49 | 78.38 | 63.99 | 88.45 | 92.74 | 79.60 | 67.55 | 85.20 |
SegNet [31] | 92.87 | 75.86 | 91.66 | 76.49 | 61.29 | 70.33 | 93.2 | 80.17 | 68.02 | 85.68 |
MobileNet [59] | 96.03 | 70.42 | 91.53 | 77.61 | 61.16 | 79.28 | 88.14 | 80.54 | 68.59 | 86.18 |
Deeplabv3 [15] | 95.28 | 77.27 | 89.34 | 72.93 | 58.60 | 76.77 | 89.86 | 80.90 | 68.97 | 86.19 |
PSPNet [30] | 93.67 | 76.02 | 91.44 | 75.74 | 63.43 | 76.29 | 93.00 | 81.71 | 70.05 | 86.36 |
REMSNet | 96.21 | 77.51 | 92.59 | 81.86 | 63.95 | 81.81 | 93.56 | 84.52 | 73.94 | 88.55 |
Method | OA (%) | mIoU (%) | |
---|---|---|---|
Basic network | 79.60 | 85.20 | 67.55 |
Basic network + SREB | 81.28 | 86.77 | 69.25 |
Basic network + SREB + CREB | 82.66 | 87.42 | 71.31 |
Basic network + SREB + CREB + MS | 84.52 | 88.55 | 73.94 |
Method | Parameters (M) | Time (s) | Mean (%) | OA (%) | mIoU (%) | |||
---|---|---|---|---|---|---|---|---|
Shang. | Vaihi. | Shang. | Vaihi. | Shang. | Vaihi. | |||
FC-DenseNet [50] | 7.86 | 0.0965 | 79.60 | 86.73 | 85.20 | 87.99 | 67.55 | 77.20 |
SegNet [31] | 34.96 | 0.0350 | 80.17 | 81.27 | 85.68 | 87.42 | 68.02 | 70.40 |
MobileNet [59] | 8.87 | 0.0325 | 80.54 | 85.56 | 86.18 | 87.76 | 68.59 | 75.56 |
Deeplabv3 [15] | 46.66 | 0.0130 | 80.90 | - | 86.19 | - | 68.97 | - |
PSPNet [30] | 55.99 | 0.0217 | 81.71 | - | 86.36 | - | 70.05 | - |
REMSNet | 10.56 | 0.1239 | 84.52 | 89.07 | 88.55 | 90.46 | 73.94 | 80.73 |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Liu, C.; Zeng, D.; Wu, H.; Wang, Y.; Jia, S.; **n, L. Urban Land Cover Classification of High-Resolution Aerial Imagery Using a Relation-Enhanced Multiscale Convolutional Network. Remote Sens. 2020, 12, 311. https://doi.org/10.3390/rs12020311
Liu C, Zeng D, Wu H, Wang Y, Jia S, **n L. Urban Land Cover Classification of High-Resolution Aerial Imagery Using a Relation-Enhanced Multiscale Convolutional Network. Remote Sensing. 2020; 12(2):311. https://doi.org/10.3390/rs12020311
Chicago/Turabian StyleLiu, Chun, Doudou Zeng, Hangbin Wu, Yin Wang, Shoujun Jia, and Liang **n. 2020. "Urban Land Cover Classification of High-Resolution Aerial Imagery Using a Relation-Enhanced Multiscale Convolutional Network" Remote Sensing 12, no. 2: 311. https://doi.org/10.3390/rs12020311