PolSAR Image Land Cover Classification Based on Hierarchical Capsule Network
Abstract
:1. Introduction
- (1)
- The HCapsNet is proposed for land cover classification of PolSAR images. It can simultaneously consider the deep features obtained at different network levels, which describes the polarimetric scattering information of land covers more comprehensively with small training sample size, and significantly reduces the misclassification of class boundaries.
- (2)
- The CRF is combined with the HCapsNet to further refine the classification results. The intra-class misclassifications can be reduced by the spatial information constraints of the CRF.
- (3)
- We adopt three discriminative attributes for land covers of PolSAR data, i.e., phase, amplitude, and polarimetric decomposition, to uniformly describe the scattering mechanism of land covers with different sensors, bands, and resolutions. Moreover, the generalization performance of the proposed method is verified to be better than other comparison methods.
2. Methodology
2.1. Polarimetric Feature Extraction
2.2. Construction of the Primary Capsule Layer
2.3. Construction of the Higher-Level Capsule Layer
2.4. Conditional Random Field
2.5. Implementation Details
Algorithm 1 Proposed method |
|
3. Experiment Results and Analysis
3.1. Data Description and Parameter Settings
3.2. Classification Results of the AIRSAR Flevoland Dataset
3.3. Classification Results of the AIRSAR San Francisco Dataset
3.4. Classification Results of the GF-3 Dataset
3.5. Analysis of the Performance
4. Discussion
4.1. Contributions of Polarimetric Features
4.2. Comparison of Different Feature Extractors
4.3. Effect of the CRF
4.4. Generalization Performance
5. Conclusions
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
References
- Zhang, F.; Ni, J.; Yin, Q.; Li, W.; Li, Z.; Liu, Y.; Hong, W. Nearest-regularized subspace classification for PolSAR imagery using polarimetric feature vector and spatial information. Remote Sens. 2017, 9, 1114. [Google Scholar] [CrossRef] [Green Version]
- Li, L.; Liu, X.; Chen, Q.; Yang, S. Building damage assessment from PolSAR data using texture parameters of statistical model. Comput. Geosci. 2018, 113, 115–126. [Google Scholar] [CrossRef]
- Eom, K.B. Fuzzy clustering approach in unsupervised sea-ice classification. Neurocomputing 1999, 25, 149–166. [Google Scholar] [CrossRef]
- Gomez, L.; Alvarez, L.; Mazorra, L.; Frery, A.C. Fully PolSAR image classification using machine learning techniques and reaction-diffusion systems. Neurocomputing 2017, 255, 52–60. [Google Scholar] [CrossRef]
- ** using RADARSAT-2 C-Band polarimetric SAR data. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2017, 10, 3213–3224. [Google Scholar] [CrossRef]
- Wang, H.; Magagi, R.; Goita, K.; Jagdhuber, T. Refining a polarimetric decomposition of multi-angular UAVSAR time series for soil moisture retrieval over low and high vegetated agricultural fields. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2019, 12, 1431–1450. [Google Scholar] [CrossRef]
- Liu, J.w.; Ding, X.h.; Lu, R.k.; Lian, Y.f.; Wang, D.z.; Luo, X.l. Multi-View Capsule Network. In International Conference on Artificial Neural Networks; Springer: Cham, Switzerland, 2019; pp. 152–165. [Google Scholar]
- Yang, S.; Lee, F.; Miao, R.; Cai, J.; Chen, L.; Yao, W.; Kotani, K.; Chen, Q. RS-CapsNet: An Advanced Capsule Network. IEEE Access 2020, 8, 85007–85018. [Google Scholar] [CrossRef]
- Cheng, X.; He, J.; He, J.; Xu, H. Cv-CapsNet: Complex-valued capsule network. IEEE Access 2019, 7, 85492–85499. [Google Scholar] [CrossRef]
- Sabour, S.; Frosst, N.; Hinton, G.E. Dynamic routing between capsules. ar** and a deep capsule network. Remote Sens. 2019, 11, 626. [Google Scholar] [CrossRef]
- Zou, T.; Yang, W.; Dai, D.; Sun, H. Polarimetric SAR image classification using multifeatures combination and extremely randomized clustering forests. EURASIP J. Adv. Signal Process. 2009, 2010, 1–9. [Google Scholar] [CrossRef] [Green Version]
- Bi, H.; Sun, J.; Xu, Z. A graph-based semisupervised deep learning model for PolSAR image classification. IEEE Trans. Geosci. Remote Sens. 2018, 57, 2116–2132. [Google Scholar] [CrossRef]
- Wang, S.; Guo, Y.; Hua, W.; Liu, X.; Song, G.; Hou, B.; Jiao, L. Semi-Supervised PolSAR Image Classification Based on Improved Tri-Training With a Minimum Spanning Tree. IEEE Trans. Geosci. Remote Sens. 2020, 58, 8583–8597. [Google Scholar] [CrossRef]
- Liu, G.; Li, Y.; Jiao, L.; Chen, Y.; Shang, R. Multiobjective Evolutionary Algorithm Assisted Stacked Autoencoder for PolSAR Image Classification. Swarm Evol. Comput. 2020, 60, 100794. [Google Scholar] [CrossRef]
- Li, L.; Ma, L.; Jiao, L.; Liu, F.; Sun, Q.; Zhao, J. Complex contourlet-CNN for polarimetric SAR image classification. Pattern Recognit. 2020, 100, 107110. [Google Scholar] [CrossRef]
- Gu, J.; Wang, Z.; Kuen, J.; Ma, L.; Shahroudy, A.; Shuai, B.; Liu, T.; Wang, X.; Wang, G.; Cai, J.; et al. Recent advances in convolutional neural networks. Pattern Recognit. 2018, 77, 354–377. [Google Scholar] [CrossRef] [Green Version]
- Huang, G.; Liu, Z.; Van Der Maaten, L.; Weinberger, K.Q. Densely connected convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 4700–4708. [Google Scholar]
- Jiang, X.; Wang, Y.; Liu, W.; Li, S.; Liu, J. Capsnet, cnn, fcn: Comparative performance evaluation for image classification. Int. J. Mach. Learn. Comput. 2019, 9, 840–848. [Google Scholar] [CrossRef]
- Shotton, J.; Winn, J.; Rother, C.; Criminisi, A. Textonboost for image understanding: Multi-class object recognition and segmentation by jointly modeling texture, layout, and context. Int. J. Comput. Vis. 2009, 81, 2–23. [Google Scholar] [CrossRef] [Green Version]
- Liu, X.; Jiao, L.; Liu, F. PolSF: PolSAR image dataset on San Francisco. ar** using polarization fraction variation with temporal RADARSAT-2 C-band full-polarimetric SAR data over the Indian Himalayas. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2018, 11, 2192–2209. [Google Scholar] [CrossRef]
H | A | ||||||
phase | amplitude | polarimetric decomposition |
Platform | Windows 10 |
Keras/TensorFlow | V 2.2.4/V 1.13.1 |
CPU | Intel Core i7-10700K |
Memory | 16 G |
GPU | Nvidia GeForce RTX 2080 SUPER |
Video memory | 8 G |
Class | Name | Train | Test | Total |
---|---|---|---|---|
1 | Stem bean | 1.00% | 99.00% | 8764 |
2 | Rapeseed | 1.00% | 99.00% | 19,326 |
3 | Bare soil | 1.01% | 98.99% | 5340 |
4 | Potatoes | 1.00% | 99.00% | 17,758 |
5 | Wheat | 1.00% | 99.00% | 17,636 |
6 | Wheat2 | 1.00% | 99.00% | 10,371 |
7 | Peas | 1.01% | 98.99% | 10,417 |
8 | Wheat3 | 1.00% | 99.00% | 22,090 |
9 | Lucerne | 1.00% | 99.00% | 10,967 |
10 | Barely | 1.01% | 98.99% | 8601 |
11 | Grasses | 1.00% | 99.00% | 8365 |
12 | Beets | 1.00% | 99.00% | 10,161 |
13 | Buildings | 1.11% | 98.89% | 904 |
14 | Water | 1.01% | 98.99% | 3477 |
15 | Forest | 1.00% | 99.00% | 22,841 |
Total | 1.00% | 99.00% | 177,018 |
Class | Name | Train | Test | Total |
---|---|---|---|---|
1 | High-density urban | 0.50% | 99.50% | 163,370 |
2 | Vegetation | 0.50% | 99.50% | 157,698 |
3 | Water | 0.50% | 99.50% | 332,252 |
4 | Developed urban | 0.50% | 99.50% | 110,918 |
5 | Low-density urban | 0.51% | 99.49% | 12,263 |
Total | 0.50% | 99.50% | 776,501 |
Class | Name | Train | Test | Total |
---|---|---|---|---|
1 | Bare soil | 0.21% | 99.80% | 23,038 |
2 | Forest | 0.20% | 99.79% | 39,099 |
3 | Cole | 0.20% | 99.80% | 49,588 |
4 | Wheat | 0.20% | 99.80% | 35,121 |
5 | Grasses | 0.21% | 99.79% | 10,214 |
6 | Water | 0.20% | 99.80% | 5382 |
7 | Sand | 0.21% | 99.79% | 10,213 |
8 | Wetland | 0.56% | 99.44% | 893 |
Total | 0.20% | 99.80% | 173,550 |
Class | 1D-CNN | 2D-CNN | DenseNet | CapsNet | CapsNet-3 | Proposed Method |
---|---|---|---|---|---|---|
Steam bean | 94.57 | 99.23 | 99.48 | 99.10 | 99.15 | 99.29 |
Rapeseed | 89.77 | 95.32 | 95.31 | 96.16 | 98.15 | 98.84 |
Bare soil | 98.01 | 99.56 | 98.15 | 99.85 | 99.92 | 100.00 |
Potatoes | 86.52 | 96.55 | 95.37 | 99.45 | 99.24 | 99.72 |
Wheat | 90.84 | 96.86 | 95.83 | 98.57 | 97.16 | 99.84 |
Wheat2 | 83.68 | 96.27 | 83.14 | 96.39 | 96.26 | 98.28 |
Peas | 92.35 | 93.64 | 92.72 | 96.52 | 97.03 | 98.30 |
Wheat3 | 96.24 | 99.85 | 99.90 | 99.75 | 99.96 | 99.98 |
Lucerne | 92.60 | 96.16 | 96.41 | 95.43 | 95.31 | 97.08 |
Barley | 91.91 | 98.39 | 98.04 | 99.74 | 99.90 | 99.87 |
Grasses | 69.11 | 97.30 | 96.51 | 99.29 | 99.08 | 99.12 |
Beets | 89.92 | 93.84 | 95.90 | 96.19 | 96.05 | 97.19 |
Buildings | 68.36 | 92.17 | 6.97 | 74.50 | 88.37 | 91.92 |
Water | 64.91 | 88.99 | 56.57 | 88.03 | 88.52 | 99.57 |
Forest | 89.15 | 98.43 | 96.32 | 94.91 | 96.88 | 99.15 |
OA | 89.27 | 96.91 | 94.58 | 97.31 | 97.74 | 99.04 |
AA | 86.53 | 96.17 | 87.11 | 95.59 | 96.73 | 98.54 |
Kappa | 0.8824 | 0.9665 | 0.9406 | 0.9754 | 0.9783 | 0.9895 |
Class | 1D-CNN | 2D-CNN | DenseNet | CapsNet | CapsNet-3 | Proposed Method |
---|---|---|---|---|---|---|
High-density urban | 82.48 | 86.62 | 97.37 | 86.51 | 83.50 | 92.62 |
Vegetation | 91.49 | 92.83 | 85.14 | 90.56 | 96.64 | 97.32 |
Water | 99.56 | 97.70 | 99.94 | 99.80 | 99.64 | 99.96 |
Developed urban | 74.10 | 88.13 | 85.47 | 94.83 | 95.23 | 97.84 |
Low-density urban | 0.00 | 0.00 | 0.00 | 17.38 | 7.22 | 33.28 |
OA | 89.12 | 91.47 | 92.75 | 93.12 | 93.54 | 96.52 |
AA | 69.52 | 73.06 | 73.58 | 77.82 | 76.44 | 84.20 |
Kappa | 0.8460 | 0.8799 | 0.8974 | 0.9032 | 0.9091 | 0.9510 |
Class | 1D-CNN | 2D-CNN | DenseNet | CapsNet | CapsNet-3 | Proposed Method |
---|---|---|---|---|---|---|
Bare soil | 94.39 | 97.43 | 99.62 | 96.78 | 97.5 | 99.09 |
Forest | 98.85 | 99.96 | 99.86 | 99.99 | 99.98 | 100 |
Cole | 99.59 | 99.49 | 99.84 | 98.63 | 99.33 | 99.91 |
Wheat | 99.47 | 98.99 | 92.34 | 99.1 | 99.28 | 99.99 |
Grasses | 24.39 | 65.58 | 23.96 | 85.17 | 80.78 | 88.77 |
Water | 64.68 | 66.09 | 76.66 | 78.04 | 80.96 | 85.97 |
Sand | 69.58 | 98.23 | 75.31 | 93.81 | 95.56 | 99.58 |
Wetland | 47.48 | 85.44 | 39.87 | 93.06 | 96.53 | 95.74 |
OA | 91.16 | 96.04 | 91.36 | 97.04 | 97.32 | 98.71 |
AA | 74.8 | 88.9 | 75.93 | 93.07 | 93.74 | 96.13 |
Kappa | 0.8886 | 0.9504 | 0.8914 | 0.963 | 0.9665 | 0.9838 |
2D-CNN | CNN-CapsNet | HCapsNet (without CRF) | |
---|---|---|---|
Layer 1 | Input. 8 @ 15×15 | Input. 8 @ 15×15 | Input. 8 @ 15×15/“Reshape” |
Layer 2 | Conv. 128 @ 3×3 | Conv. 128 @ 3×3 | Conv. 128 @ 3×3/“Reshape” |
Layer 3 | Conv. 256 @ 3×3 | Conv. 256 @ 3×3 | Conv. 256 @ 3×3/“Reshape” |
Layer 4 | Max_pooling/Flatten | “Reshape” | Concatenate |
Layer 5 | Fully Connected | Primary Capsule | Primary Capsule |
Layer 6 | Dropout | Higher-level Capsule | Higher-level Capsule |
Layer 7 | Softmax | Length | Length |
Method | Sample Rate | OA | AA | Kappa |
---|---|---|---|---|
MCCNN [52] | 1% | 95.83% | 96.02% | / |
Compact and Adaptive CNNs [57] | 1% | 96.35% | / | / |
CK-HDRF [58] | 1% | 96.75% | 97.00% | 0.9569 |
RCV-CNN [13] | 1% | 97.22% | 95.99% | 0.8930 |
HCapsNet without CRF | 1% | 98.34% | 97.44% | 0.9818 |
HCapsNet with CRF | 1% | 99.04% | 98.54% | 0.9895 |
Data Set | Accuracy | Dense- CapsNet | Res- CapsNet | CNN- CapsNet | HCapsNet (without CRF) | HCapsNet (with CRF) |
---|---|---|---|---|---|---|
AIRSAR | OA | 97.40 | 97.67 | 97.88 | 98.34 | 99.04 |
Flevoland | AA | 95.45 | 96.52 | 96.21 | 97.44 | 98.54 |
dataset | Kappa | 0.9715 | 0.9745 | 0.9769 | 0.9818 | 0.9895 |
AIRSAR | OA | 91.44 | 93.89 | 93.88 | 94.34 | 96.52 |
San Francisco | AA | 73.08 | 75.75 | 76.11 | 82.52 | 84.20 |
dataset | Kappa | 0.8796 | 0.9136 | 0.9135 | 0.9204 | 0.9510 |
GF-3 | OA | 93.45 | 96.58 | 97.12 | 97.86 | 98.71 |
Hulunbuir | AA | 80.57 | 94.50 | 90.87 | 94.51 | 96.13 |
dataset | Kappa | 0.9176 | 0.9575 | 0.9641 | 0.9731 | 0.9838 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Cheng, J.; Zhang, F.; **ang, D.; Yin, Q.; Zhou, Y.; Wang, W. PolSAR Image Land Cover Classification Based on Hierarchical Capsule Network. Remote Sens. 2021, 13, 3132. https://doi.org/10.3390/rs13163132
Cheng J, Zhang F, **ang D, Yin Q, Zhou Y, Wang W. PolSAR Image Land Cover Classification Based on Hierarchical Capsule Network. Remote Sensing. 2021; 13(16):3132. https://doi.org/10.3390/rs13163132
Chicago/Turabian StyleCheng, Jianda, Fan Zhang, Deliang **ang, Qiang Yin, Yongsheng Zhou, and Wei Wang. 2021. "PolSAR Image Land Cover Classification Based on Hierarchical Capsule Network" Remote Sensing 13, no. 16: 3132. https://doi.org/10.3390/rs13163132