An Improved YOLOv5 Method to Detect Tailings Ponds from High-Resolution Remote Sensing Images
Abstract
:1. Introduction
3.1. Sample Boosting Strategy
3.2. Improvement of YOLOv5s Network Architecture
3.2.1. The Algorithm Principle of YOLOv5
3.2.2. Swin-T Backbone
3.2.3. RepGFPN Neck
3.2.4. Decoupled Head
3.3. Large-Scale Tailings Ponds Detection
3.3.1. Overlap** Slices of Large-Scale Imagery
3.3.2. Global Non-Maximum Suppression
3.4. Evaluation Methods
3.5. Experimental Environment
4. Results and Discussion
4.1. Experimental Results of GNMS
4.2. Comparative Results of Different Experiments
4.2.1. Qualitative Results
4.2.2. Quantitative Results
4.3. Discussion
4.3.1. Ablation Experiment
4.3.2. Comparison with Other Object Detection Methods
4.3.3. Limitations and Future Works
5. Conclusions
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
Abbreviations
ELAN | Efficient layer aggregation networks |
FPN | Feature Pyramid Network |
GF-1 | Gaofen-1 |
GF-6 | Gaofen-6 |
GNMS | Global non-maximum suppression |
LN | LayerNorm |
MSA | Multi-head self-attention |
NMS | Non-maximum suppression |
PAN | Path Aggregation Network |
RepGFPN | Reparameterized Generalized-FPN |
SBS | Sample boosting strategy |
SOTA | State-of-the-art |
SPPF | Spatial Pyramid Pooling Fast |
Swin-T backbone | Swin Transformer backbone |
SW-MSA | Shifted-window MSA |
W-MSA | Window-based MSA |
YOLO | You Only Look Once |
References
- Che, D.; Liang, A.; Li, X.; Ma, B. Remote Sensing Assessment of Safety Risk of Iron Tailings Pond Based on Runoff Coefficient. Sensors 2018, 18, 4373. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Komnitsas, K.; Kontopoulos, A.; Lazar, I.; Cambridge, M. Risk assessment and proposed remedial actions in coastal tailings disposal sites in Romania. Miner. Eng. 1998, 11, 1179–1190. [Google Scholar] [CrossRef]
- Yu, D.; Tang, L.; Ye, F.; Chen, C. A virtual geographic environment for dynamic simulation and analysis of tailings dam failure. Int. J. Digit. Earth 2021, 14, 1194–1212. [Google Scholar] [CrossRef]
- Morgan, G.; Gomes, M.V.P.; Perez-Aleman, P. Transnational governance regimes in the global south: Multinationals, states and NGOs as political actors. Rev. Adm. Empresas 2016, 56, 374–379. [Google Scholar] [CrossRef] [Green Version]
- Burritt, R.L.; Christ, K.L. Water risk in mining: Analysis of the Samarco dam failure. J. Clean. Prod. 2018, 178, 196–205. [Google Scholar] [CrossRef]
- **ao, R.; Shen, W.; Fu, Z.; Shi, Y.; **ong, W.; Cao, F. The application of remote sensing in the environmental risk monitoring of tailings pond: A case study in Zhangjiakou area of China. In Earth Resources and Environmental Remote Sensing/GIS Applications III; SPIE: Edinburgh, UK, 2012. [Google Scholar] [CrossRef]
- Hu, X.; Oommen, T.; Lu, Z.; Wang, T.; Kim, J.-W. Consolidation settlement of Salt Lake County tailings impoundment revealed by time-series InSAR observations from multiple radar satellites. Remote Sens. Environ. 2017, 202, 199–209. [Google Scholar] [CrossRef]
- Rotta, L.H.S.; Alcântara, E.; Park, E.; Negri, R.G.; Lin, Y.N.; Bernardo, N.; Mendes, T.S.G.; Filho, C.R.S. The 2019 Brumadinho tailings dam collapse: Possible cause and impacts of the worst human and environmental disaster in Brazil. Int. J. Appl. Earth Obs. Geoinf. 2020, 90, 102119. [Google Scholar] [CrossRef]
- Cheng, D.; Cui, Y.; Li, Z.; Iqbal, J. Watch Out for the Tailings Pond, a Sharp Edge Hanging over Our Heads: Lessons Learned and Perceptions from the Brumadinho Tailings Dam Failure Disaster. Remote Sens. 2021, 13, 1775. [Google Scholar] [CrossRef]
- Lévesque, J.; Neville, R.A.; Staenz, K.; Truong, Q.S. Preliminary results on the investigation of hyperspectral remote sensing for the identification of uranium mine tailings. In Proceedings of the ISSSR, Quebec City, QC, Canada, 10–15 June 2001; pp. 10–15. [Google Scholar] [CrossRef]
- Ma, B.; Chen, Y.; Zhang, S.; Li, X. Remote Sensing Extraction Method of Tailings Ponds in Ultra-Low-Grade Iron Mining Area Based on Spectral Characteristics and Texture Entropy. Entropy 2018, 20, 345. [Google Scholar] [CrossRef] [Green Version]
- Hao, L.; Zhang, Z.; Yang, X. Mine tailing extraction indexes and model using remote-sensing images in southeast Hubei Province. Environ. Earth Sci. 2019, 78, 1–11. [Google Scholar] [CrossRef] [Green Version]
- Liu, K.; Liu, R.; Liu, Y. A Tailings Pond Identification Method Based on Spatial Combination of Objects. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2019, 12, 2707–2717. [Google Scholar] [CrossRef]
- Wu, X. Image Extraction of Tailings Pond Guided by Artificial Intelligence Support Vector Machine. Wirel. Commun. Mob. Comput. 2022, 2022, 1–11. [Google Scholar] [CrossRef]
- Li, Q.; Chen, Z.; Zhang, B.; Li, B.; Lu, K.; Lu, L.; Guo, H. Detection of Tailings Dams Using High-Resolution Satellite Imagery and a Single Shot Multibox Detector in the **g–**–Ji Region, China. Remote Sens. 2020, 12, 2626. [Google Scholar] [CrossRef]
- Balaniuk, R.; Isupova, O.; Reece, S. Mining and Tailings Dam Detection in Satellite Imagery Using Deep Learning. Sensors 2020, 20, 6936. [Google Scholar] [CrossRef] [PubMed]
- Ferreira, E.; Brito, M.; Balaniuk, R.; Alvim, M.S.; Santos, J.A.D. Brazildam: A benchmark dataset for tailings dam detection. In Proceedings of the 2020 IEEE Latin American GRSS & ISPRS Remote Sensing Conference (LAGIRS), Santiago, Chile, 22–26 March 2020; IEEE: New York, NY, USA, 2020. [Google Scholar]
- Yan, D.; Li, G.; Li, X.; Zhang, H.; Lei, H.; Lu, K.; Cheng, M.; Zhu, F. An Improved Faster R-CNN Method to Detect Tailings Ponds from High-Resolution Remote Sensing Images. Remote Sens. 2021, 13, 2052. [Google Scholar] [CrossRef]
- Yan, D.; Zhang, H.; Li, G.; Li, X.; Lei, H.; Lu, K.; Zhang, L.; Zhu, F. Improved Method to Detect the Tailings Ponds from Multispectral Remote Sensing Images Based on Faster R-CNN and Transfer Learning. Remote Sens. 2021, 14, 103. [Google Scholar] [CrossRef]
- Lyu, J.; Hu, Y.; Ren, S.; Yao, Y.; Ding, D.; Guan, Q.; Tao, L. Extracting the Tailings Ponds From High Spatial Resolution Remote Sensing Images by Integrating a Deep Learning-Based Model. Remote Sens. 2021, 13, 743. [Google Scholar] [CrossRef]
- Tang, L.; Liu, X.; Wang, X.; Liu, S.; Deng, H. Statistical analysis of tailings ponds in China. J. Geochem. Explor. 2020, 216, 106579. [Google Scholar] [CrossRef]
- Lasac, M. Gaofen-6 Satellite. 2018. Available online: http://sasclouds.com/chinese/satellite/chinese/gf6 (accessed on 26 January 2023).
- Wang, J.; Cao, L.; Guo, Y.; Zhao, L.; Wu, B. Feature analysis and information identification of the iron tailings by high−multispectral remote sensing. J. Yunnan Univ. Nat. Sci. Ed. 2019, 41, 974–981. [Google Scholar]
- Fauvel, M.; Chanussot, J.; Benediktsson, J. A spatial–spectral kernel-based approach for the classification of remote-sensing images. Pattern Recognit. 2012, 45, 381–392. [Google Scholar] [CrossRef]
- Zhong, Y.; Fei, F.; Zhang, L. Large patch convolutional neural networks for the scene classification of high spatial resolution imagery. J. Appl. Remote Sens. 2016, 10, 25006. [Google Scholar] [CrossRef]
- Liu, Y.; Zhong, Y.; Fei, F.; Zhu, Q.; Qin, Q. Scene Classification Based on a Deep Random-Scale Stretched Convolutional Neural Network. Remote Sens. 2018, 10, 444. [Google Scholar] [CrossRef] [Green Version]
- Liu, Z.; Gao, X.; Wan, Y.; Wang, J.; Lyu, H. An Improved YOLOv5 Method for Small Object Detection in UAV Capture Scenes. IEEE Access 2023, 11, 14365–14374. [Google Scholar] [CrossRef]
- Wang, C.-Y.; Liao, H.-Y.M.; Wu, Y.-H.; Chen, P.-Y.; Hsieh, J.-W.; Yeh, I.-H. CSPNet: A new backbone that can enhance learning capability of CNN. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA, USA, 14–19 June 2020. [Google Scholar]
- Liu, Y.; He, G.; Wang, Z.; Li, W.; Huang, H. NRT-YOLO: Improved YOLOv5 Based on Nested Residual Transformer for Tiny Remote Sensing Object Detection. Sensors 2022, 22, 4953. [Google Scholar] [CrossRef] [PubMed]
- Zhu, X.; Lyu, S.; Wang, X.; Zhao, Q. TPH-YOLOv5: Improved YOLOv5 Based on Transformer Prediction Head for Object Detection on Drone-captured Scenarios. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada, 10–17 October 2021. [Google Scholar] [CrossRef]
- Yu, Y.; Zhao, J.; Gong, Q.; Huang, C.; Zheng, G.; Ma, J. Real-Time Underwater Maritime Object Detection in Side-Scan Sonar Images Based on Transformer-YOLOv5. Remote Sens. 2021, 13, 3555. [Google Scholar] [CrossRef]
- Liu, Z.; Lin, Y.; Cao, Y.; Hu, H.; Wei, Y.; Zhang, Z.; Lin, S.; Guo, B. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada, 10–17 October 2021. [Google Scholar]
- Lyu, C.; Zhang, W.; Huang, H.; Zhou, Y.; Wang, Y.; Liu, Y.; Zhang, S.; Chen, K. RTMDet: An Empirical Study of Designing Real-Time Object Detectors. ar**v 2022, ar**v:2212.07784. [Google Scholar]
- Xu, X.; Jiang, Y.; Chen, W.; Huang, Y.; Zhang, Y.; Sun, X. DAMO-YOLO: A Report on Real-Time Object Detection Design. ar**v 2022, ar**v:2211.15444. [Google Scholar]
- Song, G.; Liu, Y.; Wang, X. Revisiting the sibling head in object detector. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 13–19 June 2020. [Google Scholar]
- Wu, Y.; Chen, Y.; Yuan, L.; Liu, Z.; Wang, L.; Li, H.; Fu, Y. Rethinking classification and localization for object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 13–19 June 2020. [Google Scholar]
- Ge, Z.; Liu, S.; Wang, F.; Li, Z.; Sun, J. Yolox: Exceeding yolo series in 2021. ar**v 2021, ar**v:2107.08430. [Google Scholar]
- Li, C.; Li, L.; Jiang, H.; Weng, K.; Geng, Y.; Li, L.; Ke, Z.; Li, Q.; Cheng, M.; Nie, W.; et al. YOLOv6: A single-stage object detection framework for industrial applications. ar**v 2022, ar**v:2209.02976. [Google Scholar]
- Xu, S.; Wang, X.; Lv, W.; Chang, Q.; Cui, C.; Deng, K.; Wang, G.; Dang, Q.; Wei, S.; Du, Y.; et al. PP-YOLOE: An evolved version of YOLO. ar**v 2022, ar**v:2203.16250. [Google Scholar]
- Liu, J.; Chen, H.; Wang, Y. Multi-Source Remote Sensing Image Fusion for Ship Target Detection and Recognition. Remote Sens. 2021, 13, 4852. [Google Scholar] [CrossRef]
- Koga, Y.; Miyazaki, H.; Shibasaki, R. A CNN-Based Method of Vehicle Detection from Aerial Images Using Hard Example Mining. Remote Sens. 2018, 10, 124. [Google Scholar] [CrossRef] [Green Version]
- Xu, Y.; Zhu, M.; Li, S.; Feng, H.; Ma, S.; Che, J. End-to-End Airport Detection in Remote Sensing Images Combining Cascade Region Proposal Networks and Multi-Threshold Detection Networks. Remote Sens. 2018, 10, 1516. [Google Scholar] [CrossRef] [Green Version]
- Hosang, J.; Benenson, R.; Schiele, B. Learning non-maximum suppression. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017. [Google Scholar]
- Van Etten, A. You only look twice: Rapid multi-scale object detection in satellite imagery. ar**v 2018, ar**v:1805.09512. [Google Scholar]
Spectral Band | Wavelength (μm) | Spatial Resolution (m) | Swath Width (km) |
---|---|---|---|
Pan | 0.45–0.90 | 2 | 90 |
Blue | 0.45–0.52 | 8 | 90 |
Green | 0.52–0.60 | ||
Red | 0.63–0.69 | ||
NIR | 0.76–0.90 |
Device | Configuration |
---|---|
Operating system | Windows 10 (64-bit) |
Processor | Intel(R) Core(TM) i7-8750H at 3.80 GHz |
RAM | 16 G |
GPU accelerator | Cuda 10.2, cuDNN 7.6.4 |
GPU | NVIDIA RTX2070, 8 G |
Framework | PyTorch 1.8.1 |
Scripting language | Python 3.7 |
Hyperparameters | Value |
---|---|
training steps | 300 epochs |
warmup epoch | 3 |
warmup momentum | 0.8 |
batch size during training | 16 |
batch size during testing | 32 |
optimization algorithm | SGD |
initial learning rate | 0.01 |
momentum | 0.937 |
weight decay | 0.0005 |
Models | Precision | Recall | F1 | Iteration Time |
---|---|---|---|---|
YOLO v5 | 61.02% | 81.20% | 69.68% | 58.05 s |
YOLO v5+SBS | 78.34% | 75.54% | 76.91% | 58.07 s |
Improved YOLO v5+SBS | 86.00% | 78.18% | 81.90% | 166.01 s |
Model | Parameters (M) | [email protected] | Improvement over YOLOv5s |
---|---|---|---|
YOLOv5s (baseline) | 7.03 | 86.20% | - |
+Swin-T Backbone | 7.27 | 90.20% | 4% |
+RepGFPN Neck | 12.25 | 89.60% | 3.4% |
+Decoupled Head | 14.33 | 88.20% | 2% |
Ours | 19.82 | 92.15% | 5.95% |
Model | Parameters (M) | [email protected] |
---|---|---|
YOLOv5l | 46.11 | 87.60% |
YOLOv8s | 11.13 | 88.00% |
YOLTv5s | 7.06 | 88.60% |
Swin-T | 47.37 | 88.70% |
Ours | 19.82 | 92.15% |
Model | YOLOv5s | YOLOv8s | ||
---|---|---|---|---|
Parameters (M) | [email protected] | Parameters (M) | [email protected] | |
+Swin-T Backbone | 7.27 | 90.20% | 10.29 | 90.10% |
+RepGFPN Neck | 12.25 | 89.60% | 15.37 | 89.30% |
Improved Model | 19.82 | 92.15% | 14.51 | 90.06% |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Sun, Z.; Li, P.; Meng, Q.; Sun, Y.; Bi, Y. An Improved YOLOv5 Method to Detect Tailings Ponds from High-Resolution Remote Sensing Images. Remote Sens. 2023, 15, 1796. https://doi.org/10.3390/rs15071796
Sun Z, Li P, Meng Q, Sun Y, Bi Y. An Improved YOLOv5 Method to Detect Tailings Ponds from High-Resolution Remote Sensing Images. Remote Sensing. 2023; 15(7):1796. https://doi.org/10.3390/rs15071796
Chicago/Turabian StyleSun, Zhenhui, Peihang Li, Qingyan Meng, Yunxiao Sun, and Yaxin Bi. 2023. "An Improved YOLOv5 Method to Detect Tailings Ponds from High-Resolution Remote Sensing Images" Remote Sensing 15, no. 7: 1796. https://doi.org/10.3390/rs15071796