Research on Apple Recognition Algorithm in Complex Orchard Environment Based on Deep Learning
Abstract
:1. Introduction
2. YOLOV5s Network
3. Data Processing
3.1. Data Collection Process and Image Sources
3.2. Production of Dataset
3.3. Enhanced Processing of Data
4. Improved Apple Recognition Algorithm
4.1. CSP_X Module [27]
4.2. Spatial Pyramid Pool SPP Module [28]
4.3. Soft NMS (Non-Maximum Suppression) Algorithm [31]
4.4. Improvement of Network Model
4.5. Optimization of Loss Function
4.5.1. Focal Loss [33]
4.5.2. CIoU Loss (Complete Cross-Over Loss)
5. Model Training
5.1. Experimental Conditions
5.2. Evaluation Index System
5.3. Training Process
5.4. Analysis of Training Data
6. Test Results
6.1. Evaluation of Test Results
6.2. Recognition Results of Different Algorithms
6.3. Comparative Experiment under Different Fruit Number
6.4. Comparison Test under Different Light Conditions
7. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Jia, W.K.; Tian, Y.Y.; Luo, R.; Zhang, Z.; Lian, J.; Zheng, Y. Detection and segmentation of overlapped fruits based on optimized mask R-CNN application in apple harvesting robot. Comput. Electron. Agric. 2020, 172, 105380. [Google Scholar] [CrossRef]
- Fu, L.S.; Feng, Y.L.; Wu, J.Z.; Liu, Z.; Gao, F.; Majeed, Y.; Al-Mallahi, A.; Zhang, Q.; Li, R.; Cui, Y. Fast and accurate detection of kiwifruit in orchard using improved YOLOv3-tiny model. Precis. Agric. 2020, 22, 754–766. [Google Scholar] [CrossRef]
- Jia, W.K.; Zhang, Y.; Lian, J.; Zheng, Y.; Zhao, D.; Li, C. Apple harvesting robot under information technology: A review. Int. J. Adv. Robot. Syst. 2020, 17, 1–16. [Google Scholar] [CrossRef]
- Kang, H.W.; Chen, C. Fruit detection and segmentation for apple harvesting using visual sensor in orchards. Sensors 2019, 19, 4599. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Lin, G.C.; Tang, Y.C.; Zou, X.J.; Li, J.; **, C.; Zhongming, S.; Hui, L. Real-time detection methodology for obstacles in orchards using improved YOLOv4. Trans. Chin. Soc. Agric. Eng. (Trans. CSAE) 2021, 37, 36–43. [Google Scholar]
- Lin, T.-Y.; Goyal, P.; Girshick, R.; He, K.; Dollár, P. Focal Loss for Dense Object Detection. IEEE Trans. Pattern Anal. Mach. Intell. 2020, 42, 318–327. [Google Scholar] [CrossRef] [Green Version]
- Yu, Y.; Zhang, K.L.; Yang, L.; Zhang, D. Fruit detection for strawberry harvesting robot in non-structural environment based on Mask-RCNN. Comput. Electron. Agric. 2019, 163, 104846. [Google Scholar] [CrossRef]
Detection Algorithm | P/% | R/% | F1/% | AP1/ % | AP2/ % | Detection Speed F·s−1 |
---|---|---|---|---|---|---|
Faster RCNN | 91.5 | 73.1 | 81.0 | 92.5 | 85.2 | 16.5 |
RetinaNet | 89.8 | 81.8 | 85.5 | 92.7 | 88.6 | 26.3 |
CenterNet | 90.4 | 70.3 | 79.0 | 90.7 | 83.6 | 32.3 |
YOLOv5 | 91.7 | 92.0 | 91.8 | 95.1 | 95.0 | 25.6 |
Improved YOLOv5 | 91.0 | 92.6 | 91.8 | 96.4 | 96.3 | 27.8 |
Apples Number | Algorithm | P/ % | R/ % | F1/ % | AP1/ % | AP2/ % |
---|---|---|---|---|---|---|
Single | YOLOv5 | 92.4 | 98.9 | 95.5 | 98.8 | 99.6 |
Improved YOLOv5 | 98.1 | 95.7 | 96.9 | 98.5 | 98.8 | |
Multiple | YOLOv5 | 89.6 | 91.2 | 90.5 | 95.3 | 93.4 |
Improved YOLOv5 | 91.9 | 93.1 | 92.5 | 96.8 | 96.9 | |
Intensive | YOLOv5 | 89.9 | 85.2 | 87.5 | 94.2 | 91.0 |
Improved YOLOv5 | 89.6 | 92.8 | 91.2 | 96.0 | 95.0 | |
Wide field | YOLOv5 | 84.4 | 82.5 | 83.0 | 90.4 | 85.6 |
Improved YOLOv5 | 85.4 | 86.6 | 88.7 | 93.2 | 92.3 |
Light Condition | Detection Algorithm | P/ % | R/ % | F1/ % | AP1/ % | AP2/ % |
---|---|---|---|---|---|---|
Natural light | YOLOv5 | 91.3 | 95.4 | 93.3 | 97.3 | 96.8 |
Improved YOLOv5 | 95.2 | 94.7 | 94.9 | 97.7 | 97.7 | |
Side light | YOLOv5 | 91.6 | 95.6 | 93.6 | 97.4 | 97.0 |
Improved YOLOv5 | 95.6 | 94.8 | 95.2 | 98.0 | 97.9 | |
Back light | YOLOv5 | 90.7 | 94.5 | 92.6 | 96.9 | 96.0 |
Improved YOLOv5 | 94.6 | 94.0 | 94.3 | 97.3 | 97.2 | |
Night | YOLOv5 | 90.8 | 95.0 | 92.9 | 97.0 | 96.5 |
Improved YOLOv5 | 95.0 | 94.2 | 94.6 | 97.4 | 97.6 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Zhao, Z.; Wang, J.; Zhao, H. Research on Apple Recognition Algorithm in Complex Orchard Environment Based on Deep Learning. Sensors 2023, 23, 5425. https://doi.org/10.3390/s23125425
Zhao Z, Wang J, Zhao H. Research on Apple Recognition Algorithm in Complex Orchard Environment Based on Deep Learning. Sensors. 2023; 23(12):5425. https://doi.org/10.3390/s23125425
Chicago/Turabian StyleZhao, Zhuoqun, Jiang Wang, and Hui Zhao. 2023. "Research on Apple Recognition Algorithm in Complex Orchard Environment Based on Deep Learning" Sensors 23, no. 12: 5425. https://doi.org/10.3390/s23125425