Leaf grading for cotton verticillium wilt based on VFNet-Improved and Deep Sort

doi:10.12398/j.issn.2096-7217.2023.02.002

Abstract

Abstract: Cotton is one of the most important economic crops in the world, and verticillium wilt is the number one disease in major cotton production areas in the world. Verticillium wilt causes the leaves to wilt, fade and even fall off by infecting the roots of cotton, resulting in severe decline both cotton quality and its yield. The national standard divides the leaves suffering from verticillium wilt into five grades. The traditional detection method mainly relies on manual labor, which has problems such as subjectivity, inefficiency, and poor repeatability. The cotton verticillium wilt disease classification method, which is based on VFNet-Improved, Deep Sort, and collision line matching mechanisms as the main algorithm framework, realizes the statistics of the number of diseased leaves and the classification of the disease level under the condition of rotating video input. Based on the VFNet target detection network, the research first combined multi-scale training, dynamic convolution and other optimization methods to achieve accurate positioning of diseased leaves in rotating videos. Then the Deep Sort tracker was used to realize the correlation of the same leaf in the front and back frames, and a mask collision line matching mechanism was designed for the ID jump problem in the tracking process; finally, OpenCV was used to perform feature extraction and disease classification of leaves passing the mask line. The results showed that the VFNet-Improved model could achieve the best effect in the object detection algorithms with 0.906 mAP75, which was 0.012 higher than the VFNet model, and the FPS reached 12.9 frames/s. The tracking result MOTA of the Deep Sort was 0.835. R2, RMSE, MAE and MAPE of VFNet-Improved were 0.890, 5.138, 4.300 and 14.967% respectively, which performed high consistency with manual measurements. In conclusion, this study has demonstrated a novel tool for the accurate and efficient evaluation of cotton verticillium wilt, which would be of great significance for the cotton breeding and genetic analysis research.

Key words: object detection, object tracking, VFNet, Deep Sort, cotton verticillium wilt, ill leaf grading

CLC Number:

S24: TP391.4

HUANG Chenglong, ZHANG Zhongfu, LU Zhihao, ZHANG Xiaojun, ZHU Longfu, YANG Wanneng. Leaf grading for cotton verticillium wilt based on VFNet-Improved and Deep Sort[J]. Journal of Intelligent Agricultural Mechanization, 2023, 4(2): 12-21.

References

［1］ LI N, YAO N, LI Y, et al. A metaanalysis of the possible impact of climate change on global cotton yield based on crop simulation approaches ［J］. Agricultural Systems, 2021, 193: 103221.
［2］　ZHANG Y, CHEN B, SUN Z W, et al. A largescale genomic association analysis identifies a fragment in Dt11 chromosome conferring cotton Verticillium wilt resistance ［J］. Plant Biotechnology Journal, 2021, 19(10): 2126-2138.
［3］　GONG Q, YANG Z E, WANG X Q, et al. Salicylic acidrelated cotton (Gossypium arboreum) ribosomal protein GaRPL18 contributes to resistance to Verticillium dahliae ［J］. BMC Plant Biology, 2017, 17(1): 1-15.

［4］　BARDAK A, ELIK S, ERDOGAN O, et al. Association mapping of Verticillium wilt disease in a worldwide collection of cotton (Gossypium hirsutum L.) ［J］. Plants, 2021, 10(2): 306.
［5］　DADDDAIGLE P, KIRKBY K, CHOWDHURY P R, et al. The verticillium wilt problem in Australian cotton ［J］. Australasian Plant Pathology, 2021, 50(2): 129-135.
［6］　SHABAN M, MIAO Y H, ULLAH A, et al. Physiological and molecular mechanism of defense in cotton against verticillium dahliae ［J］. Plant Physiology and Biochemistry, 2018, 125: 193-204.
［7］　TAO X Y, ZHANG H L, GAO M T, et al. Pseudomonas species isolated via highthroughput screening significantly protect cotton plants against verticillium wilt ［J］. AMB Express, 2020, 10: 1-12.
［8］　JING X, WANG J H, SONG X Y, et al. Continuum removal method for cotton verticillium wilt severity monitoring with hyperspectral data ［J］. Transactions of the Chinese Society of Agricultural Engineering, 2010, 26(1): 193-198.
［9］　JIN N, HUANG W J, REN Y, et al. Hyperspectral identification of cotton verticillium disease severity ［J］. OptikInternational Journal for Light and Electron Optics, 2013, 124(16): 2569-2573.
［10］　ZHANG N, PAN Y C, FENG H K, et al. Development of Fusarium head blight classification index using hyperspectral microscopy images of winter wheat spikelets ［J］. Biosystems Engineering, 2019, 186: 83-99.
［11］　CLEVERS J, KOOISTRA L. Using hyperspectral remote sensing data for retrieving canopy chlorophyll and nitrogen content ［J］. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2011, 5(2): 574-583.
［12］　陈兵, 王方永, 韩焕勇, 等. 基于光谱红边参数的棉花黄萎病叶片氮素含量诊断研究［J］. 棉花学报, 2013, 25(3): 254-261.
CHEN Bing, WANG Fangyong, HAN Huanyong, et al. Monitoring nitrogen contents in leaves of cotton under verticillium wilt stress based on spectra rededge parameters ［J］. Cotton Science, 2013, 25(3): 254-261.
［13］　陈巧玲. 基于遥感的棉花黄萎病病害严重度估测模型研究［D］. 乌鲁木齐: 新疆农业大学, 2021.
CHEN Qiaoling. Study on estimation model of cotton verticillium wilt disease severity based on remote sensing ［D］. Urumqi: Xinjiang Agricultural University, 2021.
［14］　HUANG C L, ZHANG Z F, ZHANG X J, et al. A novel intelligent system for dynamic observation of cotton verticillium wilt ［J］. Plant Phenomics, 2023, 5: 0013.
［15］　黄成龙, 柯宇曦, 华向东, 等. 边缘计算在智慧农业中的应用现状与展望［J］. 农业工程学报, 2022, 38(16): 224-234.
HUANG Chenglong, KE Yuxi, HUA Xiangdong, et al. Application status and prospect of edge computing in smart agriculture ［J］. Transactions of the Chinese Society of Agricultural Engineering, 2022, 38(16): 224-234.
［16］　孙宜田, 陈刚, 李青龙, 等. 智慧农业的应用及发展趋势与挑战［J］. 智能化农业装备学报（中英文）， 2020, 1(1): 56-59.
SUN Yitian, CHEN Gang, LI Qinglong, et al. Application and development trend of intelligent agriculture ［J］. Journal of Intelligent Agricultural Mechanization (in Chinese and English), 2020, 1(1): 56-59.
［17］　ZOU H W, LU H, LI Y N, et al. Maize tassels detection: A benchmark of the state of the art ［J］. Plant Methods, 2020, 16(1): 108.
［18］　姚青, 谷嘉乐, 吕军, 等. 改进RetinaNet的水稻冠层害虫为害状自动检测模型［J］．农业工程学报， 2020, 36(15): 182-188.
YAO Qing, GU Jiale, L Jun, et al. Automatic detection model for pest damage symptoms on rice canopy based on improved RetinaNet ［J］. Transactions of the Chinese Society of Agricultural Engineering, 2020, 36(15): 182-188.
［19］　刘洁, 李燕, 肖黎明, 等．基于改进YOLOv4模型的橙果识别与定位方法［J］．农业工程学报， 2022, 38(12): 173-182.
LIU Jie, LI Yan, XIAO Liming, et al. Recognition and location method of orange based on improved YOLOv4 model ［J］. Transactions of the Chinese Society of Agricultural Engineering, 2022, 38(12): 173-182.
［20］　FERENTINOS K P. Deep learning models for plant disease detection and diagnosis ［J］. Computers and Electronics in Agriculture, 2018, 145: 311-318.
［21］　LOEY M, ELSAWY A, AFIFY M. Deep learning in plant diseases detection for agricultural crops: A survey ［J］. International Journal of Service Science, Management, Engineering, and Technology (IJSSMET), 2020, 11(2): 41-58.
［22］　王宁, 席茂, 周文罡. 深度视觉目标跟踪进展综述［J］. 中国科学技术大学学报, 2021, 51(4): 335-344.
WANG Ning, XI Mao, ZHOU Wengang. Recent advance in deep visual object tracking ［J］. Journal of University of Science and Technology of China, 2021, 51(4): 335-344.
［23］　MOCANU B, TAPU R, ZAHARIA T. Single object tracking using offline trained deep regression networks ［C］∥2017 Seventh International Conference on Image Processing Theory, Tools and Applications (IPTA). IEEE, 2017: 1-6.
［24］　LI X, WANG K J, WANG W, et al. A multiple object tracking method using Kalman filter ［C］∥ The 2010 IEEE International Conference on Information and Automation. IEEE, 2010: 1862-1866.
［25］　宋怀波, 牛满堂, 姬存慧. 基于视频分析的多目标奶牛反刍行为监测［J］. 农业工程学报, 2018, 34(18): 211-218.
SONG Huaibo, NIU Mantang, JI Cunhui. Monitoring of multitarget cow ruminant behavior based on video analysis technology ［J］. Transactions of the Chinese Society of Agricultural Engineering, 2018, 34(18): 211-218.
［26］　文韬, 洪添胜, 李震．基于机器视觉的橘小实蝇运动轨迹跟踪与数量检测［J］．农业工程学报, 2011, 27(10): 137-141.
WEN Tao, HONG Tiansheng, LI Zhen. Statistics and tracking of Bactrocera Dorsalis based on machine vision ［J］. Transactions of the Chinese Society of Agricultural Engineering, 2011, 27(10): 137-141.
［27］　WOJKE N, BEWLEY A, PAULUS D. Simple online and realtime tracking with a deep association metric ［C］∥2017 IEEE International Conference on Image Processing (ICIP). IEEE, 2017: 3645-3649.
［28］　BEWLEY A, GE Z Y, OTT L, et al. Simple online and realtime tracking ［C］∥2016 IEEE International Conference on Image Processing (ICIP). IEEE, 2016: 3464-3468.
［29］　ZHANG H Y, WANG Y, DAYOUB F, et al. VarifocalNet: An IoUaware dense object detector ［C］∥ Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 8514-8523.
［30］　TIAN Z, SHEN C H, CHEN H, et al. FCOS: Fully convolutional onestage object detection ［C］∥Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019: 9627-9636.
［31］　ZHANG S F, CHI C, YAO Y Q, et al. Bridging the gap between anchorbased and anchorfree detection via adaptive training sample selection ［C］∥ Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020: 9759-9768.
［32］　HE K M, ZHANG X Y, REN S Q, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1904-1916.
［33］　REN S M, HE K M, GIRSHICK R, et al. Object detection networks on convolutional feature maps ［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 39(7): 1476-1481.
［34］　GOODFELLOW I, WARDEFARLEY D, MIRZA M, et al. Maxout networks ［C］∥International Conference on Machine Learning. PMLR, 2013: 1319-1327.
［35］　黄成龙, 张忠福, 华向东, 等. 基于改进Faster R-CNN和Deep Sort的棉铃跟踪计数［J/OL］.农业机械学报:1-10［2023-04-20］.http://kns.cnki.net/kcms/detail/11.1964.S.20230420.1137.004.html
HUANG Chenglong, ZHANG Zhongfu, HUA Xiangdong, et al. Cotton boll tracking and counting based on improved Faster RCNN and Deep Sort ［J/OL］. Transactions of the Chinese Society for Agricultural Machinery, 1-10［2023-04-20］.http://kns.cnki.net/kcms/detail/11.1964.S.20230420.1137.004.html
［36］　刘婉茹, 张国忠, 周勇, 等. 智能化技术在水稻生产全程机械化中的应用研究与发展趋势［J］. 华中农业大学学报, 2022, 41(1): 105-122.
LIU Wanru， ZHANG Guozhong， ZHOU Yong， et al. Application and development of intelligent technology in full mechanization of rice production ［J］. Journal of Huazhong Agricultural University, 2022, 41(1): 105-122.
［37］　陈柯屹, 朱龙付, 宋鹏, 等. 融合动态机制的改进型Faster R-CNN识别田间棉花顶芽［J］. 农业工程学报, 2021, 37(16): 161-168.
CHEN Keyi， ZHU Longfu， SONG Peng， et al. Recognition of cotton terminal bud in field using improved Faster R-CNN by integrating dynamic mechanism ［J］. Transactions of the Chinese Society of Agricultural Engineering, 2021, 37(16): 161-168.
［38］　黄成龙, 李曜辰, 骆树康, 等. 基于结构光三维点云的棉花幼苗叶片性状解析方法［J］. 农业机械学报, 2019, 50(8): 243-248， 288.
HUANG Chenglong， LI Yaochen， LUO Shukang， et al. Cotton seedling leaf traits extraction method from 3D point cloud based on structured light imaging ［J］. Transactions of the Chinese Society for Agricultural Machinery, 2019, 50(8): 243-248， 288.
［39］　黄成龙, 杨万能, 吴迪, 等. 基于X-ray透射成像的稻穗米粒粒型及谷粒饱满度测量［J］. 中国农业科技导报, 2018, 20(8): 46-53.
HUANG Chenglong， YANG Wanneng， WU Di， et al. Grain size and filling degree extraction for rice panicle based on X-ray imaging ［J］. Journal of Agricultural Science and Technology, 2018, 20(8): 46-53.

[1]	XIAO Zhangna, LUO Lufeng, CHEN Mingyou, WANG Jinhai, LU Qinghua, LUO Shaoming. Detection of grapes in orchard environment based on improved YOLO-v4 [J]. Journal of Intelligent Agricultural Mechanization, 2023, 4(2): 35-43.
[2]	LI Qianxi, SUN Xiaoming, JIANG Hanhui, WU Airu, FU Longsheng, LI Rui. Design and test of intelligent spraying unmanned vehicle for greenhouse tomato based on YOLOv4-tiny [J]. Journal of Intelligent Agricultural Mechanization, 2023, 4(2): 44-52.