Crop image autonomous acquisition system based on ROS system

doi:10.12398/j.issn.2096-7217.2023.02.006

Abstract

Abstract: The research on monitoring methods of potato water stress status plays an important role in improving potato growth quality and yield. Based on thermal infrared and visible RGB images, the Crop Water Stress Index (CWSI) can be calculated and used to determine the status of crop water stress. To achieve automated, high-throughput, and non-destructive continuous collection of visible RGB, thermal infrared images, and raw temperature data, this study integrated and developed a crop inspection image acquisition system with binocular cameras and thermal infrared cameras. According to different functional requirements, the hardware parts were designed as different modules, including the main control module, motion control module, image acquisition module, and real-time monitoring module. The main control module implements photo taking control of the image acquisition module, transmission and storage of image and temperature data, and data interaction with the motion control module. Based on the robot operating system (ROS) architecture, multiple nodes were designed to communicate between each module node in publish/subscribe mode, including motion control node, thermal infrared camera node and binocular camera node. Through the one-day system feasibility test in the laboratory, it was concluded that the control accuracy of the motion module meets the needs of image acquisition. Each module and node control program can cooperate to complete image acquisition and local storage. Through the 18-day system stability test and application in the greenhouse, a total of 648 inspections were carried out, each inspection took 3 minutes, and 11 664 visible light images, 5 832 thermal infrared images and 5 832 original temperature data were obtained. It was proved that the system was running stably, the image and temperature data were collected normally, and the function of automatic, high-throughput and non-destructive acquisition of the required data have been realized. The system provides an effective technical and equipment support for crop phenotypic information acquisition.

Key words: image acquisition, crop phenotype, water stress, robot operating system, potatoes

CLC Number:

LI Yida, WANG Liuyang, HAN Yuxiao, LI Shuai, LI Han, ZHANG Man. Crop image autonomous acquisition system based on ROS system[J]. Journal of Intelligent Agricultural Mechanization, 2023, 4(2): 53-62.

References

［1］成升魁, 李云云, 刘晓洁, 等. 关于新时代我国粮食安全观的思考［J］. 自然资源学报, 2018, 33(6): 911-926.
CHENG Shengkui, LI Yunyun, LIU Xiaojie, et al. Thoughts on food security in China in the new period ［J］. Journal of natural resources, 2018, 33(6): 911-926.
［2］　张慧春, 周宏平, 郑加强, 等. 植物表型平台与图像分析技术研究进展与展望［J］. 农业机械学报, 2020, 51(3): 1-17.
ZHANG Huichun, ZHOU Hongping, ZHENG Jiaqiang, et al. Research progress and prospect in plant phenotyping platform and image analysis technology ［J］. Transactions of the Chinese Society for Agricultural Machinery, 2020, 51(3): 1-17.

［3］　GERLAND P, RAFTERY A E, EVCKOV H, et al. World population stabilization unlikely this century［J］. Science, 2014, 346(6206): 234-237.
［4］　张慧春, 王国苏, 边黎明, 等. 基于光学相机的植物表型测量系统与时序生长模型研究［J］. 农业机械学报, 2019, 50(10): 197-207.
ZHANG Huichun, WANG Guosu, BIAN Liming, et al. Visible camerabased 3D phenotype measurement system and timeseries visual growth model of plant ［J］. Transactions of the Chinese Society for Agricultural Machinery, 2019, 50(10): 197-207.
［5］　张德荣. 植物表型性状参数快速检测研究［D］. 杭州: 浙江大学, 2019.
ZHANG Derong. Study on rapid detection of phenotypic character parameters of plants ［D］. Hangzhou: Zhejiang University, 2019.
［6］　YANG W, FENG H, ZHANG X, et al. Crop phenomics and highthroughput phenotyping: Past decades, current challenges, and future perspectives ［J］. Molecular Plant, 2020, 13(2): 187-214.
［7］　WANG L, MIAO Y, HAN Y, et al. Extraction of 3D distribution of potato plant CWSI based on thermal infrared image and binocular stereovision system ［J］. Frontiers in Plant Science, 2022, 13.
［8］　徐凌翔, 陈佳玮, 丁国辉, 等. 室内植物表型平台及性状鉴定研究进展和展望［J］. 智慧农业(中英文), 2020, 2(1): 23-42.
XU Lingxiang, CHEN Jiawei, DING Guohui, et al. Indoor phenotyping platforms and associated trait measurement: Progress and prospects ［J］. Smart Agriculture, 2020, 2(1): 23-42.
［9］　程曼, 袁洪波, 蔡振江, 等. 田间作物高通量表型信息获取与分析技术研究进展［J］. 农业机械学报, 2020, 51(S1): 314-324.
CHENG Man, YUAN Hongbo, CAI Zhenjiang, et al. Review of Fieldbased Information Acquisition and Analysis of Highthroughput Phenotyping ［J］. Transactions of the Chinese Society of Agricultural Engineering, 2020, 51(S1): 314-324.
［10］　JOSHI S, THODAYKENNEDY E, DAETWYLER H D, et al. Highthroughput phenotyping to dissect genotypic differences in safflower for drought tolerance ［J］. Plos One, 2021, 16(7): e0254908.
［11］　DISSANAYAKE R, KAHROOD H V, DIMECH A M, et al. Development and application of imagebased highthroughput phenotyping methodology for salt tolerance in lentils ［J］. Agronomy, 2020, 10(12): 1992.
［12］　WANG Y, HU S, REN H, et al. 3D Pheno MVS: A lowcost 3D tomato phenotyping pipeline using 3D reconstruction point cloud based on multiview images ［J］. Agronomy, 2022, 12(8): 1865.
［13］　WU S, WEN W, WANG Y, et al. MVS-Pheno: a portable and lowcost phenotyping platform for maize shoots using multiview stereo 3D reconstruction ［J］. Plant Phenomics, 2020.
［14］　WU S, WEN W, GOU W, et al. A miniaturized phenotyping platform for individual plants using multiview stereo 3D reconstruction ［J］. Frontiers in Plant Science, 2022, 13.
［15］　GAO T, ZHU F, PAUL P, et al. Novel 3D imaging systems for highthroughput phenotyping of plants ［J］. Remote Sensing, 2021, 13(11): 2113.
［16］　CAPORASO N, WHITWORTH M B, FISK I D. NearInfrared spectroscopy and hyperspectral imaging for nondestructive quality assessment of cereal grains ［J］. Applied spectroscopy reviews, 2018, 53(8): 667-687.
［17］　GUO Z, YANG W, CHANG Y, et al. Genomewide association studies of image traits reveal genetic architecture of drought resistance in rice ［J］. Molecular Plant, 2018, 11(6): 789-805.
［18］　LANGSTROFF A, HEUERMANN M C, STAHL A, et al. Opportunities and limits of controlledenvironment plant phenotyping for climate response traits［J］. Theoretical and Applied Genetics, 2022, 135(1): 1-16.
［19］　CHEN D, NEUMANN K, FRIEDEL S, et al. Dissecting the phenotypic components of crop plant growth and drought responses based on highthroughput image analysis ［J］. The plant cell, 2014, 26(12): 4636-4655.
［20］　PIERUSCHKA R, SCHURR U. Plant phenotyping: past, present, and future ［J］. Plant Phenomics, 2019.
［21］　何勇, 李禧尧, 杨国峰, 等. 室内高通量种质资源表型平台研究进展与展望［J］. 农业工程学报, 2022, 38(17): 127-141.
HE Yong, LI Xiyao, YANG Guofeng, et al. Research progress and prospect of indoor highthroughput germplasm phenotyping platforms ［J］. Transactions of the Chinese Society of Agricultural Engineering, 2022, 38(17): 127-141.
［22］　郭庆华, 吴芳芳, 庞树鑫, 等. Crop 3D—基于激光雷达技术的作物高通量三维表型测量平台［J］. 中国科学: 生命科学, 2016, 46(10): 1210-1221.
GUO Qinghua, WU Fangfang, PANG Shuxin, et al. Crop 3D: a platform based on LiDAR for 3D highthroughput crop phenotyping ［J］. Scientia Sinica(Vitae), 2016, 46: 1210-1221.
［23］　ZHANG C, GAO H, ZHOU J, et al. 3D robotic system development for highthroughput crop phenotyping ［J］. IFAC-Papers OnLine, 2016, 49(16): 242-247.
［24］　DABEK P, SZREK J, ZIMROZ R, et al. An automatic procedure for overheated idler detection in belt conveyors using fusion of infrared and RGB images acquired during UGV robot inspection ［J］. Energies, 2022， 15(2): 601.

[1]	HUANG Chenglong, ZHANG Zhongfu, LU Zhihao, ZHANG Xiaojun, ZHU Longfu, YANG Wanneng. Leaf grading for cotton verticillium wilt based on VFNet-Improved and Deep Sort [J]. Journal of Intelligent Agricultural Mechanization, 2023, 4(2): 12-21.
[2]	YANG Liang, WANG Hui, CHEN Ruipeng, SHENG Qingkai, YUAN Zhen, XIONG Benhai. Advances in pig-specific sensors [J]. Journal of Intelligent Agricultural Mechanization, 2023, 4(2): 22-34.
[3]	XIAO Zhangna, LUO Lufeng, CHEN Mingyou, WANG Jinhai, LU Qinghua, LUO Shaoming. Detection of grapes in orchard environment based on improved YOLO-v4 [J]. Journal of Intelligent Agricultural Mechanization, 2023, 4(2): 35-43.
[4]	NIE Pengcheng, QIAN Cheng, QIN Ruimiao, DENG Shuiguang, SUN Chongde, HE Yong. Development status and trends of space-air-ground integrated information sensing and fusion technology [J]. Journal of Intelligent Agricultural Mechanization, 2023, 4(2): 1-11.
[5]	Tang Yurong, Shen Mingxia, Xue Hongxiang, Chen Jinxin. Development status and prospect of artificial intelligence technology in livestock and poultry breeding [J]. Journal of Intelligent Agricultural Mechanization, 2023, 4(1): 1-16.
[6]	Yan Kai, Gao Yue, Dai Baisheng, Sun Hongmin, Yin Yanling, Shen Weizheng. Research on detection of aggressive behavior among gregarious pigs based on anchor-free temporal localization [J]. Journal of Intelligent Agricultural Mechanization, 2023, 4(1): 17-25.
[7]	Han Guofeng, , Bai Zongchun, , Li Yansen, Li Chunmei. Current approaches and applications for in ovo sexing of eggs: A review [J]. Journal of Intelligent Agricultural Mechanization, 2023, 4(1): 26-35.
[8]	Yuan Zhiyu, Zhao Yunhui, Wang Song, Zhao Zhuo, Wu Yujin, Wang Chunxin. Development and application of management and control system for intellectual sheep breeding based on Internet of Things platform [J]. Journal of Intelligent Agricultural Mechanization, 2023, 4(1): 54-61.
[9]	Li Yibai, Cao Gunagqiao. Research on the scheduling model of flying defense team based on simulated annealing algorithm [J]. Journal of Intelligent Agricultural Mechanization (in Chinese and English), 2022, 3(2): 1-9.
[10]	Bai Xuefeng, Chang Jiangxue, Teng Zhaoli, Lu Zhixiong. Research progress on key technologies of intelligent agricultural tractors in China [J]. Journal of Intelligent Agricultural Mechanization (in Chinese and English), 2022, 3(2): 10-21.
[11]	Liu Jianlong, Bai Zongchun, Huo Lianfei, Meng Lili, Wang Huixin. Research on temperature and humidity spatial distribution of fermented mattress based on wireless sensor network [J]. Journal of Intelligent Agricultural Mechanization (in Chinese and English), 2022, 3(2): 22-27.
[12]	Wang Zhuoxuan, Wang Lihui, Xu Ninghui, Li Yong, Shi Jiachen. Research on the control strategy of sideslip and hysteresis of intelligent harvesting robot [J]. Journal of Intelligent Agricultural Mechanization (in Chinese and English), 2022, 3(2): 37-44.
[13]	Bai Yungang, Zhu Zhongzheng, Hou Yingyong, Zhou Lichun, Xiao Maohua, Zou Xiuguo. Design and test of wireless charging module for inspection robot in chicken farm [J]. Journal of Intelligent Agricultural Mechanization (in Chinese and English), 2022, 3(2): 45-52.
[14]	Wang Xingchen, Pan Wei, Zhou Lichun, Hou Yingyong, Petr Bartos, Xiao Maohua. Research on displacement compensation method of inspection robot based on CS-BP neural network [J]. Journal of Intelligent Agricultural Mechanization (in Chinese and English), 2022, 3(2): 53-63.
[15]	Zhang Lei, Liu Yiting, Chen Guangming, Li Peijuan. Research on navigation and rectification of inspection robot based on ultrasonic sensor [J]. Journal of Intelligent Agricultural Mechanization (in Chinese and English), 2022, 3(2): 64-70.