非结构农田环境机器视觉障碍物检测方法

doi:10.12398/j.issn.2096-7217.2021.02.001

摘要/Abstract

摘要： 非结构农田环境准确感知与理解，是农业生产自动化向自主化进程中的关键技术支撑。针对农田作业机器人障碍物感知检测技术，提出3个田间避障路径规划的基本原则。分析阐述非结构复杂农田背景下基于视觉检测田间障碍物的优劣势，概述目前基于单目视觉和双目视觉检测障碍物的主要方法及各方法检测田间障碍物的局限性。探讨基于机器视觉的田间障碍物检测技术的实现方式、优缺点对比和局限性等问题，指出基于视觉的多传感器融合是农田作业机器人环境感知、认知进而自主作业的发展方向。

关键词: 机器视觉, 农田环境, 障碍物检测, 自主作业

Abstract: Accurate perception and understanding of unstructured farmland environment is the technical key in agricultural production automation to autonomy. Aiming at the obstacle perception and detection technology of farmland operation robots, three basic principles of obstacle avoidance path planning are proposed in this paper. This paper analyzes and expounds on the advantages and disadvantages of field obstacle detection based on vision under the background of unstructured, complex farmland and summarizes the main methods of obstacle detection based on monocular vision and binocular vision and the limitations of each method. This paper discusses the implementation, advantages, disadvantages, and limitations of field obstacle detection technology based on machine vision and points out that visionbased multisensor fusion is the development direction of environmental perception, cognition, and autonomous operation of farmland operation robots.

Key words: machine vision, farmland environment, obstacle detection, autonomous operation

中图分类号:

孟志军, 颜丙新, 尹彦鑫, 王侨, 刘卉, 凌琳. 非结构农田环境机器视觉障碍物检测方法[J]. 智能化农业装备学报（中英文）, 2021, 2(2): 1-6.

Zhijun Meng, Bingxin Yan, Yanxin Yin, Qiao Wang, Hui Liu, Lin Ling. Machine vision obstacle detection method in #br# unstructured farmland environment[J]. Journal of Intelligent Agricultural Mechanization, 2021, 2(2): 1-6.

参考文献

［1］胡静涛, 高雷, 白晓平, 等. 农业机械自动导航技术研究进展［J］. 农业工程学报, 2015, 31(10): 1-10.
Hu Jingtao, Gao Lei, Bai Xiaoping, et al. Review of research on automatic guidance of agricultural vehicles ［J］. Transactions of the Chinese Society of Agricultural Engineering, 2015, 31(10): 1-10.
［2］刘洋成, 耿端阳, 兰玉彬, 等. 基于自动导航的农业装备全覆盖路径规划研究进展［J］. 中国农机化学报, 2020, 41(11): 185-192.
Liu Yangcheng, Geng Duanyang, Lan Yubin, et al. Research progress of agricultural equipment full coverage path planning based on automatic navigation ［J］. Journal of Chinese Agricultural Mechanization, 2020, 41(11): 185-192.
［3］周航, 杜志龙, 武占元, 等. 机器视觉技术在现代农业装备领域的应用进展［J］. 中国农机化学报, 2017, 38(11): 86-92.
Zhou Hang, Du Zhilong, Wu Zhanyuan, et al. Application progress of machine vision technology in the field of modern agricultural equipment ［J］. Journal of Chinese Agricultural Mechanization, 2017, 38(11): 86-92.
［4］ Wang N, Yeung D Y. Learning a deep compact image representation for visual tracking ［J］. Advances in Neural Information Processing Systems, 2013.
［5］张树刚. 基于超声波的移动机器人局部避障算法及应用［D］. 哈尔滨: 哈尔滨工业大学, 2013.
Zhang Shugang. Local obstacle avoidance algorithm and application of mobile robot based on ultrasonic ［D］. Harbin: Harbin Institute of Technology, 2013.
［6］宋怀波, 何东健, 辛湘俊. 基于机器视觉的非结构化道路检测与障碍物识别方法［J］. 农业工程学报, 2011, 27(6): 225-230.
Song Huaibo, He Dongjian, Xin Xiangjun. Unstructured road detection and obstacle recognition algorithm based on machine vision ［J］. Transactions of the Chinese Society of Agricultural Engineering, 2011, 27(6): 225-230.
［7］ Liu L, Cui J, Li J. Obstacle detection and classification in dynamical background ［J］. Aasri Procedia, 2012, 1: 435-440.
［8］ Christiansen P, Nielsen L N, Steen K A, et al. DeepAnomaly: Combining background subtraction and deep learning for detecting obstacles and anomalies in an agricultural field ［J］. Sensors, 2016, 16(11): 1904.
［9］黄铝文. 苹果采摘机器人视觉识别与路径规划方法研究［D］. 杨凌: 西北农林科技大学, 2013.
Huang Lvwen. Approach of visual identification and path planning for applepickingrobot ［D］. Yangling: Northwest A & F University, 2013.
［10］ AlKaff A, García F, Martín D, et al. Obstacle detection and avoidance system based on monocular camera and size expansion algorithm for UAVs ［J］. Sensors, 2017, 17(5): 1061.
［11］ Campos Y, Sossa H, Pajares G. Spatiotemporal analysis for obstacle detection in agricultural videos ［J］. Applied Soft Computing, 2016, 45: 86-97.
［12］程嘉煜. 基于机器视觉的农业机器人运动障碍检测及避障策略研究［D］. 南京: 南京农业大学, 2011.
Cheng Jiayu. Research on moving obstacle detection and avoidance strategy for agricultural robot based on machine vision ［D］. Nanjing: Nanjing Agricultural University, 2011.
［13］钟鹏飞. 基于机器视觉的非结构化道路识别与障碍物检测研究［D］. 广州: 华南农业大学, 2016.
Zhong Pengfei. Study on unstructured road recognition and obstacle detection based on machine vision ［D］. Guangzhou: South China Agricultural University, 2016.
［14］ Gharani P, Karimi H A. Contextaware obstacle detection for navigation by visually impaired ［J］. Image and Vision Computing, 2017, 64: 103-115.
［15］侯伟. 基于双目的前驶障碍物识别方法与应用研究［D］. 合肥: 合肥工业大学, 2015.
Hou Wei. Research on the method of the front obstacle recognition based on binocular and its application［D］. Hefei: Hefei University of Technology, 2015.
［16］周俊, 程嘉煜. 基于机器视觉的农业机器人运动障碍目标检测［J］. 农业机械学报, 2011, 42(8): 154-158.
Zhou Jun, Cheng Jiayu. Moving obstacle detection based on machine vision for agricultural mobile robot ［J］. Transactions of the Chinese Society for Agricultural Machinery, 2011, 42(8): 154-158.
［17］冉冉. 基于单目视觉的移动机器人目标识别及抓取系统研究［D］. 北京: 北京交通大学, 2010.
Ran Ran. Research on system of object recognition and pickup for mobile robot based on monocular vision ［D］. Beijing: Beijing Jiaotong University, 2010.
［18］ Lee T J, Yi D H, Cho D I. A monocular vision sensorbased obstacle detection algorithm for autonomous robots ［J］. Sensors, 2016, 16(3): 311.
［19］陈伟. 基于双目视觉的智能车辆道路识别与路径规划研究［D］. 西安: 西安理工大学, 2009.
Chen Wei. Research of road identification and obstacle detection for intelligent vehicle based on binocular vision ［D］. Xian: Xian University of Technology, 2009.
［20］ Hu Z, Uchimura K. UVdisparity: An efficient algorithm for stereovision based scene analysis ［C］. IEEE Proceedings. Intelligent Vehicles Symposium, 2005. IEEE, 2005: 48-54.
［21］ Zhang Z, Xu H, Chao Z, et al. A novel vehicle reversing speed control based on obstacle detection and sparse representation ［J］. IEEE Transactions on Intelligent Transportation Systems, 2014, 16(3): 1321-1334.
［22］周自维. 基于立体视觉的环境构建及机器人路径规划研究［D］. 哈尔滨: 哈尔滨工业大学, 2014.
Zhou Ziwei. Research on environment construction based on binocular vision system and path planning of a robot ［D］. Harbin: Harbin Institute of Technology, 2014.
［23］石金进. 基于视觉的智能车辆道路识别与障碍物检测方法研究［D］. 哈尔滨: 哈尔滨工业大学, 2017.
Shi Jinjin. Research on vision based road recognition and obstacle detection for intelligent vehicles ［D］. Harbin: Harbin Institute of Technology, 2016.
［24］ Neethu S, Vinuchackravarthy S. Object detection using binocular vision ［C］. 2016 International Conference on Advances in Computing, Communications and Informatics (ICACCI). IEEE, 2016: 1558-1564.
［25］苟琴. 基于双目视觉的未知环境下农田障碍物检测技术研究［D］. 杨凌: 西北农林科技大学, 2013.
Gou Qin. Research on obstacle detection in unknown farmland based on binocular vision ［D］. Yangling: Northwest A & F University, 2013.
［26］姬长英, 沈子尧, 顾宝兴, 等. 基于点云图的农业导航中障碍物检测方法［J］. 农业工程学报, 2015, 31(7): 173-179.
Ji Changying, Shen Ziyao, Gu Baoxing, et al. Obstacle detection based on point clouds in application of agricultural navigation ［J］. Transactions of the Chinese Society of Agricultural Engineering, 2015, 31(7): 173-179.
［27］ Huang Y, Young K. Binocular image sequence analysis: Integration of stereo disparity and optic flow for improved obstacle detection and tracking ［J］. EURASIP Journal on Advances in Signal Processing, 2008, 1-10.
［28］ Wang X, Fu W, Chen W. Detection of obstacle based on nocular vision ［C］. 2010 International Conference on Intelligent Computation Technology and Automation. IEEE, 2010, 2: 71-74.
［29］ Li Y, Shen X, Bei S. Realtime tracking method for moving target based on an improved Camshift algorithm ［C］. 2011 International Conference on Mechatronic Science, Electric Engineering and Computer (MEC). IEEE, 2011: 978-981.
［30］ Zhao M, Han B. The research of autonomous obstacle avoidance of mobile robot based on multisensor integration ［C］. Advanced Sensor Systems and Applications VII. International Society for Optics and Photonics, 2017, 10025: 1002514.
［31］ Chen Z, Luo X, Dai B. Design of obstacle avoidance system for microUAV based on binocular vision ［C］. 2017 International Conference on Industrial InformaticsComputing Technology, Intelligent Technology, Industrial Information Integration (ICIICII). IEEE, 2017: 67-70.
［32］邓勇军. 越障全位置自主焊接机器人视觉传感系统研究［D］. 上海: 上海交通大学, 2012.
Deng Yongjun. Study of vision sensing system for allposition autonomous welding robot with passing obstacles capability ［D］. Shanghai: Shanghai Jiao Tong University, 2012.
［33］傅豪. 基于装配机器人双目视觉系统的障碍物三维重建研究［D］. 西安: 西安理工大学, 2018.
Fu Hao. Research on 3D reconstruction of binocular vision system based on assembly robot ［D］. Xian: Xian University of Technology, 2018.
［34］佘宏杰. 基于视觉的AGV导航控制与障碍物检测研究［D］. 杭州: 浙江工业大学, 2016.
She Hongjie. The AGV guidance control and obstacle detection based on vision ［D］. Hangzhou: Zhejiang University of Technology, 2016.
［35］杜明芳. 基于视觉的自主车道路环境理解技术研究［D］. 北京: 北京理工大学, 2015.
Du Mingfang. Research on visionbased road environment understanding technology for autonomous vehicles ［D］. Beijing: Beijing Institute of Technology, 2015.
［36］李景彬, 陈兵旗, 刘阳. 棉花铺膜播种机导航路线图像检测方法［J］. 农业机械学报, 2014, 45(1): 40-45.
Li Jingbin, Chen Bingqi, Liu Yang. Image detection method of navigation route of cotton plastic film mulch planter ［J］. Transactions of the Chinese Society for Agricultural Machinery, 2014, 45(1): 40-45.
［37］ Ross P, English A, Ball D, et al. Noveltybased visual obstacle detection in agriculture ［C］. 2014 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2014: 1699-1705.
［38］ Ball D, Upcroft B, Wyeth G, et al. Visionbased obstacle detection and navigation for an agricultural robot ［J］. Journal of Field Robotics, 2016, 33(8): 1107-1130.

［39］ Hu J S, Juan C W, Wang J J. A spatialcolor meanshift object tracking algorithm with scale and orientation estimation ［J］. Pattern Recognition Letters, 2008, 29(16): 2165-2173.

［40］ Sun H, Zou H, Zhou S, et al. Surrounding moving obstacle detection for autonomous driving using stereo vision ［J］. International Journal of Advanced Robotic Systems, 2013, 10(6): 261.
［41］ Reid D. An algorithm for tracking multiple targets ［J］. IEEE Transactions on Automatic Control, 1979, 24(6): 843-854.
［42］ Ross D A, Lim J, Lin R S, et al. Incremental learning for robust visual tracking ［J］. International Journal of Computer Vision, 2008, 77(1): 125-141.
［43］ Dong W, Chang F, Zhao Z. Visual tracking with multifeature joint sparse representation ［J］. Journal of Electronic Imaging, 2015, 24(1): 013006.
［44］ Huang H T, Bi D Y, Hou Z Q, et al. Research of sparse representationbased visual object tracking: A survey ［J］. Zidonghua Xuebao/Acta Automatica Sinica, 2018, 44(10):1747-1763.
［45］ Hare S, Golodetz S, Saffari A, et al. Struck: Structured output tracking with kernels ［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 38(10): 2096-2109.
［46］ Zhang K, Song H. Realtime visual tracking via online weighted multiple instance learning ［J］. Pattern Recognition, 2013, 46(1): 397-411.
［47］ Kalal Z, Mikolajczyk K, Matas J. Trackinglearningdetection ［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 34(7): 1409-1422.
［48］ Henriques J F, Caseiro R, Martins P, et al. Highspeed tracking with kernelized correlation filters ［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 37(3): 583-596.
［49］ Danelljan M, Hger G, Khan F S, et al. Discriminative scale space tracking ［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 39(8): 1561-1575.
［50］ Danelljan M, Hager G, Shahbaz Khan F, et al. Learning spatially regularized correlation filters for visual tracking ［C］. Proceedings of the IEEE International Conference on Computer Vision, 2015: 4310-4318.
［51］ Valmadre J, Bertinetto L, Henriques J, et al. Endtoend representation learning for correlation filter based tracking ［C］. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 2805-2813.
［52］ Dairi A, Harrou F, Senouci M, et al. Unsupervised obstacle detection in driving environments using deeplearningbased stereovision ［J］. Robotics and Autonomous Systems, 2018, 100: 287-301.
［53］张治国. 前方道路行人检测和距离估计研究［D］. 武汉: 华中科技大学, 2017.
Zhang Zhiguo. Research on pedestrian detection and depth estimation of road scene ［D］. Wuhan: Huazhong University of Science & Technology, 2017.
［54］杨娟娟, 高晓阳, 李红岭, 等. 基于机器视觉的无人机避障系统研究［J］. 中国农机化学报, 2020, 41(2): 155-160.
Yang Juanjuan, Gao Xiaoyang, Li Hongling, et al. Research on UAV obstacle avoidance system based on machine vision ［J］. Journal of Chinese Agricultural Mechanization, 2020, 41(2): 155-160.
［55］耿乾, 毛鹏军, 李鹏举, 等. 障碍物分类识别的果园机器人避障方法研究［J］. 中国农机化学报, 2020, 41(8): 170-177.
Geng Qian, Mao Pengjun, Li Pengju, et al. Research on obstacle avoidance method of orchard robot based on obstacle classification ［J］. Journal of Chinese Agricultural Mechanization, 2020, 41(8): 170-177.