Path Planning of Green Walnut Picking Robotic Arm Based on HER-TD3 Algorithm
Abstract
In response to the common problems of complex environments, large training tasks, and poor stability caused by the disorder growth of green walnut and tree branches, etc., a harvesting device based on synchronous belt module and manipulator was designed, and the path planning of harvesting manipulator was carried out by using the twin delayed deep deterministic policy gradient with hindsight experience replay (HER-TD3) algorithm. HER algorithm was used to improve the agent’s ability of exploration and alleviate the problem of sparse reward, and TD3 algorithm was used to improve the agent’s stability and reduce the oscillation in training. In order to demonstrate the feasibility and generalization ability of the HER-TD3 algorithm, TD3 and HER-DDPG algorithms were introduced for comparison. Three deep reinforcement learning agents were trained by using dimensionality reduction training methods. The results showed that the success rate of the HER-TD3 algorithm model in completing path planning tasks reached 98%, which was 4 percentage points higher than that of the HER-DDPG algorithm and 19 percentage points higher than that of TD3. The 3D model simulation environment was built in CoppeliaSim software, and the initial attitude and collision detection were designed, YOLO v4 was used to recognize green walnuts, and used this algorithm model to guide the virtual harvesting robotic arm to avoid tree branches and obstacles to reach the target position, completing collision free path planning. The success rates of path planning were 91% in the absence of obstacles and 86% in the presence of obstacles. In the experiment of picking green walnut using a physical prototype, the path planning task was still well completed. The success rate of path planning for harvesting without obstacles was 86.7%, with an average motion time of 12.8s, while the success rate in the presence of obstacles was 80.0%, with an average motion time of 13.6s. It was verified that HER-TD3 algorithm had good adaptability and stability to complex environment.
Keywords:green walnut;picking robot;robotic arm;HER-TD3 algorithm;path planning
Download Full Text:
PDFReferences
WEI Zhongcai, LI Hongwen, SUN Chuanzhu, et al. Design and experiment of potato combined harvester based on multi-stage separation technology [ J ]. Transactions of the Chinese Society for Agricultural Machinery, 2019, 50( 1 ) ; 129 - 140,112. ( in Chinese)
WANG Xiangyou, L0 Danyang, HEN Jiayi, et al. Development of a cleaning device for a bagged potato combine harvester [J] . Transactions of the CSAE, 2022,38 ( Supp. 1); 8 — 17. (in Chinese)
WANG Xiangyou, SUN Jingbin, XU Yingchao, et al. Design and experiment of potato cleaning and sorting machine [J]. Transactions of the Chinese Society for Agricultural Machinery, 2017, 48( 10) ; 316 -322,279. (in Chinese)
LtJ Jinqing, TIAN Zhongen, YANG Ying, et al. Design and experimental analysis of4U2A type double-row potato digger [J ] . Transactions of the CSAE, 2015, 31(6): 17 -24. (in Chinese)
LIU Xing, YANG Zhen, XUE Jian, et al. Progress of potato research combine harvester [J] . Journal of Agricultural Mechanization Research, 2022, 44(5) : 259 -263, 268. (in Chinese)
WEI Hongan, ZHANG Junlian, YANG Xiaoping, et al. Improved design and test of 4UFD - 1400 type potato combine harvester [J ]. Transactions of the CSAE, 2014, 30(3) : 12 - 17. (in Chinese)
WEI Zhongcai, WANG Yewei, LI Xueqiang, et al. Design and experiments of the potato combine harvester with elastic rubbing technology [ J ]. Transactions of the CSAE, 2023, 39( 14) ; 60 -69. (in Chinese)
ZHU Qianfeng, LU Rongjian, LIU Bin, el al. Research status and development trends of walnut picking machinery, [J ]. Forestry and Grassland Machinery, 2021 , 2(1); 45 -53. (in Chinese)
LIU Chengliang, GONG Liang, YUAN Jin, et al. Current status and development trends of agricultural robots [J ]. Transactions of the Chinese Society for Agricultural Machinery, 2022,53(7) :1 -22,55. (in Chinese)
XUN Yi, LI Daozheng, WANG Yong, et al. Path planning of picking robot arms based on VS — 1RRT algorithm [ J]. Transactions of the Chinese Society for Agricultural Machinery, 2023,54(2) ; 129 - 138. (in Chinese)
XIONG Juntao, LI Zhongxing, CHEN Shumian, et al. Virtual robot picking path obstacle avoidance planning based on deep reinforcement learning[ J ]. Transactions of the Chinese Society for Agricultural Machinery, 2020,51 ( Supp. 2 ) : 1 - 10. (in Chinese)
RENGANATHAN V, FATHIAN K, SAFAOUI S, et al. Spoof resilient coordination in distributed and robust robotic networks [J ]. IEEE Transactions on Control Systems Technology, 2022,30(2) ;803 -810.
LUO M, HOU X, YANG J. Surface optimal path planning using an extended Dijkstra algorithm [ J] . IEEE Access,2020, 8; 147827 - 147838.
VARGAS A N, FURLONI W, DOVAL J. Second moment constraints and the control problem of Markov jump linear systems [J ]. Numerical Linear Algebra with Applications, 2013, 20(2) : 357 -368.
LI Fenglei, HUANG Zexin, XU Lin. Path planning of 6 — DOF venipuncture robot arm based on improved А-star and collision detection algorithms [ С ]//2019 IEEE International Conference on Robotics and Biomimetics ( ROBIO). IEEE,2019: 2971 - 2976.
HENTEN E J V, HEMMING J, TUIJL В A J V, et al. Collision-free motion planning fora cucumber picking robot [ J ]. Biosystems Engineering, 2003, 86(2) ; 135 - 144.
YIN Jianjun, WU Chuanyu, YANG Simon X, et al. Obstacle avoidance path planning for tomato picking robot robotic arms [J] . Transactions of the Chinese Society for Agricultural Machinery, 2012, 43( 12) ; 171 - 175,157. (in Chinese)
JI Wei, CHENG Fengyi, ZHAO Dean, et al. Obstacle avoidance method for apple picking robot manipulator based on improved artificial potential field: J ] Transactions of the Chinese Society for Agricultural Machinery, 2013, 44 ( 11 ) : 263 - 269. (in Chinese)
XUE Vang, YU Zhicheng, WU Haidong, et al. Obstacle avoidance path planning for dual robotic arms based on improved artificial potential field method [ J ]. Journal of Mechanical Transmission, 2020,44(3) : 39 -45. (in Chinese)
CAO Xiaoman, ZUO Xiangjun, JIA Chunyang, et al. RRT-based path planning for an intelligent litchi-picking manipulator [J ]. Computers and Electronics in Agriculture ,2019 ,156 ; 105 - 118.
WANG Huaizhen, GAO Ming, WANG Jianhua, et al. Multi scene motion planning of robotic arms based on improved RRT* — Connect algorithm;[ J ]. Transactions of the Chinese Society for Agricultural Machinery, 2022, 53(4) ; 432 -440. (in Chinese)
JEONG I B, LEE S J, KIM J H. Quick-RRT : triangular inequality-based implementation of RRT with improved initial solution and convergence rate [j]. Expert Systems with Applications, 2019, 123(6): 82 -90.
ZHANG Qin, LE Xiaoliang, LI Bin, et al. Motion path planning of fruit and vegetable picking robot arm based on CTB — RRT* J . Transactions of the Chinese Society for Agricultural Machinery, 2021 , 52( 10) ; 129 - 136. (in Chinese)
HASSELT H V. Double Q-leaming[ M ]. Mit Press, 2010: 2613 -2621.
SCHAUL T, QUAN J, ANTONOGLOU I, et al. Prioritized experience replay [ С ] // ICLR. 4th International Conference on Learning Representations, 2016:1 -21.
RAUBER P, UMMADISINGU A, MUTZ F, et al. Hindsight policy gradients [ С] // ICLR. 7th International Conference on Learning Representations, 2019 :1 -38.
LILLICRAP T P, HUNT J J, PRITZEL A, et al. Continuous control with deep reinforcement learning[ J]. Computer Science, 2015,8(6): A187.
HAARNOJA T, ZHOU A, HARTIKAINEN K, et al. Soft actor-critic algorithms and applications[ J]. arXiv: 1812. 05905 , 2018.
RAKELLY K, ZIIOU A, QUILLEN D, et al. Efficient off-poliey meta-reinforcement learning via probabilistic context variables [J]. arXiv:1903.08254, 2019.
LI Heyu, LIN Tingyu, ZENG Bi, et al. Control method of space manipulator by using reinforcement learning [ j]. Aerospace Control, 2020,38(6) :38 -43. (in Chinese)
Refbacks
- There are currently no refbacks.
