Complete Coverage Path Planning Strategy for Offshore Fishing Multi-USV Based on Improved QMIX Algorithm

WU Qingyun, WANG Dong, TAO Jun, LI Zhijian, YIN Yijie

Abstract

In offshore fishing missions, multiple unmanned surface vessels (USVs) need to perform comprehensive coverage path planning within specific waters to detect fish distribution. However, traditional multi-agent reinforcement learning lacks the capability to simultaneously consider both its own and neighboring agents-states, coupled with an unclear feedback mechanism, leading to low efficiency and high redundancy in coverage tasks. To address these issues, a comprehensive coverage path planning strategy for offshore fishing was proposed by using multiple USVs based on the improved QMIX algorithm (LH-QMIX). QMIX was a multi-agent reinforcement learning method, consisting of a mixing network and multiple agent networks, which integrated the local Q-values from each agent network into a global Q-value through the mixing network to guide agent actions. Considering that communication and perception ranges were typically limited in offshore environment, a local loss function was introduced for each agent network to provide a clearer feedback mechanism. Additionally, a hybrid attention mechanism was incorporated to enhance collaboration among USVs. The proposed LH-QMIX algorithm was compared with the independent Q-learning (IQL) algorithm and the original QMIX algorithm through simulations in both simple and complex obstacle environment. Simulation result showed that compared with the traditional QMIX algorithm, the LH-QMIX achieved improvements of 6.9% and 10.6% in coverage efficiency in simple and complex obstacle environments, respectively, with more stable reward curves after convergence. The research provided an effective solution for multiple USVs to efficiently achieve comprehensive coverage in offshore fishing missions, thereby enhancing the efficiency of offshore fishing operations.

 

Keywords: multi-USV path planning, LH-QMIX algorithm, multi-agent reinforcement learning, coverage efficiency, model stability


Download Full Text:

PDF


References


MATSUSHITA Y, ONUMA A, TAKESIIITA C, et al. Unmanned surface vehicle (USV) with a fish attraction lamp to assist the purse seine operations [J]. Fisheries Science, 2024, 90(3) : 357 -367.

HANDEGARD N O, DE ROBERTIS A, HOLMIN A J, et al. Uncrewed surface vehicles (USVs) as platforms for fisheries and plankton acoustics[J]. ICES Journal of Marine Science, 2024, 81(9) ; 1712 - 1723.

SOTELO-TORRES F, ALVAREZ L V, ROBERTS R C. An unmanned surface vehicle (USV) ; development of an autonomous boat with a sensor integration system for bathymetric surveys. Sensors, 2023, 23(9) ; 4420.

EVANS T M, RUDSTAM L G, SETHI S A, et al. Fish avoidance of ships during acoustic surveys tested with quiet uncrewed surface vessels J . Fisheries Research, 2023, 267: 106817.

LIU Tianhu, LAI Jiashang, SUN Weilong, et al. Pineapple field navigation path planning based on jump point optimization ant colony algorithm [ J ]. Transactions of the Chinese Society for Agricultural Machinery, 2025,56(4) :387 -396.

XING B, WANG X, YANG L, et al. An algorithm of complete coverage path planning for unmanned surface vehicle based on reinforcement learning[J]. Journal of Marine Science and Engineering, 2023, 11(3) ; 645.

YUAN Jie, ZHANG Yinggang, JIA Erkenbieke, et al. Path planning method for transport robots based on AFD fusion algorithm J. Transactions of the Chinese Society for Agricultural Machinery, 2025,56(6) :594 -607.

ZHANG J, REN J, GUI Y, et al. Multi-USV task planning method based on improved deep reinforcement learning [J]. IEEEInternet of Things Journal, 2024, 11 ( 10) : 18549 - 18567.

CHEN Z, ZHAO Z, XU J, et al. A cooperative hunting method for multi-USV based on the A* algorithm in an environment with obstacles [J] . Sensors, 2023, 23(16) : 7058.

WANG N, LIU Y, LIU J, et al. Reinforcement learning swarm of self-organizing unmanned surface vehicles with unavailable dynamics[J] . Ocean Engineering, 2023, 289: 116313.

CHEN C, LIANG X, ZHANG Z, et al. Cooperative game method of heterogeneous unmanned surface vehicles based on distributed decision-making framework [ J ]. Ocean Engineering, 2025, 338; 122048.

JEONG S K, KIM M K, PARK H Y, et al. Study on fault diagnosis technology for efficient swarm control operation of unmanned surface vehicles. Applied Sciences, 2024, 14(10) ; 4210.

JEONG S К, KIM M K, PARK H Y, et al. Study on fault diagnosis technology for efficient swarm control operation of unmanned surface vehicles. Applied Sciences, 2024, 14( 10) : 4210.

YUAN P, ZHANG Z, LI Y, et al. Leader-follower control and APF for muIti-USV coordination and obstacle avoidance [J ] . Ocean Engineering, 2024, 313; 119487.

SONG Z, ZHENG R, ZHANG S, et al. Cooperative reward shaping for multi-agent path finding [J ]. arXiv Preprint, arXiv; 2407.10403, 2024.

KOSTRIKOV I, NAIR A, LEVINE S. Offline reinforcement learning with implicit Q-learning [ J ]. arXiv Preprint, arXiv; 2110.06169, 2021.

NANTOGMA S, ZHANG S, YU X, et al. Multi-USV dynamic navigation and target capture; a guided multi-agent reinforcement learning approach [J]. Electronics, 2023, 12(7): 1523.

WAGNER G, CHOSET H. Subdimensional expansion for multirobot path planning [j]. Artificial Intelligence, 2015, 219: 1 -24.

YU J, CHEN Z, ZHAO Z, et al. A traversal multi-target path planning method for multi-unmanned surface vessels in space- varying ocean current[J]. Ocean Engineering, 2023, 278: 114423.

HUANG T, LI J, KOENIG S, et al. Anytime multi-agent path finding via machine learning-guided large neighborhood search [ С ] // Proceedings of the AAAI Conference on Artificial Intelligence, 2022, 36(9) : 9368 -9376.

RAMEZANI M, AMIRI ATASHGAH M A, REZAEE A. A fault-tolerant multi-agent reinforcement learning framework for unmanned aerial vehicles-unmanned ground vehicle coverage path planning [J] . Drones, 2024, 8( 10) : 537.

MA Z, YOU J, ZHANG Y, et al. Reinforcement learning-based dynamic coverage control of multi-rotor UAVs with safety priority [J]. IEEE Transactions on Automation Science and Engineering, 2024, 22: 17474 - 17485.

CHEN D, QI Q, FU Q, et al. Transformer-based reinforcement learning for scalable multi-UAV area coverage [J] . IEEE Transactions on Intelligent Transportation Systems, 2024, 25(8) : 10062 - 10077.

YUAN G, XIAO J, HE J, et al. Multi-agent cooperative area coverage; a two-stage planning approach based on reinforcement learning [J]. Information Sciences, 2024, 678; 121025.

CHOI H B, KIM J B, HAN Y H, et al. MARL-based cooperative multi-AGV control in warehouse systems. IEEE Access, 2022, 10; 100478 - 100488.

RASHID T, SAMVELYAN M, DE WITT С S, et al. Monotonic value function factorisation for deep multi-agent reinforcement learning [J]. Journal of Machine Learning Research, 2020, 21; 1 -51.


Refbacks

  • There are currently no refbacks.