第17卷第4期 智能系统学报 Vol.17 No.4 2022年7月 CAAI Transactions on Intelligent Systems Jul.2022 D0:10.11992/tis.202106044 网络出版地址：https:/kns.cnki.net/kcms/detail/23.1538.TP.20220505.1636.004.html 动态环境下分布式异构多机器人避障方法研究 欧阳勇平，魏长赞，蔡帛良2 (1.河海大学机电工程学院，江苏常州213022：2.英国卡迪夫大学工学院，威尔士卡迪夫CF103A) 摘要：多机器人系统在联合搜救、智慧车间、智能交通等领域得到了日益广泛的应用。目前，多个机器人之 间、机器人与动态环境之间的路径规划和导航避障仍需依赖精确的环境地图，给多机器人系统在非结构环境下 的协调与协作带来了挑战。针对上述问题，本文提出了不依赖精确地图的分布式异构多机器人导航避障方法， 建立了基于深度强化学习的多特征策略梯度优化算法，并考虑了人机协同环境下的社会范式，使分布式机器人 能够通过与环境的试错交互，学习最优的导航避障策略：并在Gazebo仿真环境下进行了最优策略的训练学习， 同时将模型移植到多个异构实体机器人上，将机器人控制信号解码，进行真实环境测试。实验结果表明：本文 提出的多特征策略梯度优化算法能够通过自学习获得最优的导航避障策略，为分布式异构多机器人在动态环 境下的应用提供了一种技术参考。 关键词：异构多机器人；深度强化学习：非结构环境；多特征策略梯度：动态避障：自学习；分布式控制：控制策略 中图分类号：TP273+.2文献标志码：A文章编号：1673-4785(2022)04-0752-12 中文引用格式：欧阳勇平，魏长蒉，蔡帛良.动态环境下分布式异构多机器人避障方法研究川智能系统学报，2022,17(4)： 752-763. 英文引用格式：OUYANG Yongping,WEI Changyun,CAI Boliang.Collision avoidance approach for distributed heterogeneous multirobot systems in dynamic environments[J.CAAI transactions on intelligent systems,2022,17(4):752-763. Collision avoidance approach for distributed heterogeneous multirobot systems in dynamic environments OUYANG Yongping',WEI Changyun',CAI Boliang'2 (1.College of Mechanical and Electrical Engineering,Hohai University,Changzhou 213022,China;2.School of Engineering. Cardiff University,CardiffCF103AT,UK) Abstract:Multirobot systems have been widely used in cooperative search and rescue missions,intelligent warehouses. intelligent transportation,and other fields.At present,the path planning and collision avoidance problems between mul- tiple robots and the dynamic environment still rely on accurate maps,which brings challenges to the coordination and cooperation of multirobot systems in unstructured environments.To address the above problem,this paper presents a navigation and collision avoidance approach that does not require accurate maps and is based on the deep reinforcement learning framework.A multifeatured policy gradients algorithm is proposed in this work,and social norms are also in- tegrated so that the learning agent can obtain the optimal control policy via trial-and-error interactions with the environ- ment.The optimal policy is trained and obtained in the Gazebo environment,and afterward,the optimal policy is trans- ferred to several heterogeneous real robots by decoding the control signals.The experimental results show that the multi- feature policy gradients algorithm proposed can obtain the optimal navigation collision avoidance policy through self- learning,and it provides a technical reference for the application of distributed heterogeneous multirobot systems in dy- namic environments. Keywords:heterogeneous multi-robot systems;deep reinforcement learning,non-structural environment;multi-feature policy gradients;dynamic collision avoidance;self-learning;distributed control;control policy 收稿日期：2021-06-25.网络出版日期：2022-05-06 基金项目：国家自然科学基金项目(61703138)：中央高校基本 科研业务费项目(B200202224). 随着多机器人系统(multi-robot system,MRS) 通信作者：魏长赞.E-mail:c.wei@hhu.edu.cn. 的广泛应用，其路径规划和导航避障领域一直是DOI: 10.11992/tis.202106044 网络出版地址: https://kns.cnki.net/kcms/detail/23.1538.TP.20220505.1636.004.html 动态环境下分布式异构多机器人避障方法研究 欧阳勇平1 ，魏长赟1 ，蔡帛良1,2 （1. 河海大学 机电工程学院，江苏 常州 213022; 2. 英国卡迪夫大学 工学院，威尔士 卡迪夫 CF10 3A） 摘 要：多机器人系统在联合搜救、智慧车间、智能交通等领域得到了日益广泛的应用。目前，多个机器人之 间、机器人与动态环境之间的路径规划和导航避障仍需依赖精确的环境地图，给多机器人系统在非结构环境下 的协调与协作带来了挑战。针对上述问题，本文提出了不依赖精确地图的分布式异构多机器人导航避障方法， 建立了基于深度强化学习的多特征策略梯度优化算法，并考虑了人机协同环境下的社会范式，使分布式机器人 能够通过与环境的试错交互，学习最优的导航避障策略；并在 Gazebo 仿真环境下进行了最优策略的训练学习， 同时将模型移植到多个异构实体机器人上，将机器人控制信号解码，进行真实环境测试。实验结果表明：本文 提出的多特征策略梯度优化算法能够通过自学习获得最优的导航避障策略，为分布式异构多机器人在动态环 境下的应用提供了一种技术参考。 关键词：异构多机器人；深度强化学习；非结构环境；多特征策略梯度；动态避障；自学习；分布式控制；控制策略 中图分类号：TP273+.2 文献标志码：A 文章编号：1673−4785(2022)04−0752−12 中文引用格式：欧阳勇平, 魏长赟, 蔡帛良. 动态环境下分布式异构多机器人避障方法研究 [J]. 智能系统学报, 2022, 17(4): 752–763. 英文引用格式：OUYANG Yongping, WEI Changyun, CAI Boliang. Collision avoidance approach for distributed heterogeneous multirobot systems in dynamic environments[J]. CAAI transactions on intelligent systems, 2022, 17(4): 752–763. Collision avoidance approach for distributed heterogeneous multirobot systems in dynamic environments OUYANG Yongping1 ，WEI Changyun1 ，CAI Boliang1,2 (1. College of Mechanical and Electrical Engineering, Hohai University, Changzhou 213022, China; 2. School of Engineering, Cardiff University, Cardiff CF10 3AT, UK) Abstract: Multirobot systems have been widely used in cooperative search and rescue missions, intelligent warehouses, intelligent transportation, and other fields. At present, the path planning and collision avoidance problems between mul￾tiple robots and the dynamic environment still rely on accurate maps, which brings challenges to the coordination and cooperation of multirobot systems in unstructured environments. To address the above problem, this paper presents a navigation and collision avoidance approach that does not require accurate maps and is based on the deep reinforcement learning framework. A multifeatured policy gradients algorithm is proposed in this work, and social norms are also in￾tegrated so that the learning agent can obtain the optimal control policy via trial-and-error interactions with the environ￾ment. The optimal policy is trained and obtained in the Gazebo environment, and afterward, the optimal policy is trans￾ferred to several heterogeneous real robots by decoding the control signals. The experimental results show that the multi￾feature policy gradients algorithm proposed can obtain the optimal navigation collision avoidance policy through self￾learning, and it provides a technical reference for the application of distributed heterogeneous multirobot systems in dy￾namic environments. Keywords: heterogeneous multi-robot systems; deep reinforcement learning; non-structural environment; multi-feature policy gradients; dynamic collision avoidance; self-learning; distributed control; control policy 随着多机器人系统 (multi-robot system，MRS) 的广泛应用，其路径规划和导航避障领域一直是 收稿日期：2021−06−25. 网络出版日期：2022−05−06. 基金项目：国家自然科学基金项目（61703138）；中央高校基本 科研业务费项目（B200202224）. 通信作者：魏长赟. E-mail：c.wei@hhu.edu.cn. 第 17 卷第 4 期 智 能 系 统 学 报 Vol.17 No.4 2022 年 7 月 CAAI Transactions on Intelligent Systems Jul. 2022