工程科学学报，第41卷，第10期：1332-1341,2019年10月 Chinese Journal of Engineering,Vol.41,No.10:1332-1341,October 2019 D0I:10.13374/j.issn2095-9389.2018.10.15.001;http:/journals.ustb.edu.cn 基于增强学习算法的插电式燃料电池电动汽车能量管 理控制策略 林歆悠12)，夏玉田)，魏申中) 1)福州大学机械工程及自动化学院，福州350002 2)流体动力与电液智能控制福建省高校重点实验室（福州大学），福州350002 ☒通信作者，E-mail:linxinyoou@fu.cdu.cn 摘要以一款插电式燃料电池电动汽车(plug-in fuel cell electric vehicle,PFCEV)为研究对象，为改善燃料电池氢气消耗和 电池电量消耗之间的均衡，实现插电式燃料电池电动汽车的燃料电池与动力电池之间的最优能量分配，考虑燃料电池汽车实 时能量分配的即时回报及未来累积折扣回报，以整车作为环境，整车控制作为智能体，提出了一种基于增强学习算法的插电 式燃料电池电动汽车能量管理控制策略.通过Matlab/Simulink建立整车仿真模型对所提出的策略进行仿真验证，相比于基于 规则的策略，在不同行驶里程下，电池均可保持一定的电量，整车的综合能耗得到明显降低，在100、200和300km行驶里程下 整车百公里能耗分别降低8.84%、29.5%和38.6%：基于快速原型开发平台进行硬件在环试验验证，城市行驶工况工况下整 车综合能耗降低20.8%，硬件在环试验结果与仿真结果基本一致，表明了所制定能量管理策略的有效性和可行性. 关键词燃料电池汽车；增强学习；能量管理；Q_learning算法；控制策略 分类号TG142.71 Energy management control strategy for plug-in fuel cell electric vehicle based on reinforcement learning algorithm LIN Xin-you'),XIA Yu-tian',WEI Shen-shen') 1)College of Mechanical Engineering and Automation,Fuzhou University,Fuzhou 350002,China 2)Key Laboratory of Fluid Power and Intelligent Electro-Hydraulic Control,Fuzhou University,Fuzhou 350002,China Corresponding author,E-mail:linxinyoou@fa.edu.cn ABSTRACT To cope with the increasingly stringent emission regulations,major automobile manufacturers have been focusing on the development of new energy vehicles.Fuel-cell vehicles with advantages of zero emission,high efficiency,diversification of fuel sources,and renewable energy have been the focus of international automotive giants and Chinese automotive enterprises.Establishing a reasonable energy management strategy,effectively controlling the vehicle working mode,and reasonably using battery energy for hy- brid fuel-cell vehicles are core technologies in domestic and foreign automobile enterprises and research institutes.To improve the equi- librium between fuel-cell hydrogen consumption and battery consumption and realize the optimal energy distribution between fuel-cell systems and batteries for plug-in fuel-cell electric vehicles(PFCEVs),considering vehicles as the environment and vehicle control as an agent,an energy management strategy for the PFCEV based on reinforcement learning algorithm was proposed in this paper.This strategy considered the immediate return and future cumulative discounted returns of a fuel-cell vehicle's real-time energy allocation. The vehicle simulation model was built by Matlab/Simulink to carry out the simulation test for the proposed strategy.Compared with the rule-based strategy,the battery can store a certain amount of electricity,and the integrated energy consumption of the vehicle was nota- bly reduced under different mileages.The energy consumption in 100 km was reduced by 8.84%,29.5%,and 38.6%under 100. 收稿日期：2018-10-15 基金项目：国家自然科学基金资助项目(51505086)工程科学学报,第 41 卷,第 10 期:1332鄄鄄1341,2019 年 10 月 Chinese Journal of Engineering, Vol. 41, No. 10: 1332鄄鄄1341, October 2019 DOI: 10. 13374 / j. issn2095鄄鄄9389. 2018. 10. 15. 001; http: / / journals. ustb. edu. cn 基于增强学习算法的插电式燃料电池电动汽车能量管 理控制策略 林歆悠1,2) 苣 , 夏玉田1) , 魏申申1) 1)福州大学机械工程及自动化学院, 福州 350002 2)流体动力与电液智能控制福建省高校重点实验室(福州大学), 福州 350002 苣通信作者, E鄄mail: linxinyoou@ fzu. edu. cn 摘 要 以一款插电式燃料电池电动汽车(plug鄄in fuel cell electric vehicle, PFCEV)为研究对象,为改善燃料电池氢气消耗和 电池电量消耗之间的均衡,实现插电式燃料电池电动汽车的燃料电池与动力电池之间的最优能量分配,考虑燃料电池汽车实 时能量分配的即时回报及未来累积折扣回报,以整车作为环境,整车控制作为智能体,提出了一种基于增强学习算法的插电 式燃料电池电动汽车能量管理控制策略. 通过 Matlab / Simulink 建立整车仿真模型对所提出的策略进行仿真验证,相比于基于 规则的策略,在不同行驶里程下,电池均可保持一定的电量,整车的综合能耗得到明显降低,在 100、200 和 300 km 行驶里程下 整车百公里能耗分别降低 8郾 84% 、29郾 5% 和 38郾 6% ;基于快速原型开发平台进行硬件在环试验验证,城市行驶工况工况下整 车综合能耗降低 20郾 8% ,硬件在环试验结果与仿真结果基本一致,表明了所制定能量管理策略的有效性和可行性. 关键词 燃料电池汽车; 增强学习; 能量管理; Q_learning 算法; 控制策略 分类号 TG142郾 71 收稿日期: 2018鄄鄄10鄄鄄15 基金项目: 国家自然科学基金资助项目(51505086) Energy management control strategy for plug鄄in fuel cell electric vehicle based on reinforcement learning algorithm LIN Xin鄄you 1,2) 苣 , XIA Yu鄄tian 1) , WEI Shen鄄shen 1) 1)College of Mechanical Engineering and Automation, Fuzhou University, Fuzhou 350002, China 2)Key Laboratory of Fluid Power and Intelligent Electro鄄Hydraulic Control, Fuzhou University, Fuzhou 350002, China 苣Corresponding author, E鄄mail: linxinyoou@ fzu. edu. cn ABSTRACT To cope with the increasingly stringent emission regulations, major automobile manufacturers have been focusing on the development of new energy vehicles. Fuel鄄cell vehicles with advantages of zero emission, high efficiency, diversification of fuel sources, and renewable energy have been the focus of international automotive giants and Chinese automotive enterprises. Establishing a reasonable energy management strategy, effectively controlling the vehicle working mode, and reasonably using battery energy for hy鄄 brid fuel鄄cell vehicles are core technologies in domestic and foreign automobile enterprises and research institutes. To improve the equi鄄 librium between fuel鄄cell hydrogen consumption and battery consumption and realize the optimal energy distribution between fuel鄄cell systems and batteries for plug鄄in fuel鄄cell electric vehicles (PFCEVs), considering vehicles as the environment and vehicle control as an agent, an energy management strategy for the PFCEV based on reinforcement learning algorithm was proposed in this paper. This strategy considered the immediate return and future cumulative discounted returns of a fuel鄄cell vehicle爷 s real鄄time energy allocation. The vehicle simulation model was built by Matlab / Simulink to carry out the simulation test for the proposed strategy. Compared with the rule鄄based strategy, the battery can store a certain amount of electricity, and the integrated energy consumption of the vehicle was nota鄄 bly reduced under different mileages. The energy consumption in 100 km was reduced by 8郾 84% , 29郾 5% , and 38郾 6% under 100