第14卷第4期 智能系统学报 Vol.14 No.4 2019年7月 CAAI Transactions on Intelligent Systems Jul.2019 D0:10.11992/tis.201801016 网络出版地址：http:/kns.cnki.net/kcms/detail/23.1538.TP.20180628.1622.004html 基于车内外视觉信息的行人碰撞预警方法 杨会成，朱文博，童英 (安徽工程大学电气工程学院，安徽芜湖241000) 摘要：行人碰撞预警系统通常依据行人检测与碰撞时间判断的方式为驾驶员提供预警信息。为了提供更加 可靠的危险判断依据，本文提出一种同时分析道路状况与驾驶员头部姿态的行人碰撞预警方法，用两个单目相 机分别获取车辆内外环境图像。通道特征检测器用于定位行人，根据单目视觉距离测量方法估计出行人与自 车间的纵向与横向距离。多任务级联卷积网络用于定位驾驶员面部特征点，通过求解多点透视问题获取头部 方向角以反映驾驶员注意状态。结合行人位置信息与驾驶员状态信息，本文构建模糊推理系统判断碰撞风险 等级。在实际路况下的实验结果表明.根据模糊系统输出的风险等级可以为预防碰撞提供有效的指导。 关键词：碰撞预警：内外信息：行人定位：驾驶员状态；单目视觉：通道特征：多任务级联卷积网络：模糊推理系统 中图分类号：TP181文献标志码：A文章编号：1673-4785(2019)04-0752-09 中文引用格式：杨会成，朱文博，童英.基于车内外视觉信息的行人碰撞预警方法智能系统学报，2019,14(4)：752-760. 英文引用格式：YANG Huicheng,ZHU Wenbo,TONG Ying..Pedestrian collision warning system based on looking-in and looking out visual information analysis[J.CAAI transactions on intelligent systems,2019,14(4):752-760. Pedestrian collision warning system based on looking-in and looking-out visual information analysis YANG Huicheng,ZHU Wenbo,TONG Ying (College of Electrical Engineering,Anhui Polytechnic University,Wuhu 241000,China) Abstract:Pedestrian collision warning systems usually provide early warning for drivers based on the technologies of pedestrian detection and collision time measurement.To provide a more reliable basis for risk assessment,a pedestrian collision warning method that involves analyzing the road condition and driver's head pose simultaneously is proposed in this paper.Two monocular cameras are used to capture vehicle exterior and interior images,and a channel features detector is applied to locate pedestrians.The vertical and horizontal distances between pedestrians and ego-vehicle are estimated based on monocular vision distance measurement.The multi-task cascaded convolutional network is utilized for facial landmark detection.By solving a perspective-n-point(PnP)problem,the estimated head angles can reflect driver's attention states.By combining both pedestrian location information and driver's attention information,we im- plemented a fuzzy inference system to assess collision risk level.An experiment in real-world driving conditions demon- strated that the risk levels obtained from the fuzzy system are reliable and can provide guidance for collision avoidance. Keywords:collision warning;internal and external information;pedestrian positioning;driver states;monocular vision; channel features;multi-task cascaded convolutional network;fuzzy inference system 高级驾驶辅助系统(advanced driver assistance碰撞预警是ADAS的一个重要功能，碰撞预警系 system,ADAS)是目前车辆安全领域的研究热点， 统通常根据传感器获取的前方障碍物相对距离和 该系统通过车载传感器收集并分析车内外环境数 速度计算碰撞时间(time to collision.,TTC),可靠的 据，为驾驶员提供辅助信息并对危险进行提醒。 预警一般设置为潜在的碰撞前约2s的时间。目 收稿日期：2018-01-08.网络出版日期：2018-06-29 前应用的碰撞保护系统大多针对车辆间的碰撞， 基金项目：安徽省高校自然科学研究重点项目(KJ2018A0122) 通信作者：朱文博.E-mail:vembozhu@l63.com. 然而在人车碰撞事故中，没有保护装置的行人更DOI: 10.11992/tis.201801016 网络出版地址: http://kns.cnki.net/kcms/detail/23.1538.TP.20180628.1622.004.html 基于车内外视觉信息的行人碰撞预警方法 杨会成，朱文博，童英 （安徽工程大学 电气工程学院，安徽 芜湖 241000） 摘 要：行人碰撞预警系统通常依据行人检测与碰撞时间判断的方式为驾驶员提供预警信息。为了提供更加 可靠的危险判断依据，本文提出一种同时分析道路状况与驾驶员头部姿态的行人碰撞预警方法，用两个单目相 机分别获取车辆内外环境图像。通道特征检测器用于定位行人，根据单目视觉距离测量方法估计出行人与自 车间的纵向与横向距离。多任务级联卷积网络用于定位驾驶员面部特征点，通过求解多点透视问题获取头部 方向角以反映驾驶员注意状态。结合行人位置信息与驾驶员状态信息，本文构建模糊推理系统判断碰撞风险 等级。在实际路况下的实验结果表明，根据模糊系统输出的风险等级可以为预防碰撞提供有效的指导。 关键词：碰撞预警；内外信息；行人定位；驾驶员状态；单目视觉；通道特征；多任务级联卷积网络；模糊推理系统 中图分类号：TP181 文献标志码：A 文章编号：1673−4785(2019)04−0752−09 中文引用格式：杨会成, 朱文博, 童英. 基于车内外视觉信息的行人碰撞预警方法 [J]. 智能系统学报, 2019, 14(4): 752–760. 英文引用格式：YANG Huicheng, ZHU Wenbo, TONG Ying. Pedestrian collision warning system based on looking-in and looking￾out visual information analysis[J]. CAAI transactions on intelligent systems, 2019, 14(4): 752–760. Pedestrian collision warning system based on looking-in and looking-out visual information analysis YANG Huicheng，ZHU Wenbo，TONG Ying (College of Electrical Engineering, Anhui Polytechnic University, Wuhu 241000, China) Abstract: Pedestrian collision warning systems usually provide early warning for drivers based on the technologies of pedestrian detection and collision time measurement. To provide a more reliable basis for risk assessment, a pedestrian collision warning method that involves analyzing the road condition and driver’s head pose simultaneously is proposed in this paper. Two monocular cameras are used to capture vehicle exterior and interior images, and a channel features detector is applied to locate pedestrians. The vertical and horizontal distances between pedestrians and ego-vehicle are estimated based on monocular vision distance measurement. The multi-task cascaded convolutional network is utilized for facial landmark detection. By solving a perspective-n-point (PnP) problem, the estimated head angles can reflect driver’s attention states. By combining both pedestrian location information and driver’s attention information, we im￾plemented a fuzzy inference system to assess collision risk level. An experiment in real-world driving conditions demon￾strated that the risk levels obtained from the fuzzy system are reliable and can provide guidance for collision avoidance. Keywords: collision warning; internal and external information; pedestrian positioning; driver states; monocular vision; channel features; multi-task cascaded convolutional network; fuzzy inference system 高级驾驶辅助系统 (advanced driver assistance system, ADAS) 是目前车辆安全领域的研究热点， 该系统通过车载传感器收集并分析车内外环境数 据，为驾驶员提供辅助信息并对危险进行提醒。 碰撞预警是 ADAS 的一个重要功能，碰撞预警系 统通常根据传感器获取的前方障碍物相对距离和 速度计算碰撞时间 (time to collision, TTC)，可靠的 预警一般设置为潜在的碰撞前约 2 s 的时间。目 前应用的碰撞保护系统大多针对车辆间的碰撞， 然而在人车碰撞事故中，没有保护装置的行人更 收稿日期：2018−01−08. 网络出版日期：2018−06−29. 基金项目：安徽省高校自然科学研究重点项目 (KJ2018A0122). 通信作者：朱文博. E-mail：vembozhu@163.com. 第 14 卷第 4 期 智 能 系 统 学 报 Vol.14 No.4 2019 年 7 月 CAAI Transactions on Intelligent Systems Jul. 2019