This article has been accepted for publication in a future issue of this journal,but has not been fully edited.Content may change prior to final publication.Citation information:DOI 10.1109/TMC.2020.3034354.IEEE Transactions on Mobile Computing IEEE TRANSACTIONS ON MOBILE COMPUTING,VOL.XX,NO.XX,2020 SpeedTalker:Automobile Speed Estimation via Mobile Phones Xinran Lu,Lei Xie,Member,IEEE,Yafeng Yin,Member,IEEE,Wei Wang,Member,IEEE, Yanling Bu,Member,IEEE,Qing Guo,and Sanglu Lu,Member,IEEE Abstract-Among all the road accidents,speeding is the most deadly factor.To reduce speeding,it is essential to devise efficient schemes for ubiquitous speed monitoring.Traditional approaches either suffers from using special equipment(e.g.,radar speed gun)or special deployment(e.g.,position-fixed cameras).In this paper,we propose SpeedTalker,a mobile phone-based approach to perform speed detection on automobiles.By leveraging the built-in microphones and camera from the mobile phone,SpeedTalker estimates the automobile speed by passively sensing the acoustic and image signals.We propose an integrated solution to effectively estimate the automobile's speed based on COTS devices,and provide a platform for every pedestrian to help report the speeding event of automobiles.Specifically,we use the time difference of arrivals(TDOA)model based on acoustic signals to figure out the candidate trajectories of automobile,and use the pin-hole model based on image frames to figure out the vertical distance between the user's position and the automobile's trajectory,thus to estimate the unique trajectory.Combined with the time stamp of the trajectory,the automobile speed can be estimated.Besides,we propose a method to effectively mitigate the influence of the movement jitters of mobile phone.We implemented a system prototype for SpeedTalker and estimated the automobile speed with high accuracy. Experiment results show that in the scenario of single automobile,SpeedTalker can achieve an average estimation error of 6.1% compared to radar speed guns.In the scenario of multiple automobiles,SpeedTalker can achieve an average estimation error of 9.8%. which is acceptable for usage. 1 INTRODUCTION Driving direction 1.1 Motivation Nowadays,more and more traffic violations occur due to the increase of the automobile,e.g.,in 2016,the number of the road traffic deaths reached 1.35 million.Among all Sound wave kinds of the traffic violations,speeding is the most deadly factor[1].Appropriate reductions in speed can reduce fatal Top Mic bttom Mic and serious crash risk to prevent death and serious injury[2]. To reduce speeding,it is essential to devise efficient schemes for ubiquitous monitoring on traffic.Traditional ways to monitor the traffic are using speed radar or using cameras. However,they are costly and inconvenient since they need (a)Illustration of the system. wide deployment of special equipment.As a result,a low- cost and mobile solution to measure the speed is needed. 08 It is noted that,the mobile phones embedded with many kinds of sensors,such as cameras and microphones,have Current Automobile Type:Nissan become indispensable in daily life.By utilizing the built- in sensors,we can propose a method to measure the auto- Loc:Hil St mobile speed with mobile phones.Specifically,we can use the microphones and camera to recover the trajectory of the automobile and estimate the speed.IMU sensors are utilized 84km/h18 to remove jitters to raise the accuracy of the system.In this Overspeed:4km/h way,every pedestrian can help to monitor the traffic condi- (b)The application of the system. tion with his/her mobile phone.Furthermore,all people can Fig.1:Application scenario of SpeedTalker. Xinran Lu,Lei Xie,Yafeng Yin,Wei Wang,Yanling Bu,Oing Guo participate in the activities of reporting traffic conditions by and Sanglu Lu are with the State Key Laboratory for Novel Software sufficiently applying the crowdsourcing method [3]. Technology,Nanjing University,China E-mail:luxinran@smail.nju.edu.cn,lxie@nju.edu.cn,yafeng@nju.edu.cn, A typical scenario of SpeedTalker is as follows.In the ww@nju.edu.cn,yanling@smail.nju.edu.cn,guoqing@smail.nju.edu.cn, speed prone areas,the pedestrians who volunteer to moni- sanglu@nju.edu.cn. tor the traffic can arrive at the area in advance and contin- .Lei Xie is the corresponding author. uously record the acoustic and the visual signals of the au- 36-1233(c)2020 IEEE Personal use is permitted,but republication/redistribution requires IEEE permission.See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. Authorized licensed use limited to:Nanjing University.Downloaded on July 06,2021 at 04:35:27 UTC from IEEE Xplore.Restrictions apply.1536-1233 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMC.2020.3034354, IEEE Transactions on Mobile Computing IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, 2020 1 SpeedTalker: Automobile Speed Estimation via Mobile Phones Xinran Lu, Lei Xie, Member, IEEE, Yafeng Yin, Member, IEEE, Wei Wang, Member, IEEE, Yanling Bu, Member, IEEE, Qing Guo, and Sanglu Lu, Member, IEEE Abstract—Among all the road accidents, speeding is the most deadly factor. To reduce speeding, it is essential to devise efficient schemes for ubiquitous speed monitoring. Traditional approaches either suffers from using special equipment(e.g., radar speed gun) or special deployment(e.g., position-fixed cameras). In this paper, we propose SpeedTalker, a mobile phone-based approach to perform speed detection on automobiles. By leveraging the built-in microphones and camera from the mobile phone, SpeedTalker estimates the automobile speed by passively sensing the acoustic and image signals. We propose an integrated solution to effectively estimate the automobile’s speed based on COTS devices, and provide a platform for every pedestrian to help report the speeding event of automobiles. Specifically, we use the time difference of arrivals (TDOA) model based on acoustic signals to figure out the candidate trajectories of automobile, and use the pin-hole model based on image frames to figure out the vertical distance between the user’s position and the automobile’s trajectory, thus to estimate the unique trajectory. Combined with the time stamp of the trajectory, the automobile speed can be estimated. Besides, we propose a method to effectively mitigate the influence of the movement jitters of mobile phone. We implemented a system prototype for SpeedTalker and estimated the automobile speed with high accuracy. Experiment results show that in the scenario of single automobile, SpeedTalker can achieve an average estimation error of 6.1% compared to radar speed guns. In the scenario of multiple automobiles, SpeedTalker can achieve an average estimation error of 9.8%, which is acceptable for usage. ✦ 1 INTRODUCTION 1.1 Motivation Nowadays, more and more traffic violations occur due to the increase of the automobile, e.g., in 2016, the number of the road traffic deaths reached 1.35 million. Among all kinds of the traffic violations, speeding is the most deadly factor[1]. Appropriate reductions in speed can reduce fatal and serious crash risk to prevent death and serious injury[2]. To reduce speeding, it is essential to devise efficient schemes for ubiquitous monitoring on traffic. Traditional ways to monitor the traffic are using speed radar or using cameras. However, they are costly and inconvenient since they need wide deployment of special equipment. As a result, a low￾cost and mobile solution to measure the speed is needed. It is noted that, the mobile phones embedded with many kinds of sensors, such as cameras and microphones, have become indispensable in daily life. By utilizing the built￾in sensors, we can propose a method to measure the auto￾mobile speed with mobile phones. Specifically, we can use the microphones and camera to recover the trajectory of the automobile and estimate the speed. IMU sensors are utilized to remove jitters to raise the accuracy of the system. In this way, every pedestrian can help to monitor the traffic condi￾tion with his/her mobile phone. Furthermore, all people can • Xinran Lu, Lei Xie, Yafeng Yin, Wei Wang, Yanling Bu, Qing Guo and Sanglu Lu are with the State Key Laboratory for Novel Software Technology, Nanjing University, China. E-mail: luxinran@smail.nju.edu.cn, lxie@nju.edu.cn, yafeng@nju.edu.cn, ww@nju.edu.cn, yanling@smail.nju.edu.cn, guoqing@smail.nju.edu.cn, sanglu@nju.edu.cn. • Lei Xie is the corresponding author. (a) Illustration of the system. (b) The application of the system. Fig. 1: Application scenario of SpeedTalker. participate in the activities of reporting traffic conditions by sufficiently applying the crowdsourcing method [3]. A typical scenario of SpeedTalker is as follows. In the speed prone areas, the pedestrians who volunteer to moni￾tor the traffic can arrive at the area in advance and contin￾uously record the acoustic and the visual signals of the au￾Authorized licensed use limited to: Nanjing University. Downloaded on July 06,2021 at 04:35:27 UTC from IEEE Xplore. Restrictions apply