IEEE TRANSACTIONS ON MOBILE COMPUTING,VOL.XX,NO.XX,2020 Microphone Signal Segmentation Time Delay Curve Top Microphone 20 t unW 200400 15 t2 Filter 20400 02 3 20400 Sample -t 0 Bottom Microphone 0.2 Cross- 0.5 020400 Correlation 02 t2 24040 0.2 Fig.15:Illustration of generating time delay curve. TABLE 1:Different parameters of mobile phones. Type Horizontal Distance 21 Equivalent Focal Length Samsung S5 0.142 meter 31mm Iphone 6 0.132 meter 29mm Honor 7 0.14 meter 28mm Mi6 0.145 meter 27mm Note 8 0.15 meter 26mm the smoothed delay curve.From the two asymptotes and the trace we can get the distance the automobile moves at (a)Automobile recognition a period of time so that the speed can be estimated.We can choose two asymptotes with time delay Adi and Adk and calculate the slope m;and mk.To reduce the error,we can make Adi-Adk as large as possible.After choosing the asymptotes,we can get the distance between the two (b)Automobile extraction. (c)Narrow the range. intersection points of the asymptotes and the trace.At last, the speed of the automobile can be estimated. 5 PERFORMANCE EVALUATION 5.1 Experimental Setup and Methodology (d)Edge detection. (e)Circle detection. We run SpeedTalker on different types of mobile phones, Fig.16:Wheel hub extraction including Samsung S5,Iphone 6,HUAWEI Honor 7,Mi 6 an angle 0 around an axis <x,y,z >The three elements and Samsung Note 8.The parameters of different types of of the rotation vector are<xsin号，ysin号，zsin号>.The mobile phones are shown in Table 1.The angle of view three elements are equal to the last three components of a can be calculated from equivalent focal length.The user unit quaternion<cos号，-xsin号，-ysin号，-zsin号>.q stands on the side of the road and hold the mobile phone describes the orientation of coordinate system CB relative to in the landscape orientation.When the automobile passes coordinate system CA and [y,z]is a vector described in by,the mobile phone continuously collect the acoustic and CA.Since there do not exist rapid rotations in our scenario, visual signals to estimate the speed of the automobile.In the Euler angle can be approximately seen as the rotations our experiments,the default parameter set is that recording the sound at 44.1 kHz,the sound is first arrive at bottom around the three axes. microphone and the direction is 0 degree.Default resolution of the visual signals is 1280 x 720.Default segmentation 4.5 Speed Estimation size is 441,which means each segment lasts for 0.01s. In Section 3.2.3,we propose the least square estima- Default FPS is 30 fps,and other camera settings including tion(LSE)to estimate the slope m of the trajectory.The shutter time,aperture,ISO condition(the sensitivity to light) co-ordinates(L tano,L)of the point can be given through and focusing are automatically determined by the camera image processing,since L is the distance between the au- system itself.SpeedTalker is set on these mobile phones with tomobile and the camera and o is the offset angle from the default settings to evaluate the effectiveness of the system. mid line of the camera.We can recover the real trajectory of Besides the type of mobile phones,we test SpeedTalker the automobile.That means we get the spatial information in different usage scenarios with different parameter sets of the automobile.Moreover,the points on the trajectory using Samsung Note 8.There are several parameters in have its own timestamp since the trajectory is calculated by the experimental setup:1)Sampling rate:we set the sam- the time delay curve.We can choose two asymptotes from pling rate when recording the sound of the automobile Authorized licensed use limited to:Nanjing University.Downloaded on December 24,2020 at 09:11:21 UTC from IEEE Xplore.Restrictions applyIEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, 2020 13 0 0.5 1 1.5 2 2.5 Time(s) -20 -15 -10 -5 0 5 10 15 20 Time Delay(sample) Microphone Signal Segmentation Time Delay Curve Top Microphone Bottom Microphone Filter Cross￾Correlation �" �" �# �# �$ �$ �… �… 0 200 400 -0.2 0 0.2 0 200 400 -0.2 0 0.2 0 200 400 -0.2 0 0.2 0 200 400 -0.2 0 0.2 0 200 400 -0.2 0 0.2 0 200 400 -0.2 0 0.2 �" �# 0 5 10 �$ Sample 104 -0.5 0 0.5 0 5 10 Sample 104 -0.5 0 0.5 0 5 10 Sample 104 -0.5 0 0.5 0 5 10 Sample 104 -0.5 0 0.5 Fig. 15: Illustration of generating time delay curve. (a) Automobile recognition. (b) Automobile extraction. (c) Narrow the range. (d) Edge detection. (e) Circle detection. Fig. 16: Wheel hub extraction an angle θ around an axis < x, y, z >. The three elements of the rotation vector are < x sin θ 2 , y sin θ 2 , z sin θ 2 >. The three elements are equal to the last three components of a unit quaternion < cos θ 2 , −x sin θ 2 , −y sin θ 2 , −z sin θ 2 >. A Bq describes the orientation of coordinate system CB relative to coordinate system CA and [x, y, z] is a vector described in CA. Since there do not exist rapid rotations in our scenario, the Euler angle can be approximately seen as the rotations around the three axes. 4.5 Speed Estimation In Section 3.2.3, we propose the least square estima￾tion(LSE) to estimate the slope m of the trajectory. The co-ordinates(Ltan φ, L) of the point can be given through image processing, since L is the distance between the au￾tomobile and the camera and φ is the offset angle from the mid line of the camera. We can recover the real trajectory of the automobile. That means we get the spatial information of the automobile. Moreover, the points on the trajectory have its own timestamp since the trajectory is calculated by the time delay curve. We can choose two asymptotes from TABLE 1: Different parameters of mobile phones. Type Horizontal Distance 2l Equivalent Focal Length Samsung S5 0.142 meter 31mm Iphone 6 0.132 meter 29mm Honor 7 0.14 meter 28mm Mi 6 0.145 meter 27mm Note 8 0.15 meter 26mm the smoothed delay curve. From the two asymptotes and the trace we can get the distance the automobile moves at a period of time so that the speed can be estimated. We can choose two asymptotes with time delay ∆di and ∆dk and calculate the slope mi and mk. To reduce the error, we can make |∆di − ∆dk| as large as possible. After choosing the asymptotes, we can get the distance between the two intersection points of the asymptotes and the trace. At last, the speed of the automobile can be estimated. 5 PERFORMANCE EVALUATION 5.1 Experimental Setup and Methodology We run SpeedTalker on different types of mobile phones, including Samsung S5, Iphone 6, HUAWEI Honor 7, Mi 6 and Samsung Note 8. The parameters of different types of mobile phones are shown in Table 1. The angle of view can be calculated from equivalent focal length. The user stands on the side of the road and hold the mobile phone in the landscape orientation. When the automobile passes by, the mobile phone continuously collect the acoustic and visual signals to estimate the speed of the automobile. In our experiments, the default parameter set is that recording the sound at 44.1 kHz, the sound is first arrive at bottom microphone and the direction is 0 degree. Default resolution of the visual signals is 1280 × 720. Default segmentation size is 441, which means each segment lasts for 0.01s. Default FPS is 30 fps, and other camera settings including shutter time, aperture, ISO condition(the sensitivity to light) and focusing are automatically determined by the camera system itself. SpeedTalker is set on these mobile phones with default settings to evaluate the effectiveness of the system. Besides the type of mobile phones, we test SpeedTalker in different usage scenarios with different parameter sets using Samsung Note 8. There are several parameters in the experimental setup: 1) Sampling rate: we set the sam￾pling rate when recording the sound of the automobile Authorized licensed use limited to: Nanjing University. Downloaded on December 24,2020 at 09:11:21 UTC from IEEE Xplore. Restrictions apply