100 8 2 3. 56 7891011 Mate7 Note3 Mate7 Note3 Position Smartphone types Smartphone types (a)Accuracy for different positions (b)Movement error for different phones (c)Touch accuracy for different phones Figure 13:Micro benchmark results for touching and generalization (a)Path delay calibration VSkin incurs a power consumption of 491.3 mW on com- Down conversion Cross-correlation Total mercial smartphones.We use Powertutor [36]to measure Time 0.363ms 14.582ms 14.945ms the power consumption of our system on Samsung Galaxy S5.Without considering the LCD power consumption,the (b)Movement and touch sensing power consumptions of CPU and Audio are 121.3 mW and Down conversion Upsampling Phase Total 370 mW,respectively.To measure the power consumption measurement overhead of VSkin,we measured the average power con- Time 0.363ms 3.249ms 0.324ms 3.939ms sumption in three different states with VSkin:1)Backlight, Table 3:Processing time with the screen displaying the results,2)Web Browsing, (a)Power consumption surfing the Internet with the WiFi on,3)Gaming,playing CPU Audio Total mobile games with the WiFi on.The power consumption Power 121.3 mW 370 mW 491.3 mW overheads for these states are 47.8%,25.4%,and 15.4%,re- spectively.More than 74.2%additional power consumption (b)Power consumption overhead comes from the speaker hardware.One possible solution is to Backlight Web Browsing Gaming design low-power speakers that are specialized for emitting Power overhead 47.8% 25.4% 15.4% ultrasounds Table 4:Power consumption 8.5 Discussions change to classify the positions into three different classes and the results are shown in Figure 13(a).The localization Different phone setups:VSkin can work on different accuracies of position 2,6,and 9 are lower than other posi- types of smartphones.We conducted our experiments on four tions because these three positions are not on the path of different smartphones,Samsung S5,Huawei Mate7,Samsung propagation from the rear speaker to the top microphone. Note3,and Samsung S7,with parameters based on the loca- tions of speaker and microphones.As shown in Figure 13(b). 8.4 Latency and Power Consumption VSkin achieves an average movement distance error of 3.59 VSkin achieves a latency of 4.83 ms on commercial smart- cm,2.96 cm,4.02 cm and 6.05 cm on the four models,respec- phones.We measured the processing time for a Samsung S5 tively.VSkin also achieves more than 98%touch accuracy with Qualcomm Snapdragon 2.5GHz quad-core CPU.Our for all types of smartphones,as shown in Figure 13(c). system processes sound segments with a size of 1,024 sam- Locations of speakers and microphones:The speak- ples(time duration of 21.3 ms at 48 kHz sampling rate).To ers on Samsung S5 and Huawei Mate7 are on the back of reduce the processing time,we only perform the path delay the smartphones,while the speakers on Samsung S7 and calibration on the first ten data segments and later process- Samsung Note3 are at the bottom of the smartphones.Our ing does not require recalibration.Furthermore,we use FFT experimental results show that VSkin achieves higher accu- to accelerate the cross-correlation.The processing time of racy on S5/Mate7 than S7/Note3.Therefore,the locations of one segments is 14.58 ms and 3.93 ms for the computational the speaker and microphones are critical to the VSkin perfor- heavy path delay calibration process and the light-weight mance.The current design of VSkin requires one speaker and movement/touch-sensing algorithm.With processing time two microphones,with one microphone close to the speaker for other operations,the overall latency for VSkin to process to measure the movement and the other at the opposite side 21.3 ms of data is 4.832 ms on average.Therefore,VSkin can of the speaker to measure the touch.Further generalization perform realtime movement and touch sensing on commod- of VSkin to different speaker/microphone setups is left for ity mobile devices. future study.1 2 3 4 5 6 7 8 9 10 11 Position 0 20 40 60 80 100 Localization accuracy (%) (a) Accuracy for different positions S5 Mate7 Note3 S7 Smartphone types 0 5 10 15 Error (mm) (b) Movement error for different phones S5 Mate7 Note3 S7 Smartphone types 90 92 94 96 98 100 Touch accuracy (%) (c) Touch accuracy for different phones Figure 13: Micro benchmark results for touching and generalization (a) Path delay calibration Down conversion Cross-correlation Total Time 0.363 ms 14.582 ms 14.945 ms (b) Movement and touch sensing Down conversion Upsampling Phase measurement Total Time 0.363 ms 3.249 ms 0.324 ms 3.939 ms Table 3: Processing time (a) Power consumption CPU Audio Total Power 121.3 mW 370 mW 491.3 mW (b) Power consumption overhead Backlight Web Browsing Gaming Power overhead 47.8% 25.4% 15.4% Table 4: Power consumption change to classify the positions into three different classes and the results are shown in Figure 13(a). The localization accuracies of position 2, 6, and 9 are lower than other posi￾tions because these three positions are not on the path of propagation from the rear speaker to the top microphone. 8.4 Latency and Power Consumption VSkin achieves a latency of 4.83 ms on commercial smart￾phones. We measured the processing time for a Samsung S5 with Qualcomm Snapdragon 2.5GHz quad-core CPU. Our system processes sound segments with a size of 1,024 sam￾ples (time duration of 21.3 ms at 48 kHz sampling rate). To reduce the processing time, we only perform the path delay calibration on the first ten data segments and later process￾ing does not require recalibration. Furthermore, we use FFT to accelerate the cross-correlation. The processing time of one segments is 14.58 ms and 3.93 ms for the computational heavy path delay calibration process and the light-weight movement/touch-sensing algorithm. With processing time for other operations, the overall latency for VSkin to process 21.3 ms of data is 4.832 ms on average. Therefore, VSkin can perform realtime movement and touch sensing on commod￾ity mobile devices. VSkin incurs a power consumption of 491.3 mW on com￾mercial smartphones. We use Powertutor [36] to measure the power consumption of our system on Samsung Galaxy S5. Without considering the LCD power consumption, the power consumptions of CPU and Audio are 121.3 mW and 370 mW , respectively. To measure the power consumption overhead of VSkin, we measured the average power con￾sumption in three different states with VSkin: 1) Backlight, with the screen displaying the results, 2) Web Browsing, surfing the Internet with the WiFi on, 3) Gaming, playing mobile games with the WiFi on. The power consumption overheads for these states are 47.8%, 25.4%, and 15.4%, re￾spectively. More than 74.2% additional power consumption comes from the speaker hardware. One possible solution is to design low-power speakers that are specialized for emitting ultrasounds. 8.5 Discussions Different phone setups: VSkin can work on different types of smartphones. We conducted our experiments on four different smartphones, Samsung S5, Huawei Mate7, Samsung Note3, and Samsung S7, with parameters based on the loca￾tions of speaker and microphones. As shown in Figure 13(b), VSkin achieves an average movement distance error of 3.59 cm, 2.96 cm, 4.02 cm and 6.05 cm on the four models, respec￾tively. VSkin also achieves more than 98% touch accuracy for all types of smartphones, as shown in Figure 13(c). Locations of speakers and microphones: The speak￾ers on Samsung S5 and Huawei Mate7 are on the back of the smartphones, while the speakers on Samsung S7 and Samsung Note3 are at the bottom of the smartphones. Our experimental results show that VSkin achieves higher accu￾racy on S5/Mate7 than S7/Note3. Therefore, the locations of the speaker and microphones are critical to the VSkin perfor￾mance. The current design of VSkin requires one speaker and two microphones, with one microphone close to the speaker to measure the movement and the other at the opposite side of the speaker to measure the touch. Further generalization of VSkin to different speaker/microphone setups is left for future study