27 0 +MM block PCA网 44 -Direct (100-cap 72 47D 102 03 10 T nt SVM Classifier Number of Candidate Passwords 2 24 3020 (a)SVM classifiers performance (b)Password inference 2 101 2 321012 3 Figure 15.Keystroke recognition performance with arbitrary initial positions: (a)horizontal (b)vertical (a)SVM classifier performance for 100-category and decoupled directions Figure 14.Confusion matrix of SVM classifier of the decoupled horizontal SVM:(b)Password inference accuracy with different methods. and vertical direction movement distance. Then we evaluate the recognition performance under fixed initial position mode.Figure 13(b)shows the confusion matrix for key recognition.The average recognition accuracy is 77%. and the recognition accuracy of the key'5'is the lowest(57%). 20 80 20 0 80 The possible reason could be that the key5'is located at the center of the numeric keypad so that it has the largest number (a)Impact of different distances (b)Impact of different orientations Figure 16.Password inference accuracy under the impact of different of adjacent keys. directions,distances,and victims. We observe that most of the errors come from adjacent keys. For example,all recognition errors of the key '1'are due to sequences based on their probability in the decreasing order the key 0'and 4'.The key '7'has a 17%probability of and report the probability that the ground truth sequence is being recognized as the key '4'and 7%being recognized as in the top-K candidates.For example,a top-K accuracy of the key '8'.We also noticed that misidentification are more 50%indicates that 50%of the true PIN codes can be found in inclined to the vertical key groups like '147',258'instead of the first K candidate sequences.Figure 15(b)shows the top-1 the horizontal key groups,such as'123'.We believe that this is accuracy is 25.0%,top-10 accuracy is 54.5%for the HMM- related to the position of the keyboard during our experiments based inference.The top-1 accuracy when directly using the (see Figure 12).Given our keyboard placement,for a keystroke output of the 100-category SVM classifier is less than 2%.We action from the'.'key to the target key,the corresponding path also consider the method that directly uses the horizontal and length change in the horizontal direction is more pronounced vertical SVM result to calculate key sequences probabilities than in the vertical direction. As shown with the yellow line,to achieve a success rate In exiting work that use Wi-Fi CSI as side-channel, of 25%,the attacker may need 790 trials using the direct WindTalker [12]achieves comparable 80%mean accuracy at a probability calculation without HMM. distance of 0.75m,but quickly drops to 40%when the distance is 1.5m.Wikey [10]only works for scenarios where the AP is D.Performance under different scenarios within 30cm.Benefit from GPS-regulated oscillators and low- We conducted keystroke recognition experiments in differ- noise amplifiers used in commercial cellular BSs,our LTE- ent environments,to see the impact of different distances, based approach can operate in a distance of 5~15m. NLOS scenario.keyboard orientations.and different victims. Impact of Distance and NLOS:We first evaluate the sys- C.Performance under continuous typing mode tem performance when the victim was at different distances to To evaluate the performance of continuous keystrokes,we the receiving antenna.Figure 16(a)shows the top-K password first evaluate the performance of the keystroke movement inference accuracy under a distance of 5m,10m,15m,and SVM classifier for two different approaches:the 100-category an NLOS scenario (where the attack devices are blocked by SVM that directly estimates the possibility of the 100 possible a 21cm thick concrete wall)as shown in Figure 12.In a key pair transitions and the decoupled horizontal-vertical SVM distance of 5m,we can recover a 6-digits password with over that estimates the movement in the two directions separately. 87%probability within 100 trials.Even at a distance of 15m, The top-3 classification accuracies for different approaches are SpiderMon can still achieve 36%accuracy in ten trials and showed in Figure 15(a).We observe that the performance of over 60%in 100 trials.Because of the good penetration of the 100-category SVM is quite poor due to the much larger LTE signals,our system can achieve 51%accuracy in ten trials number of categories to be classified when compared to the in the NLOS environment with a distance of 5m. decoupled SVM.For the 100-category classifier,the top-3 Impact of Keyboard Orientation:The relative direction accuracy is less than 30%and the top-50 accuracy is still less between the victim and the attacker has serious impacts on than 90%. the performance of our system,as different directions will We evaluate the continuous keystroke sequence inference induce different multi-path environments.We evaluate the performance as follows.For each test keystroke waveform of performance of SpiderMon by placing the keyboard in four 6 digits,we calculate the probabilities for all possible 6-digit different directions (at a distance of 10 meters)so that the sequences with the HMM method.We sort the candidate key receiving antenna was pointed to the left,right,front,and58.6 16.8 14.7 2.9 7.0 6.5 57.7 26.2 7.9 1.7 1.6 7.0 84.7 6.1 0.6 2.5 7.2 18.1 67.0 5.2 3.8 2.4 10.0 18.3 65.5 -2 -1 0 1 2 -2 -1 0 1 2 (a) horizontal 48.6 27.0 5.4 0.0 1.4 16.2 1.4 0.0 57.5 14.0 2.8 8.1 17.5 0.0 0.0 6.5 65.5 16.2 7.3 4.4 0.2 0.1 1.4 7.6 79.5 8.2 3.1 0.0 0.0 4.3 8.6 12.7 62.9 11.4 0.0 1.0 6.7 3.4 4.0 13.5 69.7 1.7 0.0 2.6 0.0 0.0 5.1 38.5 53.8 -3 -2 -1 0 1 2 3 -3 -2 -1 0 1 2 3 (b) vertical Figure 14. Confusion matrix of SVM classifier of the decoupled horizontal and vertical direction movement distance. Then we evaluate the recognition performance under fixed initial position mode. Figure 13(b) shows the confusion matrix for key recognition. The average recognition accuracy is 77%, and the recognition accuracy of the key ‘5’ is the lowest (57%). The possible reason could be that the key ‘5’ is located at the center of the numeric keypad so that it has the largest number of adjacent keys. We observe that most of the errors come from adjacent keys. For example, all recognition errors of the key ‘1’ are due to the key ‘0’ and ‘4’. The key ‘7’ has a 17% probability of being recognized as the key ‘4’ and 7% being recognized as the key ‘8’. We also noticed that misidentification are more inclined to the vertical key groups like ‘147’, ‘258’ instead of the horizontal key groups, such as ‘123’. We believe that this is related to the position of the keyboard during our experiments (see Figure 12). Given our keyboard placement, for a keystroke action from the ‘.’ key to the target key, the corresponding path length change in the horizontal direction is more pronounced than in the vertical direction. In exiting work that use Wi-Fi CSI as side-channel, WindTalker [12] achieves comparable 80% mean accuracy at a distance of 0.75m, but quickly drops to 40% when the distance is 1.5m. Wikey [10] only works for scenarios where the AP is within 30cm. Benefit from GPS-regulated oscillators and low￾noise amplifiers used in commercial cellular BSs, our LTE￾based approach can operate in a distance of 5 ∼ 15m. C. Performance under continuous typing mode To evaluate the performance of continuous keystrokes, we first evaluate the performance of the keystroke movement SVM classifier for two different approaches: the 100-category SVM that directly estimates the possibility of the 100 possible key pair transitions and the decoupled horizontal-vertical SVM that estimates the movement in the two directions separately. The top-3 classification accuracies for different approaches are showed in Figure 15(a). We observe that the performance of the 100-category SVM is quite poor due to the much larger number of categories to be classified when compared to the decoupled SVM. For the 100-category classifier, the top-3 accuracy is less than 30% and the top-50 accuracy is still less than 90%. We evaluate the continuous keystroke sequence inference performance as follows. For each test keystroke waveform of 6 digits, we calculate the probabilities for all possible 6-digit sequences with the HMM method. We sort the candidate key 100-category horizontal-5 vertical-7 Different SVM Classifier 0 50 100 Recognition Accuracy (%) top-1 top-2 top-3 (a) SVM classifiers performance 100 101 102 103 104 Number of Candidate Passwords 0 50 100 Inference Accuracy (%) HMM (block PCA) HMM (traditional PCA) Direct (decoupled) Direct (100-category) (b) Password inference Figure 15. Keystroke recognition performance with arbitrary initial positions: (a) SVM classifier performance for 100-category and decoupled directions SVM; (b) Password inference accuracy with different methods. 20 40 60 80 100 Top-K candidates 0 20 40 60 80 Inference Accuracy (%) 5m 5m NLOS 10m 15m (a) Impact of different distances 20 40 60 80 100 Top-K candidates 0 20 40 60 80 Inference Accuracy (%) Front Back Left Right (b) Impact of different orientations Figure 16. Password inference accuracy under the impact of different directions, distances, and victims. sequences based on their probability in the decreasing order and report the probability that the ground truth sequence is in the top-K candidates. For example, a top-K accuracy of 50% indicates that 50% of the true PIN codes can be found in the first K candidate sequences. Figure 15(b) shows the top-1 accuracy is 25.0%, top-10 accuracy is 54.5% for the HMM￾based inference. The top-1 accuracy when directly using the output of the 100-category SVM classifier is less than 2%. We also consider the method that directly uses the horizontal and vertical SVM result to calculate key sequences probabilities. As shown with the yellow line, to achieve a success rate of 25%, the attacker may need 790 trials using the direct probability calculation without HMM. D. Performance under different scenarios We conducted keystroke recognition experiments in differ￾ent environments, to see the impact of different distances, NLOS scenario, keyboard orientations, and different victims. Impact of Distance and NLOS: We first evaluate the sys￾tem performance when the victim was at different distances to the receiving antenna. Figure 16(a) shows the top-K password inference accuracy under a distance of 5m, 10m, 15m, and an NLOS scenario (where the attack devices are blocked by a 21cm thick concrete wall) as shown in Figure 12. In a distance of 5m, we can recover a 6-digits password with over 87% probability within 100 trials. Even at a distance of 15m, SpiderMon can still achieve 36% accuracy in ten trials and over 60% in 100 trials. Because of the good penetration of LTE signals, our system can achieve 51% accuracy in ten trials in the NLOS environment with a distance of 5m. Impact of Keyboard Orientation: The relative direction between the victim and the attacker has serious impacts on the performance of our system, as different directions will induce different multi-path environments. We evaluate the performance of SpiderMon by placing the keyboard in four different directions (at a distance of 10 meters) so that the receiving antenna was pointed to the left, right, front, and