adapted to recognize keystrokes because such coarse grained CSI VALUES information does not capture the minor variations in the CSl DIFFERENT SUMCARRIERS values caused by human micro-movements such as those of hands and fingers while typing.Some recent work,namely VIFI ROUTER WiHear.uses CSI values to extract the micro-movements of mouth to recognize 9 syllables in the spoken words 10 However,WiHear uses special hardware including direc- USER'S KEYOARE tional antennas and stepper motors to direct WiFi beams towards speaker's mouth and extract the micro-movements. Figure 1:WiKey System We implemented the WiKey system using COTS devices,i.e. a TP-Link TL-WR1043ND WiFi router and a Lenovo X200 Based on this observation,we design a keystroke extraction laptop with Intel 5300 WiFi NIC.In the evaluation process algorithm that utilizes CSI streams of all transmit-receive we build a keystroke database of 10 human subjects with antenna (TX-RX)pair pairs to determine the approximate IRB approval.WiKey achieves more than 97.5%detection start and end points of individual keystrokes in a given CSI- rate for detecting the keystroke and 96.4%recognition ac- waveform by continuously matching the trends in CSI time curacy for classifying single keys.In real-world experiments, series with the experimentally observed trends using a slid- WiKey can recognize keystrokes in a continuously typed sen- tence with an accuracy of 93.5% ing window approach. The second technical challenge is to extract distinguishing In this paper,we have shown that fine grained activity features for generating classification models for each of the recognition is possible by using COTS WiFi devices.Thus, 37 keys (10 digits,26 alphabets and 1 space-bar).As the the techniques proposed in this paper can be used for sev- keys on a keyboard are closely placed,conventional features eral HCI applications.Examples include zoom-in,zoom-out, such as maximum peak power,mean amplitude,root mean scrolling,sliding,and rotating gestures for operating per- square deviation of signal amplitude,second/third central sonal computers,gesture recognition for gaming consoles moment,rate of change,signal energy or entropy,and num- in-home gesture recognition for operating various household ber of zero crossings cannot be used because the values of devices,and applications such as writing and drawing in the these features for adjacent keys are almost identical.To air.Other than being a potential attack,our WiKey tech- address this challenge,we use the CSI-waveform shapes of nology can be potentially used to build virtual keyboards each key from each TX-RX antenna pair as features.As the where human users type on a printed keyboard. waveforms for each key contain a large number of samples. we apply the Discrete Wavelet Transform (DWT)technique 2. RELATED WORK on these waveforms to reduce the number of samples while keeping the shape preserving time and frequency domain in- 2.1 Device Free Activity Recognition formation intact.We use the waveforms resulting from the Device-free activity recognition solutions use the vari- DWT of individual keystrokes as their shape features. ations in wireless channel to recognize human activities in a The third technical challenge is to compare shape fea- given environment.Existing solutions can be grouped into tures of any two keystrokes.The midpoints of extracted three categories:(1)Received Signal Strength (RSS)based, CSI-wavforms of different keystrokes rarely align with each (2)CSI based,and (3)Software Defined Radio (SDR)based. other because the start and end points determined by ex- RSS Based:Sigg et al.proposed activity recognition traction algorithm are never exact.Moreover,the lengths schemes that utilize RSS values of WiFi signals to recog- of different keystroke waveforms also differ because the dur- nize four activities including crawling,lying down,standing ation of pressing any key is often different.Consequently up,and walking [11,12].They achieved activity recognition the midpoints and lengths of shape features do not match rates of over 80%for these four activities.To obtain the either.Another issue is that the shape of different keystroke RSS values from WiFi signals,they used USRPs,which are waveforms of the same key are often distorted versions of specialized hardware devices compared to the COTS WiFi each other because of slightly different formation and dir- devices that we used in our work.While RSS values can be ection of motion of hands and fingers while pressing that used for recognizing macro-movements,they are not suit- key.Thus,two shape features cannot be compared using able to recognize the micro-movements such as those of fin- standard measures like correlation coefficient or Euclidean gers and hands in keyboard typing because RSS values only distance.To address this challenge,we use the Dynamic provide coarse-grained information about the channel vari- Time Warping (DTW)technique to quantify the distance ations and do not contain fine-grained information about between the two shape features.DTW can find the min- small scale fading and multi-path effects caused by these imum distance alignment between two waveforms of differ- micro-movements. ent lengths. CSI Based:CSI values obtained from COTS WiFI net- The key novelty of this paper is on proposing the first work interface cards (NICs)(such as Intel 5300 and Ath- WiFi signal based keystroke recognition approach.Some re- eros 9390)have been recently proposed for activity recogni- cent work uses CSI values to recognize various macro aspects tion [6-10,13]and localization [14-16].Han et al.proposed of human movements such as falling down 6,household WiFall that detects fall of a human subject in an indoor activities [7],detection of human presence [8],and estim- environment using CSI values [6].Zhou et al.proposed a ating the number of people in a crowd [9].These schemes passive human detection scheme which exploits multi-path extract coarse grained information from the CSI values to variations for detecting human presence in an indoor envir- recognize the macro-movements such as falling down or re- onment using CSI values [8].Zou et al.proposed Electronic cognizing fullbody/limb gestures.They cannot be directly Frog Eye that counts the number of people in a crowd usingFigure 1: WiKey System Based on this observation, we design a keystroke extraction algorithm that utilizes CSI streams of all transmit-receive antenna (TX-RX) pair pairs to determine the approximate start and end points of individual keystrokes in a given CSI￾waveform by continuously matching the trends in CSI time series with the experimentally observed trends using a slid￾ing window approach. The second technical challenge is to extract distinguishing features for generating classification models for each of the 37 keys (10 digits, 26 alphabets and 1 space-bar). As the keys on a keyboard are closely placed, conventional features such as maximum peak power, mean amplitude, root mean square deviation of signal amplitude, second/third central moment, rate of change, signal energy or entropy, and num￾ber of zero crossings cannot be used because the values of these features for adjacent keys are almost identical. To address this challenge, we use the CSI-waveform shapes of each key from each TX-RX antenna pair as features. As the waveforms for each key contain a large number of samples, we apply the Discrete Wavelet Transform (DWT) technique on these waveforms to reduce the number of samples while keeping the shape preserving time and frequency domain in￾formation intact. We use the waveforms resulting from the DWT of individual keystrokes as their shape features. The third technical challenge is to compare shape fea￾tures of any two keystrokes. The midpoints of extracted CSI-wavforms of different keystrokes rarely align with each other because the start and end points determined by ex￾traction algorithm are never exact. Moreover, the lengths of different keystroke waveforms also differ because the dur￾ation of pressing any key is often different. Consequently, the midpoints and lengths of shape features do not match either. Another issue is that the shape of different keystroke waveforms of the same key are often distorted versions of each other because of slightly different formation and dir￾ection of motion of hands and fingers while pressing that key. Thus, two shape features cannot be compared using standard measures like correlation coefficient or Euclidean distance. To address this challenge, we use the Dynamic Time Warping (DTW) technique to quantify the distance between the two shape features. DTW can find the min￾imum distance alignment between two waveforms of differ￾ent lengths. The key novelty of this paper is on proposing the first WiFi signal based keystroke recognition approach. Some re￾cent work uses CSI values to recognize various macro aspects of human movements such as falling down [6], household activities [7], detection of human presence [8], and estim￾ating the number of people in a crowd [9]. These schemes extract coarse grained information from the CSI values to recognize the macro-movements such as falling down or re￾cognizing fullbody/limb gestures. They cannot be directly adapted to recognize keystrokes because such coarse grained information does not capture the minor variations in the CSI values caused by human micro-movements such as those of hands and fingers while typing. Some recent work, namely WiHear, uses CSI values to extract the micro-movements of mouth to recognize 9 syllables in the spoken words [10]. However, WiHear uses special hardware including direc￾tional antennas and stepper motors to direct WiFi beams towards speaker’s mouth and extract the micro-movements. We implemented the WiKey system using COTS devices, i.e. a TP-Link TL-WR1043ND WiFi router and a Lenovo X200 laptop with Intel 5300 WiFi NIC. In the evaluation process, we build a keystroke database of 10 human subjects with IRB approval. WiKey achieves more than 97.5% detection rate for detecting the keystroke and 96.4% recognition ac￾curacy for classifying single keys. In real-world experiments, WiKey can recognize keystrokes in a continuously typed sen￾tence with an accuracy of 93.5%. In this paper, we have shown that fine grained activity recognition is possible by using COTS WiFi devices. Thus, the techniques proposed in this paper can be used for sev￾eral HCI applications. Examples include zoom-in, zoom-out, scrolling, sliding, and rotating gestures for operating per￾sonal computers, gesture recognition for gaming consoles, in-home gesture recognition for operating various household devices, and applications such as writing and drawing in the air. Other than being a potential attack, our WiKey tech￾nology can be potentially used to build virtual keyboards where human users type on a printed keyboard. 2. RELATED WORK 2.1 Device Free Activity Recognition Device-free activity recognition solutions use the vari￾ations in wireless channel to recognize human activities in a given environment. Existing solutions can be grouped into three categories: (1) Received Signal Strength (RSS) based, (2) CSI based, and (3) Software Defined Radio (SDR) based. RSS Based: Sigg et al. proposed activity recognition schemes that utilize RSS values of WiFi signals to recog￾nize four activities including crawling, lying down, standing up, and walking [11, 12]. They achieved activity recognition rates of over 80% for these four activities. To obtain the RSS values from WiFi signals, they used USRPs, which are specialized hardware devices compared to the COTS WiFi devices that we used in our work. While RSS values can be used for recognizing macro-movements, they are not suit￾able to recognize the micro-movements such as those of fin￾gers and hands in keyboard typing because RSS values only provide coarse-grained information about the channel vari￾ations and do not contain fine-grained information about small scale fading and multi-path effects caused by these micro-movements. CSI Based: CSI values obtained from COTS WiFI net￾work interface cards (NICs) (such as Intel 5300 and Ath￾eros 9390) have been recently proposed for activity recogni￾tion [6–10, 13] and localization [14–16]. Han et al. proposed WiFall that detects fall of a human subject in an indoor environment using CSI values [6]. Zhou et al. proposed a passive human detection scheme which exploits multi-path variations for detecting human presence in an indoor envir￾onment using CSI values [8]. Zou et al. proposed Electronic Frog Eye that counts the number of people in a crowd using