CSI values by treating the people reflecting the WiFi signals used cepstrum features [22]instead of FFT as keystroke fea- as "virtual antennas"[9].Wang et al.proposed E-eyes that tures and used unsupervised learning with language model exploits CSI values for recognizing household activities such correction on the collected features before using them for as washing dishes and taking a shower [7.Nandakumar et supervised training and recognition of different keystrokes al.leverage the CSI and RSS information from off-the-shelf Zhu et al.proposed a context-free geometry-based approach WiFi devices to classify four arm gestures-push,pull,lever for recognizing keystrokes that leverage the acoustic eman- and punch [13].The fundamental difference between these ations from keystrokes to first calculate the time difference schemes and our scheme is that these schemes extract coarse of keystroke arrival and then estimate the physical locations grained features from the CSI values provided by the COTS of the keystrokes to identify which keys are pressed [3. WiFi NIC to perform these tasks while our proposed scheme Electromagnetic Emissions Based Vuagnoux et al. refines these CSI to capture fine grained variations in the used a USRP to capture the electromagnetic emanations wireless channel for recognizing keystrokes.Wang et al.pro- while pressing the keys [4.These electromagnetic emana- pose WiHear that uses CSI values recognizes the shape of tions originated from the electrical circuit underneath each mouth while speaking to detect whether a person is utter- key in conventional keyboards.The authors proposed to cap- ing one of a set of nine predefined nine syllables [10].While ture the entire raw electromagnetic spectrum and process it WiHear can capture the micro-movements of lips,it uses to recognize the keystrokes.Unfortunately,this scheme is special purpose directional antennas with stepper motors highly susceptible to background electromagnetic noise that for directing the antenna beams towards a person's mouth exists in almost all environments these days such as due to to obtain a clean signal for recognizing mouth movements microwave ovens,refrigerators,and televisions. In contrast,our proposed scheme does not use any special Video Camera Based Balzarotti et al.proposed purpose equipment and recognizes the micro-movements of ClearShot that processes the video of a person typing to fingers and hands using COTS WiFi NIC reconstruct the sentences (s)he types 5.The authors pro- SDR Based:Researchers have proposed schemes that pose to use context and language sensitive analysis for re- utilize SRDs and special purpose hardware to transmit and constructing the sentences receive custom modulated signals for activity recognition [17-20].Pu et al.proposed WiSee that uses a special pur- 3.CHANNEL STATE INFORMATION pose receiver design on USRPs to extract small Doppler shifts from OFDM WiFi transmissions to recognize human Modern WiFi devices that support IEEE 802.11n/ac gestures [17].Kellogg et al.proposed to use a special pur- standard typically consist of multiple transmit and mul- pose analog envelop detector circuit for recognizing gestures tiple receive antennas and thus support MIMO.Each MIMO within a distance of up to 2.5 feet using backscatter sig- channel between each transmit-receive (TX-RX)antenna nals from RFID or TV transmissions 18.Lyonnet et al. pair of a transmitter and receiver comprises of multiple sub- use micro Doppler signatures to classify gaits of human carriers.These WiFi devices continuously monitor the state subjects into multiple categories using specialized Doppler of the wireless channel to effectively perform transmit power radars [19].Adib et al.proposed WiTrack that uses a spe- allocations and rate adaptations for each individual MIMO stream such that the available capacity of the wireless chan- cially designed frequency modulated carrier wave radio fron- tend to track human movements behind a wall [20].Recently. nel is maximally utilized [23.These devices quantify the state of the channel in terms of CSI values.The CSI val- Chen et al.proposed an SDR based custom receiver design which can be used to track keystrokes using wireless sig- ues essentially characterize the Channel Frequency Response nals [21].In contrast to all these schemes,our scheme does (CFR)for each subcarrier between each transmit-receive not use any specialized hardware or SDRs rather utilizes (TX-RX)antenna pair.As the received signal is the res- COTS WiFi NICs to recognize keystrokes ultant of constructive and destructive interference of several multipath signals scattered from the walls and surrounding objects,the disturbances caused by movement of hands and 2.2 Keystrokes Recognition fingers while typing on a keyboard near the WiFi receiver To the best of our knowledge,there is no prior work on re- not only lead to changes in previously existing multipaths cognizing keystrokes by leveraging variations in wireless sig- but also to the creation of new multipaths.These changes nals using commodity WiFi devices.Other than the SDRs are captured in the CSI values for all subcarriers between based keystroke tracking approach proposed in [21 which every TX-RX antenna pair and can then be used to recog- uses wireless signals to track keystrokes,researchers have nize keystrokes. proposed several keystrokes recognition schemes that are Let Mr denote the number of transmit antennas,MR de- based on other sensing modalities such as acoustics 1-3,22 note the number of receive antennas and Se denote the num- electromagnetic emissions 4,and video cameras 5.Next, ber of OFDM sub-carriers.Let Xi and Y;represent the MT we give a brief overview of the other existing schemes that dimensional transmitted signal vector and MR dimensional utilize these sensing modalities to recognize keystrokes. received signal vector,respectively,for subcarrier i and let Acoustics Based:Asonov et al.proposed a scheme Ni represent an MR dimensional noise vector.An MR x MT to recognize keystrokes by leveraging the observation that MIMO system at any time instant can be represented by the different keys of a given keyboard produce slightly dif- following equation. ferent sounds during regular typing [1].They used back- Yi=H:Xi+Wii∈1，Se (1) propagation neural network for keystroke recognition and fast fourier transform (FFT)of the time window of every In the equation above,the MR x Mr dimensional channel keystroke peak as features for training the classifiers.Zhuang matrix Hi represents the Channel State Information (CSI) et al.proposed another scheme that recognizes keystrokes for the sub-carrier i.Any two communicating WiFi devices based on the sounds generated during key presses [2].They estimate this channel matrix Hi for every subcarrier by reg-CSI values by treating the people reflecting the WiFi signals as “virtual antennas” [9]. Wang et al. proposed E-eyes that exploits CSI values for recognizing household activities such as washing dishes and taking a shower [7]. Nandakumar et al. leverage the CSI and RSS information from off-the-shelf WiFi devices to classify four arm gestures - push, pull, lever, and punch [13]. The fundamental difference between these schemes and our scheme is that these schemes extract coarse grained features from the CSI values provided by the COTS WiFi NIC to perform these tasks while our proposed scheme refines these CSI to capture fine grained variations in the wireless channel for recognizing keystrokes. Wang et al. pro￾pose WiHear that uses CSI values recognizes the shape of mouth while speaking to detect whether a person is utter￾ing one of a set of nine predefined nine syllables [10]. While WiHear can capture the micro-movements of lips, it uses special purpose directional antennas with stepper motors for directing the antenna beams towards a person’s mouth to obtain a clean signal for recognizing mouth movements. In contrast, our proposed scheme does not use any special purpose equipment and recognizes the micro-movements of fingers and hands using COTS WiFi NIC. SDR Based: Researchers have proposed schemes that utilize SRDs and special purpose hardware to transmit and receive custom modulated signals for activity recognition [17–20]. Pu et al. proposed WiSee that uses a special pur￾pose receiver design on USRPs to extract small Doppler shifts from OFDM WiFi transmissions to recognize human gestures [17]. Kellogg et al. proposed to use a special pur￾pose analog envelop detector circuit for recognizing gestures within a distance of up to 2.5 feet using backscatter sig￾nals from RFID or TV transmissions [18] . Lyonnet et al. use micro Doppler signatures to classify gaits of human subjects into multiple categories using specialized Doppler radars [19]. Adib et al. proposed WiTrack that uses a spe￾cially designed frequency modulated carrier wave radio fron￾tend to track human movements behind a wall [20]. Recently, Chen et al. proposed an SDR based custom receiver design which can be used to track keystrokes using wireless sig￾nals [21]. In contrast to all these schemes, our scheme does not use any specialized hardware or SDRs rather utilizes COTS WiFi NICs to recognize keystrokes. 2.2 Keystrokes Recognition To the best of our knowledge, there is no prior work on re￾cognizing keystrokes by leveraging variations in wireless sig￾nals using commodity WiFi devices. Other than the SDRs based keystroke tracking approach proposed in [21] which uses wireless signals to track keystrokes, researchers have proposed several keystrokes recognition schemes that are based on other sensing modalities such as acoustics [1–3,22], electromagnetic emissions [4], and video cameras [5]. Next, we give a brief overview of the other existing schemes that utilize these sensing modalities to recognize keystrokes. Acoustics Based: Asonov et al. proposed a scheme to recognize keystrokes by leveraging the observation that different keys of a given keyboard produce slightly dif￾ferent sounds during regular typing [1]. They used back￾propagation neural network for keystroke recognition and fast fourier transform (FFT) of the time window of every keystroke peak as features for training the classifiers. Zhuang et al. proposed another scheme that recognizes keystrokes based on the sounds generated during key presses [2]. They used cepstrum features [22] instead of FFT as keystroke fea￾tures and used unsupervised learning with language model correction on the collected features before using them for supervised training and recognition of different keystrokes. Zhu et al. proposed a context-free geometry-based approach for recognizing keystrokes that leverage the acoustic eman￾ations from keystrokes to first calculate the time difference of keystroke arrival and then estimate the physical locations of the keystrokes to identify which keys are pressed [3]. Electromagnetic Emissions Based Vuagnoux et al. used a USRP to capture the electromagnetic emanations while pressing the keys [4]. These electromagnetic emana￾tions originated from the electrical circuit underneath each key in conventional keyboards. The authors proposed to cap￾ture the entire raw electromagnetic spectrum and process it to recognize the keystrokes. Unfortunately, this scheme is highly susceptible to background electromagnetic noise that exists in almost all environments these days such as due to microwave ovens, refrigerators, and televisions. Video Camera Based Balzarotti et al. proposed ClearShot that processes the video of a person typing to reconstruct the sentences (s)he types [5]. The authors pro￾pose to use context and language sensitive analysis for re￾constructing the sentences. 3. CHANNEL STATE INFORMATION Modern WiFi devices that support IEEE 802.11n/ac standard typically consist of multiple transmit and mul￾tiple receive antennas and thus support MIMO. Each MIMO channel between each transmit-receive (TX-RX) antenna pair of a transmitter and receiver comprises of multiple sub￾carriers. These WiFi devices continuously monitor the state of the wireless channel to effectively perform transmit power allocations and rate adaptations for each individual MIMO stream such that the available capacity of the wireless chan￾nel is maximally utilized [23]. These devices quantify the state of the channel in terms of CSI values. The CSI val￾ues essentially characterize the Channel Frequency Response (CFR) for each subcarrier between each transmit-receive (TX-RX) antenna pair. As the received signal is the res￾ultant of constructive and destructive interference of several multipath signals scattered from the walls and surrounding objects, the disturbances caused by movement of hands and fingers while typing on a keyboard near the WiFi receiver not only lead to changes in previously existing multipaths but also to the creation of new multipaths. These changes are captured in the CSI values for all subcarriers between every TX-RX antenna pair and can then be used to recog￾nize keystrokes. Let MT denote the number of transmit antennas, MR de￾note the number of receive antennas and Sc denote the num￾ber of OFDM sub-carriers. Let Xi and Yi represent the MT dimensional transmitted signal vector and MR dimensional received signal vector, respectively, for subcarrier i and let Ni represent an MR dimensional noise vector. An MR ×MT MIMO system at any time instant can be represented by the following equation. Yi = HiXi + Ni i ∈ [1, Sc] (1) In the equation above, the MR × MT dimensional channel matrix Hi represents the Channel State Information (CSI) for the sub-carrier i. Any two communicating WiFi devices estimate this channel matrix Hi for every subcarrier by reg-