RFID Peak-to-peak antenna amplitude of RSSI RFID tag Peak-to-peak amplitude of phas 60 Vertical Horizontal 500 1000 1500 200 4006008001000 1200 Sample index Sample index (a)Experiment setup (b)Received signal of vertical movement (c)Received signal of horizontal movement Fig.2.Preliminary study of the RF signal reflection. VR applications and IoT operations. perpendicular way to reduce the interference. The recent advances demonstrate that the emerging RFID The contributions of RF-finger are summarized as follows: technology not only can sense the status of objects with i)We design a new device-free solution based on Commercial- device-based solutions [7,10-12,20],but also has the po- Off-The-Shelf (COTS)RFID for both finger tracking and tential to provide device-free sensing by leveraging the multi- multi-touch gesture recognition.To the best of our knowledge, path effect [4,21].In this work,we present RF-finger,a we are the first to recognize the multi-touch gestures based on device-free system based on RFID tag array,to sense the a RFID system through a device-free approach.ii)We build a fine-grained finger movements.Unlike previous studies,which theoretical model to depict the reflection relationship between either locate the human body in a coarse-grained manner [21] the tag array and the fingers caused by the multi-path effect. or simply detect single stroke from the hand movement for The theoretical model provides guidelines to develop two algo- letter recognition [4],RF-finger focuses on tracking the finger rithms to track the finger trajectories and recognize the multi- trace and recognizing the multi-touch gestures,which involves touch gestures.iii)We experimentally investigate the impact a smaller tracking subject and more complicated multi-touch of tag array deployment on the signal quality.We analyze gestures than existing problems.As shown in Figure 1,by the mutual interference between tags via a signal model and leveraging the tag array attached on a letter-size paper,RF- provide recommendations on tag deployment to reduce the finger seeks to support different applications including writing, interference.iv)We implement a system prototype,RF-finger, multi-touch operations,gaming,etc. for finger tacking and gesture recognition.Experiments show Specifically,we deploy only one RFID antenna behind that RF-finger can achieve the average accuracy of 88%and the tag array to continuously measure the signals emitted 92%for finger tracking and gesture recognition,respectively. from the tag array,and recognize the gestures based on II.PRELIMINARIES CHALLENGES the corresponding signal changes.In designing the RF-finger system,we need to solve three main challenging problems.i) In order to design a system to track the fine-grained finger How to track the trajectory of the finger writings?Since the movements,we first conduct several preliminary studies on finger usually affects several adjacent tags due to the multi- the impact of finger movement on the RF-signals,and the path effect,it is inaccurate to locate the finger as the position feasibility to use RFID tag array for gesture recognition. of tags.In our work,we theoretically model the impact of Based on the observations,we summary three challenges for the moving finger on the tag array to extract the reflection designing our system. features,and then exploit the reflection feature to pinpoint A.Preliminaries the finger with a cm-level resolution.ii)How to recognize Impact of Finger Movement on RF-Signals.RFID tech- the multi-touch gesture?Multi-touch gesture indicates the RF- nique has been widely used in locating and sensing system signals reflected from multiple fingers are mixed together in based on the physical modalities on RF-signal [20],i.e.,phase the tag array,making it even more difficult to distinguish these and Received Signal Strength Indicator (RSSD).Moreover. fingers for gesture recognition.To address this problem,we when a human moves around the tag,both the phase and regard the multiple fingers as a whole for recognition and RSSI are changing accordingly due to the multi-path en- then extract the reflection feature of the multiple fingers as vironment variance [21.Therefore,we first investigate the images.We then leverage a Convolutional Neural Network impact of finger movement on RF-signals,which is much (CNN)to automatically classify the corresponding gestures smaller than human body.As shown in Figure 2(a),a typical from the image features.iii)How to obtain stable signal finger movement can be decomposed into two basic directions: quality from the tag array?In real RFID systems,misreading horizontal movement (i.e.,swipe in front of the tag)and is a common phenomenon due to the dynamic environments vertical movement (i.e.,approach/departure the tag).Hence. that affects the signal quality,especially when reading multiple we conduct two experiments to investigate the influence of tags simultaneously,such as a tag array.To address this these two finger movements.Figure 2(b)presents the signal's problem,we utilize a signal model to depict the mutual phase and RSSI readings when the finger is moving towards interference between tags,which provides recommendations (i.e.,vertically)the tag from 20cm away.We find that both on tag deployment that re-arranges the adjacent tags in a the phase and RSSI readings change in a wavy pattern,and 2RFID antenna RFID tag Vertical Horizontal (a) Experiment setup Sample index 0 500 1000 1500 Phase (radian) 1 2 3 4 RSSI (dBm) -65 -60 -55 -50 Peak-to-peak amplitude of RSSI Peak-to-peak amplitude of phase (b) Received signal of vertical movement Sample index 0 200 400 600 800 1000 1200 Phase (radian) 3 3.5 4 4.5 5 RSSI (dBm) -48 -47 -46 -45 -44 -43 (c) Received signal of horizontal movement Fig. 2. Preliminary study of the RF signal reflection. VR applications and IoT operations. The recent advances demonstrate that the emerging RFID technology not only can sense the status of objects with device-based solutions [7, 10–12, 20], but also has the po￾tential to provide device-free sensing by leveraging the multi￾path effect [4, 21]. In this work, we present RF-finger, a device-free system based on RFID tag array, to sense the fine-grained finger movements. Unlike previous studies, which either locate the human body in a coarse-grained manner [21] or simply detect single stroke from the hand movement for letter recognition [4], RF-finger focuses on tracking the finger trace and recognizing the multi-touch gestures, which involves a smaller tracking subject and more complicated multi-touch gestures than existing problems. As shown in Figure 1, by leveraging the tag array attached on a letter-size paper, RF- finger seeks to support different applications including writing, multi-touch operations, gaming, etc. Specifically, we deploy only one RFID antenna behind the tag array to continuously measure the signals emitted from the tag array, and recognize the gestures based on the corresponding signal changes. In designing the RF-finger system, we need to solve three main challenging problems. i) How to track the trajectory of the finger writings? Since the finger usually affects several adjacent tags due to the multi￾path effect, it is inaccurate to locate the finger as the position of tags. In our work, we theoretically model the impact of the moving finger on the tag array to extract the reflection features, and then exploit the reflection feature to pinpoint the finger with a cm-level resolution. ii) How to recognize the multi-touch gesture? Multi-touch gesture indicates the RF￾signals reflected from multiple fingers are mixed together in the tag array, making it even more difficult to distinguish these fingers for gesture recognition. To address this problem, we regard the multiple fingers as a whole for recognition and then extract the reflection feature of the multiple fingers as images. We then leverage a Convolutional Neural Network (CNN) to automatically classify the corresponding gestures from the image features. iii) How to obtain stable signal quality from the tag array? In real RFID systems, misreading is a common phenomenon due to the dynamic environments that affects the signal quality, especially when reading multiple tags simultaneously, such as a tag array. To address this problem, we utilize a signal model to depict the mutual interference between tags, which provides recommendations on tag deployment that re-arranges the adjacent tags in a perpendicular way to reduce the interference. The contributions of RF-finger are summarized as follows: i) We design a new device-free solution based on Commercial￾Off-The-Shelf (COTS) RFID for both finger tracking and multi-touch gesture recognition. To the best of our knowledge, we are the first to recognize the multi-touch gestures based on a RFID system through a device-free approach. ii) We build a theoretical model to depict the reflection relationship between the tag array and the fingers caused by the multi-path effect. The theoretical model provides guidelines to develop two algo￾rithms to track the finger trajectories and recognize the multi￾touch gestures. iii) We experimentally investigate the impact of tag array deployment on the signal quality. We analyze the mutual interference between tags via a signal model and provide recommendations on tag deployment to reduce the interference. iv) We implement a system prototype, RF-finger, for finger tacking and gesture recognition. Experiments show that RF-finger can achieve the average accuracy of 88% and 92% for finger tracking and gesture recognition, respectively. II. PRELIMINARIES & CHALLENGES In order to design a system to track the fine-grained finger movements, we first conduct several preliminary studies on the impact of finger movement on the RF-signals, and the feasibility to use RFID tag array for gesture recognition. Based on the observations, we summary three challenges for designing our system. A. Preliminaries Impact of Finger Movement on RF-Signals. RFID tech￾nique has been widely used in locating and sensing system based on the physical modalities on RF-signal [20], i.e., phase and Received Signal Strength Indicator (RSSI). Moreover, when a human moves around the tag, both the phase and RSSI are changing accordingly due to the multi-path en￾vironment variance [21]. Therefore, we first investigate the impact of finger movement on RF-signals, which is much smaller than human body. As shown in Figure 2(a), a typical finger movement can be decomposed into two basic directions: horizontal movement (i.e., swipe in front of the tag) and vertical movement (i.e., approach/departure the tag). Hence, we conduct two experiments to investigate the influence of these two finger movements. Figure 2(b) presents the signal’s phase and RSSI readings when the finger is moving towards (i.e., vertically) the tag from 20cm away. We find that both the phase and RSSI readings change in a wavy pattern, and 2