This article has been accepted for inclusion in a future issue of this journal.Content is final as presented,with the exception of pagination. XIE et al:MULTI-TOUCH IN THE AIR:CONCURRENT MICROMOVEMENT RECOGNITION USING RF SIGNALS phase profiles.Similar to the method proposed in Section 4.4. in order to guarantee the scalability to orientation variation, we can also reconstruct the template phase profiles based on the orientation,by figuring out the angle a;and computing the corresponding phase change.To estimate the facing direction, we let the human subject perform a specified movement,e.g, punch.We can use the 3D positioning method to figure out the moving trajectory,and detect the facing direction accordingly. Then,we can perform the phase profile matching between the RF-Glove testing phase profiles and the reconstructed template phase profiles. Fig.13. Experimental deployment. 2)Robustness to Interference and Noise:When the tag arrays are worn on the hands,due to the factors such as the V.SYSTEM EVALUATION mutual interference,multi-path fading,and energy absorption, various issues such as missing tag readings and distorted phase A.Experimental Setup values might impact the actual system performance.Our solu- We evaluated our system using the ImpinJ [34]R420 tion addresses these issues based on the following techniques. reader,three Laird S9028 RFID antennas,and five Alien 9640 As we deploy three separated antennas and attach one tag on general purpose tags.We deployed the three antennas on a each of the fingers,when the human subject is performing flat plane with the antenna pairs in a mutually orthogonal the micromovement,a total of 3 sets of phase profiles are approach,as shown in Fig.13.By default,we set the distance continuously collected from the tagged fingers.Therefore,the between adjacent antennas to 60cm.Besides,we set the robustness is well guaranteed by providing enough redun- distance between the antenna plane and the operation plane to dancies on the RFID antennas and tags.For example,an about 1.5~2m.We performed recognition on the eight finger arbitrary tag reading might be missing from one antenna,but micromovements in Fig.1.In the following,for simplicity,we it usually can still be read by another antenna,due to different used the abbrevations in Fig.I to denote the corrresponding multi-path environment.Similarly,the phase profiles of one micromovements.For the large-range movement including or more antenna-tag pairs might be distorted due to some swipe left/right and punch,we used the 3D positioning method reason.nevertheless,we can investigate the correlations among to perform movement recognition.For the small-range micro- multiple phase profiles in both time and space domain,e.g.,the movement including zoom in/out,rotate left/right,and flick. phase changes of adjacent tags should be no greater than the we used the two-phase phase profile matching to perform physical constraints of fingers,so as to detect the outliers from micromovement.We obtained the template phase profiles for the phase profiles and further eliminate the distorted phase each micromovement based on 100 training samples from profiles. 10 human subjects. 3)Offset Unexpected Body Movements:In performing the micromovement,people often unintentionally introduce some unexpected body movements such as arm movements into the B.Macro Benchmark micromovements.These body movements result in consistent To evaluate the positioning functionality,we implemented changes to each of the tags'micromovements,and further the 3D positioning method,and evaluated the performance in make their corresponding phase profiles distorted in a con- positioning accuracy and recognition accuracy of large-range sistent approach.The distorted phase profiles often cause the movements.To evaluate the functionality of micromovement test micromovement fail to match to the exact template micro- recognition,we implemented our Phase Profiles-based scheme movement.To accurately classify the finger micromovement, to evaluate the system performance in accuracy and time delay. we actually only concern the relative movement of fingers We let 10 volunteers conduct the three large-range move- instead of the absolute movements introduced by unexpected ments and five small-range micromovements in Fig.1.In this body movements.To evaluate the consistent distortion brought experiment,each movement/micromovement was performed by the arm movement,we can attach one more tag to the back by various volunteers for a total of 100 times.We thus obtained of the hand to serve as the reference tag for measuring arm a total of 800 samples for all gestures.The average sampling movement.By measuring the phase variation of the reference rate of RF signals is about 23 samples/second per tag.We then tag,we can obtain the phase variation of arm movement.used the 10-fold cross-validation to evaluate the classification Moreover,it is known that the actually captured micromove- performance. ment is a composition of the original micromovement and the 1)Evaluate the Positioning Performance:Our position- arm movement.By mapping the displacement to the antenna's ing solution achieves fairly good performance in positioning Z-dimensional axis,the variation of the displacement is accuracy.The average positioning error of our positioning linearly related to the variation of the phase value.There-method is 25cm,whereas the average positioning error of the fore,the phase variation of the original micromovement hologram-based solution is 30cm.To evaluate the performance can be calculated by substracting the phase variation of of our positioning method against the state-of-art solutions, arm movement from the phase variation of the actual we compared it with the hologram-based positioning,which micromovement. is adopted in Tagoram [12].We selected a 4mx4m square areaThis article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. XIE et al.: MULTI-TOUCH IN THE AIR: CONCURRENT MICROMOVEMENT RECOGNITION USING RF SIGNALS 11 phase profiles. Similar to the method proposed in Section 4.4, in order to guarantee the scalability to orientation variation, we can also reconstruct the template phase profiles based on the orientation, by figuring out the angle αi and computing the corresponding phase change. To estimate the facing direction, we let the human subject perform a specified movement, e.g, punch. We can use the 3D positioning method to figure out the moving trajectory, and detect the facing direction accordingly. Then, we can perform the phase profile matching between the testing phase profiles and the reconstructed template phase profiles. 2) Robustness to Interference and Noise: When the tag arrays are worn on the hands, due to the factors such as the mutual interference, multi-path fading, and energy absorption, various issues such as missing tag readings and distorted phase values might impact the actual system performance. Our solu￾tion addresses these issues based on the following techniques. As we deploy three separated antennas and attach one tag on each of the fingers, when the human subject is performing the micromovement, a total of 3 sets of phase profiles are continuously collected from the tagged fingers. Therefore, the robustness is well guaranteed by providing enough redun￾dancies on the RFID antennas and tags. For example, an arbitrary tag reading might be missing from one antenna, but it usually can still be read by another antenna, due to different multi-path environment. Similarly, the phase profiles of one or more antenna-tag pairs might be distorted due to some reason, nevertheless, we can investigate the correlations among multiple phase profiles in both time and space domain, e.g., the phase changes of adjacent tags should be no greater than the physical constraints of fingers, so as to detect the outliers from the phase profiles and further eliminate the distorted phase profiles. 3) Offset Unexpected Body Movements: In performing the micromovement, people often unintentionally introduce some unexpected body movements such as arm movements into the micromovements. These body movements result in consistent changes to each of the tags’ micromovements, and further make their corresponding phase profiles distorted in a con￾sistent approach. The distorted phase profiles often cause the test micromovement fail to match to the exact template micro￾movement. To accurately classify the finger micromovement, we actually only concern the relative movement of fingers instead of the absolute movements introduced by unexpected body movements. To evaluate the consistent distortion brought by the arm movement, we can attach one more tag to the back of the hand to serve as the reference tag for measuring arm movement. By measuring the phase variation of the reference tag, we can obtain the phase variation of arm movement. Moreover, it is known that the actually captured micromove￾ment is a composition of the original micromovement and the arm movement. By mapping the displacement to the antenna’s Z-dimensional axis, the variation of the displacement is linearly related to the variation of the phase value. There￾fore, the phase variation of the original micromovement can be calculated by substracting the phase variation of arm movement from the phase variation of the actual micromovement. Fig. 13. Experimental deployment. V. SYSTEM EVALUATION A. Experimental Setup We evaluated our system using the ImpinJ [34] R420 reader, three Laird S9028 RFID antennas, and five Alien 9640 general purpose tags. We deployed the three antennas on a flat plane with the antenna pairs in a mutually orthogonal approach, as shown in Fig. 13. By default, we set the distance between adjacent antennas to 60cm. Besides, we set the distance between the antenna plane and the operation plane to about 1.5∼2m. We performed recognition on the eight finger micromovements in Fig.1. In the following, for simplicity, we used the abbrevations in Fig.1 to denote the corrresponding micromovements. For the large-range movement including swipe left/right and punch, we used the 3D positioning method to perform movement recognition. For the small-range micro￾movement including zoom in/out, rotate left/right, and flick, we used the two-phase phase profile matching to perform micromovement. We obtained the template phase profiles for each micromovement based on 100 training samples from 10 human subjects. B. Macro Benchmark To evaluate the positioning functionality, we implemented the 3D positioning method, and evaluated the performance in positioning accuracy and recognition accuracy of large-range movements. To evaluate the functionality of micromovement recognition, we implemented our Phase Profiles-based scheme to evaluate the system performance in accuracy and time delay. We let 10 volunteers conduct the three large-range move￾ments and five small-range micromovements in Fig. 1. In this experiment, each movement/micromovement was performed by various volunteers for a total of 100 times. We thus obtained a total of 800 samples for all gestures. The average sampling rate of RF signals is about 23 samples/second per tag. We then used the 10-fold cross-validation to evaluate the classification performance. 1) Evaluate the Positioning Performance: Our position￾ing solution achieves fairly good performance in positioning accuracy. The average positioning error of our positioning method is 25cm, whereas the average positioning error of the hologram-based solution is 30cm. To evaluate the performance of our positioning method against the state-of-art solutions, we compared it with the hologram-based positioning, which is adopted in Tagoram [12]. We selected a 4m×4m square area