This article has been accepted for publication in a future issue of this journal,but has not been fully edited.Content may change prior to final publication.Citation information:DOI 10.1109/TMC.2018.2857812.IEEE Transactions on Mobile Computing objects are set to 10 and 50cm,respectively,the average traces of tags to the corresponding human subjects,we are missing ratio of tags/objects is set to 10%,the average able to match the mobile tagged human subjects.Therefore, cardinality and distance of extra interference objects are set to recognize multiple tagged human subjects in the mobile to 2 and 50cm,respectively.In Fig.10(a),we evaluate the situation,in this section,we propose a continuous scanning- match ratios by varying the cardinalities of tagged objects. based solution to pair the mobile tags with moving human It is found that SMM always achieves the best performance subjects via trace matching. than the other two solutions.In Fig.10(b),we evaluate the match ratio by varying the missing ratio of tags or objects for the tagged objects.As the missing ratio increases from 0% 6.2 Pair the Tags with Mobile Human Subjects via Trace to 40%,the matching ratio of HM gradually decreases from Matching 98%to 73%.This implies that HM cannot effectively tackle When deploying our system in front of multiple human the outliers due to the missing tags/objects.Nevertheless, subjects,where the human subjects wearing RFID badges SMM always achieves the match ratios greater than 92%, are moving around,it is known that the state-of-art depth it effectively tackles the outliers of missing tags/objects.In camera such as Kinect is able to extact the skeleton models Fig.10(c)and Fig.10(d),we evaluate the match ratios by from the human subjects.Based on the skeleton model,we varying the cardinalities of the extra interference objects, can further extract the spinemid point [1]from the skeleton to and the average distance between the interference objects represent the human subject,which is also very close to the and tagged objects,respectively.In all situations,SMM place of RFID badge worn by the human subject,as shown achieves the best performance over the other solutions. in Fig.11(a).According to the two-dimensional coordinate of the spinemid point in the horizontal plane,we can figure out the moving traces of different human subjects from the depth camera,as shown in Fig.11(b).Moreover,suppose the reader/depth camera is deployed in the origin O,for any spinemid point P,we can use the angle profile to denote the (HM angle between the vector OP and the X-axis OX,as shown in Fig.11(b). of Tagged Objects ( (a) Different As aforementioned,using the RFID antenna pair,our tagged objects tagged objects system can estimate the Angle of Arrival(AoA)of the RFID tag in the horizontal plane.Then,we can similarly use the angle profile to denote the angle between the AoA direction of the tag and the X-axis.Recall that according to the phase values collected from the RFID antenna pair,there could be multiple solutions for the angle of arrival of the RFID tag. ng (HM CHM Mamage Matching (SMM Hence,there could be multiple angle profiles corresponding 100 150 200 to the specified tag.Therefore,while the tagged human 0 4 Average Distance between Interference Cardinality of Interference Objects Objects and Tagged Objects(cm) subjects are moving from time to time,we can plot the (c)Different cardinalities of inter-(d)Different distances between angle profiles for both the human subjects and the tags over ference objects interference objects and tagged objects time.Fig.11(c)shows the corresponding angle profiles for the human subjects and RFID tags over time,where the Tag Fig.10.Performance Evaluation i is worn on the Body i.Note that for a specified tag,there 6 MATCH THE MOBILE TAGGED HUMAN SUB- are multiple solutions for its angle profile,we use the same JECTS VIA CONTINUOUS SCANNING color to label them.We can observe that the angle profile of the specified body has very close variation trend to one 6.1 Motivation of the angle profiles of the corresponding RFID tag,as they In most cases,the AR systems are designed towards a share very similar moving traces in the horizontal plane. mobile scenario,e.g.,multiple human subjects wearing Therefore,in order to evaluate the correlation of the angle RFID badges are continuously moving around.For this profile between the bodies and tags,we use the difference mobile situation,the rotate scanning-based solution for score to denote this correlation.Specifically,in a specified recognizing multiple stationary tagged objects is no longer sliding window W with length L,for the Ith snapshot suitable.Since the locations of the tagged human subjects (1<I<L),suppose the angle profiles of the body Oi are continuously changing,the scanning frequency of the and the tag Ti are ai(l)and [a ()}respectively.Then,the rotate scanning-based solution cannot be high enough to difference score sij between ai and a,in W is as follows: locate the positions of the tags and human subjects in a real-time manner.Nevertheless,we observe that,when multiple tagged human subjects are continuously moving, Si.j= min ajeta;)L =1 (a(0-ag0)2. (6) their moving traces in the two-dimensional space can be distinguishable among each other.Hence,according to the Here we enumerate all feasible angle profiles a;for the tag depth information and the phase information extracted from T;to compare with the angle profile of ai for the body multiple tagged human subjects,we are able to derive some Oi,and obtain the minimum value as the difference score metric to depict the moving traces for the tags and human si.j.Fig.11(d)shows the difference scores in angle profiles subjects,respectively.In this way,by matching the moving between various pairs of tags and bodies,it is found that 1536-1233(c)2018 IEEE Personal use is permitted,but republication/redistribution requires IEEE permission.See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.1536-1233 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMC.2018.2857812, IEEE Transactions on Mobile Computing 9 objects are set to 10 and 50cm, respectively, the average missing ratio of tags/objects is set to 10%, the average cardinality and distance of extra interference objects are set to 2 and 50cm, respectively. In Fig. 10(a), we evaluate the match ratios by varying the cardinalities of tagged objects. It is found that SMM always achieves the best performance than the other two solutions. In Fig. 10(b), we evaluate the match ratio by varying the missing ratio of tags or objects for the tagged objects. As the missing ratio increases from 0% to 40%, the matching ratio of HM gradually decreases from 98% to 73%. This implies that HM cannot effectively tackle the outliers due to the missing tags/objects. Nevertheless, SMM always achieves the match ratios greater than 92%, it effectively tackles the outliers of missing tags/objects. In Fig. 10(c) and Fig. 10(d), we evaluate the match ratios by varying the cardinalities of the extra interference objects, and the average distance between the interference objects and tagged objects, respectively. In all situations, SMM achieves the best performance over the other solutions. Cardinality of Tagged Objects 5 10 15 20 Matching Ratio 0 0.2 0.4 0.6 0.8 1 Greedy Matching (GM) Hungarian Matching (HM) Stable Marriage Matching (SMM) (a) Different cardinalities of tagged objects Missing Ratio of Tagged Objects (%) 0 10 20 30 40 Matching Ratio 0 0.2 0.4 0.6 0.8 1 Greedy Matching (GM) Hungarian Matching (HM) Stable Marriage Matching (SMM) (b) Different missing ratios of tagged objects Cardinality of Interference Objects 0 1 2 3 4 5 Matching Ratio 0 0.2 0.4 0.6 0.8 1 Greedy Matching (GM) Hungarian Matching (HM) Stable Marriage Matching (SMM) (c) Different cardinalities of inter￾ference objects Average Distance between Interference Objects and Tagged Objects (cm) 50 100 150 200 Matching Ratio 0 0.2 0.4 0.6 0.8 1 Greedy Matching (GM) Hungarian Matching (HM) Stable Marriage Matching (SMM) (d) Different distances between interference objects and tagged objects Fig. 10. Performance Evaluation 6 MATCH THE MOBILE TAGGED HUMAN SUB￾JECTS VIA CONTINUOUS SCANNING 6.1 Motivation In most cases, the AR systems are designed towards a mobile scenario, e.g., multiple human subjects wearing RFID badges are continuously moving around. For this mobile situation, the rotate scanning-based solution for recognizing multiple stationary tagged objects is no longer suitable. Since the locations of the tagged human subjects are continuously changing, the scanning frequency of the rotate scanning-based solution cannot be high enough to locate the positions of the tags and human subjects in a real-time manner. Nevertheless, we observe that, when multiple tagged human subjects are continuously moving, their moving traces in the two-dimensional space can be distinguishable among each other. Hence, according to the depth information and the phase information extracted from multiple tagged human subjects, we are able to derive some metric to depict the moving traces for the tags and human subjects, respectively. In this way, by matching the moving traces of tags to the corresponding human subjects, we are able to match the mobile tagged human subjects. Therefore, to recognize multiple tagged human subjects in the mobile situation, in this section, we propose a continuous scanning￾based solution to pair the mobile tags with moving human subjects via trace matching. 6.2 Pair the Tags with Mobile Human Subjects via Trace Matching When deploying our system in front of multiple human subjects, where the human subjects wearing RFID badges are moving around, it is known that the state-of-art depth camera such as Kinect is able to extact the skeleton models from the human subjects. Based on the skeleton model, we can further extract the spinemid point [1] from the skeleton to represent the human subject, which is also very close to the place of RFID badge worn by the human subject, as shown in Fig.11(a). According to the two-dimensional coordinate of the spinemid point in the horizontal plane, we can figure out the moving traces of different human subjects from the depth camera, as shown in Fig.11(b). Moreover, suppose the reader/depth camera is deployed in the origin O, for any spinemid point P, we can use the angle profile to denote the angle between the vector OP and the X-axis OX, as shown in Fig.11(b). As aforementioned, using the RFID antenna pair, our system can estimate the Angle of Arrival (AoA) of the RFID tag in the horizontal plane. Then, we can similarly use the angle profile to denote the angle between the AoA direction of the tag and the X-axis. Recall that according to the phase values collected from the RFID antenna pair, there could be multiple solutions for the angle of arrival of the RFID tag. Hence, there could be multiple angle profiles corresponding to the specified tag. Therefore, while the tagged human subjects are moving from time to time, we can plot the angle profiles for both the human subjects and the tags over time. Fig.11(c) shows the corresponding angle profiles for the human subjects and RFID tags over time, where the Tag i is worn on the Body i. Note that for a specified tag, there are multiple solutions for its angle profile, we use the same color to label them. We can observe that the angle profile of the specified body has very close variation trend to one of the angle profiles of the corresponding RFID tag, as they share very similar moving traces in the horizontal plane. Therefore, in order to evaluate the correlation of the angle profile between the bodies and tags, we use the difference score to denote this correlation. Specifically, in a specified sliding window W with length L, for the lth snapshot (1 ≤ l ≤ L), suppose the angle profiles of the body Oi and the tag Tj are αi(l) and {α 0 j (l)}, respectively. Then, the difference score si,j between αi and α 0 j in W is as follows: si,j = min α0 j∈{α0 j } 1 L X L l=1 ￾ αi(l) − α 0 j (l)
2 . (6) Here we enumerate all feasible angle profiles α 0 j for the tag Tj to compare with the angle profile of αi for the body Oi , and obtain the minimum value as the difference score si,j . Fig.11(d) shows the difference scores in angle profiles between various pairs of tags and bodies, it is found that