IEEE TRANSACTIONS ON MOBILE COMPUTING,VOL.XX,NO.XX,2019 Therefore,in this paper,we propose a passive RFID- This paper presents the first study of using RFID to based 3D reconstruction approach,called RF-3DScan.As perform the 3D reconstruction on tagged packages.We shown in Fig.1,RF-3DScan aims at performing the 3D make three contributions.First,for the 3D reconstruction reconstruction on packaged objects attached with passive on packages,we attach a set of passive RFID tags onto RFID tags,including the package orientation and the pack- packages,and respectively handle issues of the package age stacking.The basic idea is that by attaching multiple orientation and the package stacking through angle profiles tags onto the surface of packages,we are capable of obtain- of tags.We build an angle-profile-based model to depict ing the orientation of each single package and the stacking the relationship between RF-signals of tags and the orien- status of multiple packages based on the backscattered RF- tation/stacking status of packages.Second,we propose a signals from these tags.RF-3DScan works as follows.We mobile scanning approach to perform the 3D reconstruction attach a set of passive RFID tags onto the package surface, of tagged packages via RFID.Generally,with the 1D mobile and leverage one mobile RFID antenna to move along the scanning,we can determine the package orientation and straight line to continuously scan the tagged packages.With coarse-grained package stacking;while with the 2D mobile the mobile scanning,we collect RF-signals from tags when scanning,we can determine the fine-grained package stack- the antenna is at different positions.Then,we extract phase ing.Third,We implement a prototype system of RF-3DScan differences of tags at different time points,and build angle to evaluate its performance.Our experiment results in real profiles for each tag to depict the geometry angle variation settings show that RF-3DScan can achieve about 92.5% between antenna-tag pairs during the moving process.Re- identification accuracy of the bottom face,and average error ferring to the angle profiles of tags,we can derive their about 4.08 of the rotation angle.The 1D scanning is much relative positions,and further determine the package place- easier to perform than the 2D scanning,while achieving the ment status,including the package orientation for each single comparable performance in terms of the package stacking. package and the package stacking for multiple packages. To realize the 3D reconstruction via RFID systems,there 2 RELATED WORK are three key challenges.The first challenge is that the 2.1 Computer Vision and Sensor-based Approach uncertain tag direction is easy to create dead zones of RFID communication.How to optimize the layout of multiple Computer-vision-based solutions mainly leverage the depth tags for avoiding dead zones and achieving the robust 3D camera to perform 3D reconstruction of multiple objects reconstruction is a key problem.To tackle this challenge,we [1,2].To avoid the blind angles in 3D reconstruction for deploy tags along three mutual orthogonal orientations,so specified objects,usually multiple depth cameras are de- that there are always some tags that can be collected by the ployed at different positions to perform multi-view recon- reader easily,which guarantees the high sampling rate and struction for their 3D models [2],or a moving depth camera reliable 3D reconstruction.The second challenge is that the is used to build the 3D models in a mobile approach [1].In existing work can only derive the 2D relative localization a word,these approaches suffer from the line-of-sight(LOS) of tag objects via once mobile scanning.How to locate the constraint in 3D perception,and they are vulnerable to the package and determine the package placement in the 3D limitation of the light intensity.Sensor-based solutions [3,4] space is still under-investigated.To tackle this challenge,we mainly attach the battery-powered sensors(such as inertial build an angle-profile model and combine this model with sensors or GPS modules)to the surface of the objects,and the priori knowledge of tag layout to sense the package continuously monitor the 3D placement of specified objects placement in the 3D space.Through once mobile linear scan- so as to track the orientation variation [3],or the stacking ning,we can extract angle profiles from phase differences to situation among multiple objects.However,they suffer from obtain position indicators and further determine the pack- the high hardware cost of sensors,as well as the limited age orientation with the known tag layout.By performing battery life of the sensor. one more scanning along the direction orthogonal to the previous one,we can combine the twice position indicators 2.2 RFID-based Approach to accurately estimate the package stacking.Although the Orientation tracking:By attaching RFID tags onto the spec- 2D scanning is a fine-grained solution for the package stack- ified object,it is possible to track the orientation variation ing,it requires the extra mobile scanning,so we propose of the object according to the variation of the corresponding a coarse-grained solution by the 1D scanning.With the RF-signals [5-10].Tagball [5]is proposed as a 3D human- known tag layout,we can localize the package via only computer interaction system,where multiple passive tags once scanning to determine the package stacking.The third are attached to a controlling ball,such that the motions challenge is that the RF-signal is likely to be distorted due of the ball from users can be detected from the phase to the tag orientation or the multi-path effect.How to select changes of multiple tags.Tagyro [6]attaches an array of effective data to ensure the performance is to be studied.To passive RFID tags as orientation sensors on the objects, tackle this challenge,we leverage phase differences to derive by transforming the runtime phase offsets between tags angle profiles,so as to eliminate the phase variation caused into the orientation angle.Compared with our RF-3DScan by the changing tag's orientation relative to the antenna system,these approaches track the orientation variation of during the mobile scanning.Furthermore,at the start or end the dynamically moving objects,whereas our approach aims of the scanning,as the antenna is relatively far from the tag, to determine the orientation of statically placed packages. the RF-signal is more seriously distorted,so we propose an Localization:RFID localization generally falls into two adaptive algorithm to automatically filter outliers and keep categories:absolute localization [11-17]and relative local- remaining data for the later estimation. ization [18-25].By attaching multiple tags and pinpointingIEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, 2019 2 Therefore, in this paper, we propose a passive RFID￾based 3D reconstruction approach, called RF-3DScan. As shown in Fig. 1, RF-3DScan aims at performing the 3D reconstruction on packaged objects attached with passive RFID tags, including the package orientation and the pack￾age stacking. The basic idea is that by attaching multiple tags onto the surface of packages, we are capable of obtain￾ing the orientation of each single package and the stacking status of multiple packages based on the backscattered RF￾signals from these tags. RF-3DScan works as follows. We attach a set of passive RFID tags onto the package surface, and leverage one mobile RFID antenna to move along the straight line to continuously scan the tagged packages. With the mobile scanning, we collect RF-signals from tags when the antenna is at different positions. Then, we extract phase differences of tags at different time points, and build angle profiles for each tag to depict the geometry angle variation between antenna-tag pairs during the moving process. Re￾ferring to the angle profiles of tags, we can derive their relative positions, and further determine the package place￾ment status, including the package orientation for each single package and the package stacking for multiple packages. To realize the 3D reconstruction via RFID systems, there are three key challenges. The first challenge is that the uncertain tag direction is easy to create dead zones of RFID communication. How to optimize the layout of multiple tags for avoiding dead zones and achieving the robust 3D reconstruction is a key problem. To tackle this challenge, we deploy tags along three mutual orthogonal orientations, so that there are always some tags that can be collected by the reader easily, which guarantees the high sampling rate and reliable 3D reconstruction. The second challenge is that the existing work can only derive the 2D relative localization of tag objects via once mobile scanning. How to locate the package and determine the package placement in the 3D space is still under-investigated. To tackle this challenge, we build an angle-profile model and combine this model with the priori knowledge of tag layout to sense the package placement in the 3D space. Through once mobile linear scan￾ning, we can extract angle profiles from phase differences to obtain position indicators and further determine the pack￾age orientation with the known tag layout. By performing one more scanning along the direction orthogonal to the previous one, we can combine the twice position indicators to accurately estimate the package stacking. Although the 2D scanning is a fine-grained solution for the package stack￾ing, it requires the extra mobile scanning, so we propose a coarse-grained solution by the 1D scanning. With the known tag layout, we can localize the package via only once scanning to determine the package stacking. The third challenge is that the RF-signal is likely to be distorted due to the tag orientation or the multi-path effect. How to select effective data to ensure the performance is to be studied. To tackle this challenge, we leverage phase differences to derive angle profiles, so as to eliminate the phase variation caused by the changing tag’s orientation relative to the antenna during the mobile scanning. Furthermore, at the start or end of the scanning, as the antenna is relatively far from the tag, the RF-signal is more seriously distorted, so we propose an adaptive algorithm to automatically filter outliers and keep remaining data for the later estimation. This paper presents the first study of using RFID to perform the 3D reconstruction on tagged packages. We make three contributions. First, for the 3D reconstruction on packages, we attach a set of passive RFID tags onto packages, and respectively handle issues of the package orientation and the package stacking through angle profiles of tags. We build an angle-profile-based model to depict the relationship between RF-signals of tags and the orien￾tation/stacking status of packages. Second, we propose a mobile scanning approach to perform the 3D reconstruction of tagged packages via RFID. Generally, with the 1D mobile scanning, we can determine the package orientation and coarse-grained package stacking; while with the 2D mobile scanning, we can determine the fine-grained package stack￾ing. Third, We implement a prototype system of RF-3DScan to evaluate its performance. Our experiment results in real settings show that RF-3DScan can achieve about 92.5% identification accuracy of the bottom face, and average error about 4.08◦ of the rotation angle. The 1D scanning is much easier to perform than the 2D scanning, while achieving the comparable performance in terms of the package stacking. 2 RELATED WORK 2.1 Computer Vision and Sensor-based Approach Computer-vision-based solutions mainly leverage the depth camera to perform 3D reconstruction of multiple objects [1, 2]. To avoid the blind angles in 3D reconstruction for specified objects, usually multiple depth cameras are de￾ployed at different positions to perform multi-view recon￾struction for their 3D models [2], or a moving depth camera is used to build the 3D models in a mobile approach [1]. In a word, these approaches suffer from the line-of-sight (LOS) constraint in 3D perception, and they are vulnerable to the limitation of the light intensity. Sensor-based solutions [3, 4] mainly attach the battery-powered sensors (such as inertial sensors or GPS modules) to the surface of the objects, and continuously monitor the 3D placement of specified objects, so as to track the orientation variation [3], or the stacking situation among multiple objects. However, they suffer from the high hardware cost of sensors, as well as the limited battery life of the sensor. 2.2 RFID-based Approach Orientation tracking: By attaching RFID tags onto the spec￾ified object, it is possible to track the orientation variation of the object according to the variation of the corresponding RF-signals [5–10]. Tagball [5] is proposed as a 3D human￾computer interaction system, where multiple passive tags are attached to a controlling ball, such that the motions of the ball from users can be detected from the phase changes of multiple tags. Tagyro [6] attaches an array of passive RFID tags as orientation sensors on the objects, by transforming the runtime phase offsets between tags into the orientation angle. Compared with our RF-3DScan system, these approaches track the orientation variation of the dynamically moving objects, whereas our approach aims to determine the orientation of statically placed packages. Localization: RFID localization generally falls into two categories: absolute localization [11–17] and relative local￾ization [18–25]. By attaching multiple tags and pinpointing