IEEE TRANSACTIONS ON MOBILE COMPUTING,VOL.XX,NO.XX,2019 RF-3DScan:RFID-based 3D Reconstruction on Tagged Packages Yanling Bu,Student Member,IEEE,Lei Xie,Member,IEEE,Yinyin Gong,Jia Liu,Member,IEEE, Bingbing He,Jiannong Cao,Fellow,IEEE,Baoliu Ye,Member,IEEE,and Sanglu Lu,Member,IEEE Abstract-Currently,the logistic industry has introduced 3D reconstruction to monitor the package placement in the warehouse. Previous 3D reconstruction solutions mainly utilize computer vision or sensor-based methods,which are restricted to the line-of-sight or the battery life.Therefore,we propose a passive RFID-based solution,called RF-3DScan,to perform 3D reconstruction on tagged packages,including the package orientation and the package stacking.The basic idea is that a moving antenna can obtain RF-signals from the tags attached on packages with the 1D linear mobile scanning.Through extracting phase differences to build angle profiles for each tag,RF-3DScan derives their relative positions,further determines the package orientation and the coarse-grained package stacking.By simply performing the 2D scanning,RF-3DScan can provide the fine-grained package stacking determination.We implement a prototype system of RF-3DScan and evaluate its performance in real settings.Our experiment results show that RF-3DScan can achieve about 92.5%identification accuracy of the bottom face,and average error about 4.08 of the rotation angle. For the package stacking,1D scanning can achieve the comparable performance in comparison with 2D scanning. Index Terms-RFID,3D reconstruction,package orientation,package stacking. INTRODUCTION URRENTLY,in the logistic industry,traditional applica- tions like the warehouse management and the logistic transportation,are emerging with brand new requirements. For instance,considering the safety and space utilization Package <right Tag issues,packages are required to be placed based on specified Stacking left Tagged Package regulations.Specifically,with regard to a single package,if it contains orientation-sensitive objects,i.e.,chemical reagents, precision instruments,it is protected from getting rollover or upside down.While with regard to multiple packages,to Antenna ensure the package safety during the transportation process, Package unaligned Orientation rollover Linear Mobile Scanning they are required to be precisely arranged in order,i.e., Linear Track heavy packages are placed on the bottom and light ones are on the top.To satisfy the above requirements,3D reconstruc- Fig.1.3D reconstruction on tagged packages via linear mobile scanning tion has been introduced to handle these issues for monitor- ing the package placement.Generally,3D reconstruction is Previous 3D reconstruction solutions mainly utilize com- a process of capturing the shape and appearance of a single puter vision or sensor-based methods.Computer vision- or multiple real objects.Fig.1 shows the principle of 3D based solutions capture the appearance of objects with cam- reconstruction on packaged objects:1)Package orientation eras,and then build 3D profiles of objects [1,2].They can of a single object,which means determining the relative reconstruct objects in a vivid way.However,they suffer from orientation of each object,i.e.,pinpointing the bottom/top the line-of-sight constraint,easily leading to blind angles face and estimating angles of vertical sides of the object in when capturing images.Sensor-based approaches attach the specified coordinate system.2)Package stacking of mul- inertial sensors onto items so as to monitor the orientation tiple objects,which means determining the relative stacking variation of targets [3,4].However,the main disadvantages situation of multiple packages,i.e.,figuring out the up- of them are the high hardware cost and the limited battery down,front-back or left-right relationships among objects. life of sensors.Thankfully,the promising RFID technology has brought great chances for the 3D reconstruction on packaged objects in the logistic industry.Nowadays,passive Yanling Bu,Lei Xie,Yinyin Gong,Jia Liu,Bingbing He,Baoliu Ye, and Sanglu Lu are with the State Key Laboratory for Novel Software RFID tags have been broadly used to label packages with Technology,Nanjing University,China. detailed logistics information.Compared to the above two E-mail: yanling@smail.nju.edu.cn, Ixie@nju.edu.cn, yy- approaches,the passive RFID tag is battery-free and cheap. gong@dislab.nju.edu.cn,jialiu@nju.edu.cn, hebb@dislab.nju.edu.cn, yebl@nju.edu.cn,sanglu@nju.edu.cn Also,RFID technology uses the backscatter communication, Jiannong Cao is with the Department of Computing,The Hong Kong so it has no requirement of the line of sight or the light Polytechnic University,Hong Kong,China. condition.Most importantly,in order to scan and identify E-mail:csjcao@comp.polyu.edu.hk. packages,RFID systems have been already widely deployed Lei Xie and Baoliu Ye are the co-corresponding authors. in the sites for most logistic applications in our daily life.IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. XX, NO. XX, 2019 1 RF-3DScan: RFID-based 3D Reconstruction on Tagged Packages Yanling Bu, Student Member, IEEE, Lei Xie, Member, IEEE, Yinyin Gong, Jia Liu, Member, IEEE, Bingbing He, Jiannong Cao, Fellow, IEEE, Baoliu Ye, Member, IEEE, and Sanglu Lu, Member, IEEE Abstract—Currently, the logistic industry has introduced 3D reconstruction to monitor the package placement in the warehouse. Previous 3D reconstruction solutions mainly utilize computer vision or sensor-based methods, which are restricted to the line-of-sight or the battery life. Therefore, we propose a passive RFID-based solution, called RF-3DScan, to perform 3D reconstruction on tagged packages, including the package orientation and the package stacking. The basic idea is that a moving antenna can obtain RF-signals from the tags attached on packages with the 1D linear mobile scanning. Through extracting phase differences to build angle profiles for each tag, RF-3DScan derives their relative positions, further determines the package orientation and the coarse-grained package stacking. By simply performing the 2D scanning, RF-3DScan can provide the fine-grained package stacking determination. We implement a prototype system of RF-3DScan and evaluate its performance in real settings. Our experiment results show that RF-3DScan can achieve about 92.5% identification accuracy of the bottom face, and average error about 4.08◦ of the rotation angle. For the package stacking, 1D scanning can achieve the comparable performance in comparison with 2D scanning. Index Terms—RFID, 3D reconstruction, package orientation, package stacking. ✦ 1 INTRODUCTION C URRENTLY, in the logistic industry, traditional applica￾tions like the warehouse management and the logistic transportation, are emerging with brand new requirements. For instance, considering the safety and space utilization issues, packages are required to be placed based on specified regulations. Specifically, with regard to a single package, if it contains orientation-sensitive objects, i.e., chemical reagents, precision instruments, it is protected from getting rollover or upside down. While with regard to multiple packages, to ensure the package safety during the transportation process, they are required to be precisely arranged in order, i.e., heavy packages are placed on the bottom and light ones are on the top. To satisfy the above requirements, 3D reconstruc￾tion has been introduced to handle these issues for monitor￾ing the package placement. Generally, 3D reconstruction is a process of capturing the shape and appearance of a single or multiple real objects. Fig. 1 shows the principle of 3D reconstruction on packaged objects: 1) Package orientation of a single object, which means determining the relative orientation of each object, i.e., pinpointing the bottom/top face and estimating angles of vertical sides of the object in the specified coordinate system. 2) Package stacking of mul￾tiple objects, which means determining the relative stacking situation of multiple packages, i.e., figuring out the up￾down, front-back or left-right relationships among objects. • Yanling Bu, Lei Xie, Yinyin Gong, Jia Liu, Bingbing He, Baoliu Ye, and Sanglu Lu are with the State Key Laboratory for Novel Software Technology, Nanjing University, China. E-mail: yanling@smail.nju.edu.cn, lxie@nju.edu.cn, yy￾gong@dislab.nju.edu.cn, jialiu@nju.edu.cn, hebb@dislab.nju.edu.cn, yebl@nju.edu.cn, sanglu@nju.edu.cn. • Jiannong Cao is with the Department of Computing, The Hong Kong Polytechnic University, Hong Kong, China. E-mail: csjcao@comp.polyu.edu.hk. • Lei Xie and Baoliu Ye are the co-corresponding authors. Tag Package Orientation Package Stacking Linear Mobile Scanning Antenna Linear Track Tagged Package above below left right rollover unaligned Fig. 1. 3D reconstruction on tagged packages via linear mobile scanning Previous 3D reconstruction solutions mainly utilize com￾puter vision or sensor-based methods. Computer vision￾based solutions capture the appearance of objects with cam￾eras, and then build 3D profiles of objects [1, 2]. They can reconstruct objects in a vivid way. However, they suffer from the line-of-sight constraint, easily leading to blind angles when capturing images. Sensor-based approaches attach inertial sensors onto items so as to monitor the orientation variation of targets [3, 4]. However, the main disadvantages of them are the high hardware cost and the limited battery life of sensors. Thankfully, the promising RFID technology has brought great chances for the 3D reconstruction on packaged objects in the logistic industry. Nowadays, passive RFID tags have been broadly used to label packages with detailed logistics information. Compared to the above two approaches, the passive RFID tag is battery-free and cheap. Also, RFID technology uses the backscatter communication, so it has no requirement of the line of sight or the light condition. Most importantly, in order to scan and identify packages, RFID systems have been already widely deployed in the sites for most logistic applications in our daily life