3-Dimensional Reconstruction on Tagged Packages via RFID Systems Yanling Bu,Lei Xie,Jia Liu,Bingbing He,Yinyin Gong and Sanglu Lu State Key Laboratory for Novel Software Technology,Nanjing University,China yanlingbu@foxmail.com,Ixie@nju.edu.cn.jialiu.cs@gmail.com, [hebb,yygong}@dislab.nju.edu.cn,sanglu@nju.edu.cn Abstract-Nowadays,3D reconstruction has been introduced in monitoring the package placement in logistic industry-related applications.Existing 3D Package reconstruction methods are mainly orientation Package based on computer vision or sensor-based approaches,which are stacking limited by the line-of-sight or battery life constraint.In this paper, we propose RF-3DScan to perform 3D reconstruction on tagged packages via passive RFID,by attaching multiple reference tags -Antenna onto the surface of the packages.The basic idea is that by moving the antenna along straight lines within a constrained 2-dimensional space,the antenna obtains the RF-signals of the Moving direction reference tags attached on the packages.By extracting the phase differences to build the angle profile for each tag,RF-3DScan can compare the angle profiles of the different reference tags and Fig.1.3D reconstruction on tagged packages via mobile scanning derive their relative positions,then further determine the package Existing 3D reconstruction methods are mainly based on orientation and stacking for 3D reconstruction.We implement computer vision or sensor-based approaches.Computer vision- RF-3DScan and evaluate its performance in real settings.The experiment results show that the average identification accuracy based approaches leverage the cameras to capture the appear- of the bottom face is about 92.5%,and the average estimation ance and build the 3D profiles of objects [1.21.They are able error of the rotation angle is about 4.08. to reconstruct the shape of objects in a vivid approach.The main disadvantage is the line-of-sight constraint in capturing I.INTRODUCTION images,which leads to blind angles in 3D reconstruction Nowadays,the traditional logistic industry-related appli- with only one fixed camera.Sensor-based approaches usually cations,such as warehouse management and logistic trans- leverage inertial sensors attached to the items to detect the portation,are emerging with brand new requirements.For orientation variation of the specified items [3,4].However, example,during the process of warehouse management or they suffer from high hardware cost of the sensors,as well logistic transportation,the packages are usually required to be as the limited battery life for the sensors.Fortunately,the placed according to some specified regulations.In particular,rising use of RFID technology in the logistic industry has in regard to a single package,if it contains orientation-sensitive brought brand new opportunities to 3D reconstruction on goods,such as chemical reagents or precision instruments, packaged objects.In current logistic industry,RFID tags have then it is prohibited from being rollover or upside down; been widely used to label the packages with exact logistics in regard to multiple packages,they are also required to be information.In comparison to the above two approaches,the precisely arranged in some specified order,e.g.,heavy objects passive RFID tag is battery-free and very cheap,and the are placed on the bottom,whereas light objects are placed backscatter-based communication from RFID is not limited on the top,to ensure safety in the transportation.To deal by the line of sight requirement.Most importantly,for most with the above requirements,the technology of 3-dimensional logistic applications,the RFID systems are already deployed (3D)reconstruction has been introduced to tackle these issues in the sites to scan and identify the tagged packages. in monitoring the package placement.3D reconstruction is a Therefore,in this paper,we propose RF-3DScan,which process of capturing the shape and appearance of a single or aims to perform 3D reconstruction on packaged objects via multiple real objects.In principle,there are two key aspects the RFID systems (in Fig.1).Our idea is based on the to realize 3D reconstruction for packaged objects:1)Package observation that by attaching multiple tags onto the surface orientation for a single object,it refers to determining the of the packages,we are able to derive the 3D orientation of relative orientation for each package,i.e..figuring out the each single package and the 3D stacking situation of multiple bottom/top face,as well as the angles of the other vertical sides packages according to the backscattered RF-signals from these for the specified object in the specified coordinate system.2) reference tags.Our approach of RF-3DScan is as follows:We Package stacking for multiple objects,it refers to determining attach a set of reference tags on the surface of the packages, the relative stacking situation for multiple packages,i.e.,then we utilize a single RFID antenna to continuously scan performing the relative localization of multiple objects. the tagged packages,while the antenna is moving along3-Dimensional Reconstruction on Tagged Packages via RFID Systems Yanling Bu, Lei Xie, Jia Liu, Bingbing He, Yinyin Gong and Sanglu Lu State Key Laboratory for Novel Software Technology, Nanjing University, China yanlingbu@foxmail.com, lxie@nju.edu.cn, jialiu.cs@gmail.com, {hebb, yygong}@dislab.nju.edu.cn, sanglu@nju.edu.cn Abstract—Nowadays, 3D reconstruction has been introduced in monitoring the package placement in logistic industry-related applications. Existing 3D reconstruction methods are mainly based on computer vision or sensor-based approaches, which are limited by the line-of-sight or battery life constraint. In this paper, we propose RF-3DScan to perform 3D reconstruction on tagged packages via passive RFID, by attaching multiple reference tags onto the surface of the packages. The basic idea is that by moving the antenna along straight lines within a constrained 2-dimensional space, the antenna obtains the RF-signals of the reference tags attached on the packages. By extracting the phase differences to build the angle profile for each tag, RF-3DScan can compare the angle profiles of the different reference tags and derive their relative positions, then further determine the package orientation and stacking for 3D reconstruction. We implement RF-3DScan and evaluate its performance in real settings. The experiment results show that the average identification accuracy of the bottom face is about 92.5%, and the average estimation error of the rotation angle is about 4.08◦. I. INTRODUCTION Nowadays, the traditional logistic industry-related appli￾cations, such as warehouse management and logistic trans￾portation, are emerging with brand new requirements. For example, during the process of warehouse management or logistic transportation, the packages are usually required to be placed according to some specified regulations. In particular, in regard to a single package, if it contains orientation-sensitive goods, such as chemical reagents or precision instruments, then it is prohibited from being rollover or upside down; in regard to multiple packages, they are also required to be precisely arranged in some specified order, e.g., heavy objects are placed on the bottom, whereas light objects are placed on the top, to ensure safety in the transportation. To deal with the above requirements, the technology of 3-dimensional (3D) reconstruction has been introduced to tackle these issues in monitoring the package placement. 3D reconstruction is a process of capturing the shape and appearance of a single or multiple real objects. In principle, there are two key aspects to realize 3D reconstruction for packaged objects: 1) Package orientation for a single object, it refers to determining the relative orientation for each package, i.e., figuring out the bottom/top face, as well as the angles of the other vertical sides for the specified object in the specified coordinate system. 2) Package stacking for multiple objects, it refers to determining the relative stacking situation for multiple packages, i.e., performing the relative localization of multiple objects. 7DJ $QWHQQD 3DFNDJH RULHQWDWLRQ 3DFNDJH VWDFNLQJ Fig. 1. 3D reconstruction on tagged packages via mobile scanning Existing 3D reconstruction methods are mainly based on computer vision or sensor-based approaches. Computer vision￾based approaches leverage the cameras to capture the appear￾ance and build the 3D profiles of objects [1, 2]. They are able to reconstruct the shape of objects in a vivid approach. The main disadvantage is the line-of-sight constraint in capturing images, which leads to blind angles in 3D reconstruction with only one fixed camera. Sensor-based approaches usually leverage inertial sensors attached to the items to detect the orientation variation of the specified items [3, 4]. However, they suffer from high hardware cost of the sensors, as well as the limited battery life for the sensors. Fortunately, the rising use of RFID technology in the logistic industry has brought brand new opportunities to 3D reconstruction on packaged objects. In current logistic industry, RFID tags have been widely used to label the packages with exact logistics information. In comparison to the above two approaches, the passive RFID tag is battery-free and very cheap, and the backscatter-based communication from RFID is not limited by the line of sight requirement. Most importantly, for most logistic applications, the RFID systems are already deployed in the sites to scan and identify the tagged packages. Therefore, in this paper, we propose RF-3DScan, which aims to perform 3D reconstruction on packaged objects via the RFID systems (in Fig. 1). Our idea is based on the observation that by attaching multiple tags onto the surface of the packages, we are able to derive the 3D orientation of each single package and the 3D stacking situation of multiple packages according to the backscattered RF-signals from these reference tags. Our approach of RF-3DScan is as follows: We attach a set of reference tags on the surface of the packages, then we utilize a single RFID antenna to continuously scan the tagged packages, while the antenna is moving along