2146 IEEE/ACM TRANSACTIONS ON NETWORKING,VOL.26.NO.5.OCTOBER 2018 Synchronize Inertial Readings From Multiple Mobile Devices in Spatial Dimension Lei Xie,Member,IEEE,Qingliang Cai,Student Member,IEEE,Alex X.Liu,Senior Member,IEEE, Wei Wang,Member;IEEE,Yafeng Yin,Member,IEEE,and Sanglu Lu,Member,IEEE Abstract-In this paper,we investigate the problem of space synchronization,i.e,synchronizing inertial readings from multiple mobile devices in the spatial dimension,in other words,mul- tiple mobile devices are space synchronized to have the same 3-D coordinates except that each device is the origin of its corresponding coordinate.We propose a scheme called MObile Space Synchronization (MOSS)for devices with two sensors: an accelerometer and a gyroscope,which are available on most Local Coordinat (a) mobile devices.Accelerometer readings from multiple mobile 6) devices on a human subject are used to achieve space synchroniza- Fig.1.Synchronize inertial readings from multi-devices in spatial dimension tion when the human subject is moving forward,such as walking (a)Synchronize 3-D coordinates among devices in VR games.(b)Local and running.Gyroscope readings from multiple mobile devices coordinates vs synchronized coordinates. on a human subject are used to maintain space synchronization when the human subject stops moving forward,which means that multiple mobile devices are space synchronized to have the we can no longer obtain the consistent acceleration caused by same 3-D coordinates except that each device is the origin body moving forward.Experiment results show that our MOSS scheme can achieve an average angle deviation of 9.8 and an of its corresponding coordinate.A modern mobile device is average measurement similarity of 97%. often equipped with an Inertial Measurement Unit (IMU), Index Terms-Space synchronization,mobile device which typically includes an accelerometer and a gyroscope, and sometimes a magnetometer as well for advanced models. I.INTRODUCTION A.Motivation The readings from IMU sensors are based on a local coordinate determined by the orientation of its body frame (or its mother TOWADAYS,the mobile devices equipped with inertial board to be more precise).Fig.1(b)shows the two local sensors are widely used to perform motion sensing and coordinates of a smartphone and a smartwatch,respectively, human computer interaction [1]-[4].For example,in virtual in solid arrows.For the multiple mobile devices placed at dif- reality games,a moving human subject may be equipped with ferent body parts of a human subject,due to the different orien- multiple mobile devices on different body parts(such as heads, tation of their body frames,their local coordinates are mostly arms,and legs)to capture the human's movement,as shown likely different.Without space synchronization,the readings in Fig.1(a).In this case,it is very essential to fuse the from the IMU sensors of different mobile devices are difficult readings from multiple mobile devices based on synchronized to be correlated with each other.With space synchronization, coordinates,so that they can be combined together to recover these readings can be used jointly to better describe human better orientation of the users'body parts and recognize the movements.We call the synchronized coordinate of a device as human activities. its global coordinate.Fig.1(a)and (b)shows the synchronized In this paper,we investigate the problem of space synchro- coordinates of the devices in dashed arrows.Space synchro- nization,i.e.,synchronizing inertial readings from multiple nization allows us to capture the movement of different body mobile devices in the spatial dimension,in other words, parts in the aligned 3-D coordinates,which helps to build more Manuscript received October 29,2017:revised May 10,2018;accepted accurate 3-D models of human actions. July 17,2018:approved by IEEE/ACM TRANSACTIONS ON NETWORKING A straightforward solution to mobile device space syn- Editor K.Tang.Date of publication August 14,2018;date of current chronization is to use compasses or magnetometer sensors in version October 15,2018.This work was supported in part by the National Natural Science Foundation of China under Grant 61472185,Grant 61472184 mobile devices.Magnetometer readings allow each device to Grant 61321491,Grant 61502224,and Grant 61702257,in part by the obtain the same magnetic field direction of the earth.Together Jiangsu Natural Science Foundation under Grant BK20151390 and Grant with the same gravity direction of each device,all devices BK20170648.in part by the Fundamental Research Funds for the Central Universities under Grant 020214380035.in part by the National Science can therefore achieve space synchronization.This solution, Foundation under Grant CNS-1421407,in part by the Jiangsu Innovation and although simple,have two weaknesses.First,most state-of- Entrepreneurship (Shuangchuang)Program,and in part by the Collaborative the-art wearable devices (such as Apple Watch S1,Samsung Innovation Center of Novel Software Technology and Industrialization.(Cor responding authors:Alex X.Liu:Sanglu Lu.) Watch Gear 2,and MOTO Watch 360)are not equipped The authors are with the State Key Laboratory for Novel Software with magnetometer sensors;in comparison,accelerometer and Technology.Nanjing University,Nanjing 210023,China (e-mail:Ixie@nju. gyroscope sensors are available in most wearable devices.Sec- edu.cn:qingliangcai@dislab.nju.edu.cn:alexliu@nju.edu.cn:ww@nju.edu.cn: yafeng@nju.edu.cn:sanglu@nju.edu.cn). ond,magnetometer readings are often notoriously inaccurate Digital Object Identifier 10.1109/TNET.2018.2859246 in indoor environments 5,due to the magnetic interferences 1063-66922018 IEEE.Personal use is permitted,but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.2146 IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 26, NO. 5, OCTOBER 2018 Synchronize Inertial Readings From Multiple Mobile Devices in Spatial Dimension Lei Xie , Member, IEEE, Qingliang Cai, Student Member, IEEE, Alex X. Liu, Senior Member, IEEE, Wei Wang, Member, IEEE, Yafeng Yin, Member, IEEE, and Sanglu Lu, Member, IEEE Abstract— In this paper, we investigate the problem of space synchronization, i.e., synchronizing inertial readings from multiple mobile devices in the spatial dimension, in other words, mul￾tiple mobile devices are space synchronized to have the same 3-D coordinates except that each device is the origin of its corresponding coordinate. We propose a scheme called MObile Space Synchronization (MOSS) for devices with two sensors: an accelerometer and a gyroscope, which are available on most mobile devices. Accelerometer readings from multiple mobile devices on a human subject are used to achieve space synchroniza￾tion when the human subject is moving forward, such as walking and running. Gyroscope readings from multiple mobile devices on a human subject are used to maintain space synchronization when the human subject stops moving forward, which means that we can no longer obtain the consistent acceleration caused by body moving forward. Experiment results show that our MOSS scheme can achieve an average angle deviation of 9.8◦ and an average measurement similarity of 97%. Index Terms— Space synchronization, mobile device. I. INTRODUCTION A. Motivation NOWADAYS, the mobile devices equipped with inertial sensors are widely used to perform motion sensing and human computer interaction [1]–[4]. For example, in virtual reality games, a moving human subject may be equipped with multiple mobile devices on different body parts (such as heads, arms, and legs) to capture the human’s movement, as shown in Fig. 1(a) . In this case, it is very essential to fuse the readings from multiple mobile devices based on synchronized coordinates, so that they can be combined together to recover better orientation of the users’ body parts and recognize the human activities. In this paper, we investigate the problem of space synchro￾nization, i.e., synchronizing inertial readings from multiple mobile devices in the spatial dimension, in other words, Manuscript received October 29, 2017; revised May 10, 2018; accepted July 17, 2018; approved by IEEE/ACM TRANSACTIONS ON NETWORKING Editor K. Tang. Date of publication August 14, 2018; date of current version October 15, 2018. This work was supported in part by the National Natural Science Foundation of China under Grant 61472185, Grant 61472184, Grant 61321491, Grant 61502224, and Grant 61702257, in part by the Jiangsu Natural Science Foundation under Grant BK20151390 and Grant BK20170648, in part by the Fundamental Research Funds for the Central Universities under Grant 020214380035, in part by the National Science Foundation under Grant CNS-1421407, in part by the Jiangsu Innovation and Entrepreneurship (Shuangchuang) Program, and in part by the Collaborative Innovation Center of Novel Software Technology and Industrialization. (Cor￾responding authors: Alex X. Liu; Sanglu Lu.) The authors are with the State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210023, China (e-mail: lxie@nju. edu.cn; qingliangcai@dislab.nju.edu.cn; alexliu@nju.edu.cn; ww@nju.edu.cn; yafeng@nju.edu.cn; sanglu@nju.edu.cn). Digital Object Identifier 10.1109/TNET.2018.2859246 Fig. 1. Synchronize inertial readings from multi-devices in spatial dimension. (a) Synchronize 3-D coordinates among devices in VR games. (b) Local coordinates vs synchronized coordinates. multiple mobile devices are space synchronized to have the same 3-D coordinates except that each device is the origin of its corresponding coordinate. A modern mobile device is often equipped with an Inertial Measurement Unit (IMU), which typically includes an accelerometer and a gyroscope, and sometimes a magnetometer as well for advanced models. The readings from IMU sensors are based on a local coordinate determined by the orientation of its body frame (or its mother board to be more precise). Fig. 1(b) shows the two local coordinates of a smartphone and a smartwatch, respectively, in solid arrows. For the multiple mobile devices placed at dif￾ferent body parts of a human subject, due to the different orien￾tation of their body frames, their local coordinates are mostly likely different. Without space synchronization, the readings from the IMU sensors of different mobile devices are difficult to be correlated with each other. With space synchronization, these readings can be used jointly to better describe human movements. We call the synchronized coordinate of a device as its global coordinate. Fig. 1(a) and (b) shows the synchronized coordinates of the devices in dashed arrows. Space synchro￾nization allows us to capture the movement of different body parts in the aligned 3-D coordinates, which helps to build more accurate 3-D models of human actions. A straightforward solution to mobile device space syn￾chronization is to use compasses or magnetometer sensors in mobile devices. Magnetometer readings allow each device to obtain the same magnetic field direction of the earth. Together with the same gravity direction of each device, all devices can therefore achieve space synchronization. This solution, although simple, have two weaknesses. First, most state-of￾the-art wearable devices (such as Apple Watch S1, Samsung Watch Gear 2, and MOTO Watch 360) are not equipped with magnetometer sensors; in comparison, accelerometer and gyroscope sensors are available in most wearable devices. Sec￾ond, magnetometer readings are often notoriously inaccurate in indoor environments [5], due to the magnetic interferences 1063-6692 © 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information