XIE et al:SYNCHRONIZE INERTIAL READINGS FROM MULTIPLE MOBILE DEVICES IN SPATIAL DIMENSION 2151 record the accelerometer and gyroscope measurements after receiving this beacon.Let this time be to. (2)Local Measurement with Gyroscope Tracking:For each time t within a time window W.each device D;extracts the gravitational acceleration g(t)and the linear acceleration fi(t) from the acceleration measurements in its local coordinate Meanwhile,each device D;tracks the rotation speed wi(t) in each axis from the gyroscope,and computes the rotation matrix Af.t from time toto t.After time window W,device Di sends the following to the server:the rotation matrix Fig.7.System architecture. Atthe gravity gi(t),and the linear acceleration fi(t)for all t∈W. where (3)PCA Analysis:After receiving the responses from all 0 -w2(t) wy(t) devices,the server performs PCA to extract the magnitude and 2(t)= wz(t) 0 -w:(t) (6 estimates the direction of the consistent acceleration f(t)in -wy(t) wz(t) 0 the local coordinate of each device.With the gravity direction, We further use Taylor expansion of the exponential term to it constructs a rotation matrix C for each device at time to. approximate the value of exp(() and sends C to each device. (4)Local Rotation Update:Each device Di locally updates As the angular velocity signals obtained from the gyro- the rotation matrix C using the matrix A.from gyroscope scopes are"integrated"to track and update the rotation matrix, tracking,i.e..C=CAIt then transforms the accelerom- the errors in the gyroscope signals propagate into the calcu- eter and gyroscope measurements from the local coordinate to lated orientation leading to large cumulative tracking errors. global coordinate. The tracking errors in the rotation matrix At.t+t are amplified in each axis through the exponential function in Eq.(5).The VI.CONSISTENT ACCELERATION EXTRACTION larger the angular velocities and linear accelerations in the forward moving direction are,the more the tracking errors As the accelerometer measurements from multi-devices are e(t).ey(t),and e=(t)in that direction.Thus,the rotation highly correlated,we perform Principal Component Analy- sis (PCA)to extract the consistent acceleration during the matrix At.t+t derived from gyroscope tracking is prone to process of moving forward.This consists of three steps: large accumulation of errors after long periods.It is essential preprocessing,principal component extraction,and direction to leverage other measurements (such as the accelerometer estimation. measurements)for further calibration. V.SYSTEM OVERVIEW A.Preprocessing The system architecture is shown in Fig.7.MOSS con- Suppose the acceleration measurements fi(t)for each device sists of two components:a Consistent Direction Estimator Di can be obtained as (xi(t),(t),zi(t))from the x,y,and for achieving space synchronization,and a Gyroscope-based z-axes in its local coordinate Li(t)at time t.Then,we can Orientation Tracker for maintaining space synchronization. use the rotation matrix(A)T to transform the acceleration Consistent Direction Estimator uses principal component measurements from the current local coordinate Li(t)at time analysis to extract the magnitude of consistent acceleration t to the reference local coordinate Li(to)at time to,i.e.. from multiple devices,and further estimate the direction of zi(to) x:(t) consistent acceleration.Combined with the gravity direction h(to） (Ai)T (t) (7) extracted from the acceleration measurements,it generates a 2:(to)】 2(t) rotation matrix for space synchronization.Gyroscope-based Orientation Tracker continuously estimates the orientation Thus,we obtain the acceleration measurements in the refer- variation of the body frame from the gyroscope measurements. ence local coordinate,i.e.,the local coordinate at time to, Leveraging the stability of the gravitational accelerations in the which is therefore consistent over time. global coordinate,we calibrate the rotation matrix using the Further,as we focus on extracting the forwarding accelera- Minimum Mean Square Error(MMSE)estimator. tions on the horizontal plane,to avoid the interferences such To perform space synchronization among multiple devices, as the up-and-down accelerations from the vertical direction MOSS uses a server to communicate with these devices and during human motions,we project the linear acceleration fi(t) process the corresponding measurements.We can set one of onto the horizontal plane in the reference local coordinate the mobile devices such as a smart phone as the server,as long system,as shown in Fig.8. as it can communicate with the other devices via wireless connections,and it has enough computing power for signal B.Principal Component Extraction processing.MOSS consists of the following four steps The acceleration measurements of different axes on multiple (1)Beacon Broadcasting:The server first broadcasts a devices are highly correlated,as shown in Eq.(1),because synchronization beacon to all devices.All devices start to of the consistent acceleration fe(t).This can be observedXIE et al.: SYNCHRONIZE INERTIAL READINGS FROM MULTIPLE MOBILE DEVICES IN SPATIAL DIMENSION 2151 Fig. 7. System architecture. where Ω(t) = ⎡ ⎣ 0 −ωz(t) ωy(t) ωz(t) 0 −ωx(t) −ωy(t) ωx(t) 0 ⎤ ⎦ (6) We further use Taylor expansion of the exponential term to approximate the value of exp( t+δt t Ω(t)dt). As the angular velocity signals obtained from the gyro￾scopes are “integrated” to track and update the rotation matrix, the errors in the gyroscope signals propagate into the calcu￾lated orientation leading to large cumulative tracking errors. The tracking errors in the rotation matrix At,t+δt are amplified in each axis through the exponential function in Eq.(5). The larger the angular velocities and linear accelerations in the forward moving direction are, the more the tracking errors ex(t), ey(t), and ez(t) in that direction. Thus, the rotation matrix A t,t+δt derived from gyroscope tracking is prone to large accumulation of errors after long periods. It is essential to leverage other measurements (such as the accelerometer measurements) for further calibration. V. SYSTEM OVERVIEW The system architecture is shown in Fig. 7. MOSS con￾sists of two components: a Consistent Direction Estimator for achieving space synchronization, and a Gyroscope-based Orientation Tracker for maintaining space synchronization. Consistent Direction Estimator uses principal component analysis to extract the magnitude of consistent acceleration from multiple devices, and further estimate the direction of consistent acceleration. Combined with the gravity direction extracted from the acceleration measurements, it generates a rotation matrix for space synchronization. Gyroscope-based Orientation Tracker continuously estimates the orientation variation of the body frame from the gyroscope measurements. Leveraging the stability of the gravitational accelerations in the global coordinate, we calibrate the rotation matrix using the Minimum Mean Square Error (MMSE) estimator. To perform space synchronization among multiple devices, MOSS uses a server to communicate with these devices and process the corresponding measurements. We can set one of the mobile devices such as a smart phone as the server, as long as it can communicate with the other devices via wireless connections, and it has enough computing power for signal processing. MOSS consists of the following four steps. (1) Beacon Broadcasting: The server first broadcasts a synchronization beacon to all devices. All devices start to record the accelerometer and gyroscope measurements after receiving this beacon. Let this time be t0. (2) Local Measurement with Gyroscope Tracking: For each time t within a time window W, each device Di extracts the gravitational acceleration gi(t) and the linear acceleration fi(t) from the acceleration measurements in its local coordinate. Meanwhile, each device Di tracks the rotation speed ωi(t) in each axis from the gyroscope, and computes the rotation matrix Ai t0,t from time t0 to t. After time window W, device Di sends the following to the server: the rotation matrix Ai t0,t, the gravity gi(t), and the linear acceleration fi(t) for all t ∈ W. (3) PCA Analysis: After receiving the responses from all devices, the server performs PCA to extract the magnitude and estimates the direction of the consistent acceleration fc(t) in the local coordinate of each device. With the gravity direction, it constructs a rotation matrix Ci t0 for each device at time t0, and sends Ci t0 to each device. (4) Local Rotation Update: Each device Di locally updates the rotation matrix Ci t using the matrix Ai t0,t from gyroscope tracking, i.e., Ci t = Ci t0Ai t0,t. It then transforms the accelerom￾eter and gyroscope measurements from the local coordinate to global coordinate. VI. CONSISTENT ACCELERATION EXTRACTION As the accelerometer measurements from multi-devices are highly correlated, we perform Principal Component Analy￾sis (PCA) to extract the consistent acceleration during the process of moving forward. This consists of three steps: preprocessing, principal component extraction, and direction estimation. A. Preprocessing Suppose the acceleration measurements fi(t) for each device Di can be obtained as xi(t), yi(t), zi(t) from the x, y, and z-axes in its local coordinate Li(t) at time t. Then, we can use the rotation matrix (Ai t0,t)T to transform the acceleration measurements from the current local coordinate Li(t) at time t to the reference local coordinate Li(t0) at time t0, i.e., ⎡ ⎣ xi(t0) yi(t0) zi(t0) ⎤ ⎦ = (Ai t0,t) T ⎡ ⎣ xi(t) yi(t) zi(t) ⎤ ⎦ (7) Thus, we obtain the acceleration measurements in the refer￾ence local coordinate, i.e., the local coordinate at time t0, which is therefore consistent over time. Further, as we focus on extracting the forwarding accelera￾tions on the horizontal plane, to avoid the interferences such as the up-and-down accelerations from the vertical direction during human motions, we project the linear acceleration fi(t) onto the horizontal plane in the reference local coordinate system, as shown in Fig. 8. B. Principal Component Extraction The acceleration measurements of different axes on multiple devices are highly correlated, as shown in Eq.(1), because of the consistent acceleration fc(t). This can be observed