sound in different materials and the propagation distance. reflection air path(Path 5 and Path 6),respectively.We call Table 2 lists the theoretical propagation delays and ampli- this process as path delay calibration,which is performed tude for the six different paths between the speaker and the once when the system starts transmitting and recording the two microphones on the example shown in Figure 2.Given sound signal.The path delay calibration is based on the first the high speed of sound for the structure-borne sound,the ten data segments(213 ms)of IR estimation.We use an 1- two structure sound paths(Path 1 and Path 2)have similar nearest neighbor algorithm to confirm the path delays based delays even if their path lengths are slightly different.Since on the results of the ten segments. the acoustic attenuation coefficient of metal is close to air Note that the calibration time is 14.95 ms for one segment [26],the amplitude of structure sound path is close to the (21.3 ms).Thus,we can perform calibration for each seg- amplitude of the LOS air path.The LOS air paths(Path 3 and ment in real-time.To save the computational cost,we only Path 4)have longer delays than the structure paths due to calibrate the LOS path and structure-borne path delays for the slower speed of sound in the air.The reflection air paths the first ten segments(213 ms).The path delay calibration is (Path 5 and Path 6)arrive after the LOS air paths due to the only performed once after the system initialization because longer path length.The amplitudes of reflection air paths are holding styles hardly change delays of the structure-borne smaller than other two types of paths due to the attenuation path and the LOS path.For the reflection path delay,we along the reflection and propagation process. adaptively estimate it as shown in Section 6.2 so that our system will be robust to different holding styles. 5.3 Sound Propagation Separation Typical impulse response estimations of the two micro- 5.4 Path Coefficient Measurement phones are shown in Figure 3.Although the theoretical delay After finding the delay of each propagation path,we mea- difference between Path 1 and Path 3 is 0.13 ms(6 samples). sure the path coefficient of each path.For a path i with the time resolution of the interpolated ZC sequence is not a delay of ni samples in the IR estimation,the path coef- enough to separate Path 1 and Path 3 on Mic 1.Thus.the ficient is the complex value of h[ni]on the correspond- first peak in the IR estimation of the Mic 1 represents the ing microphone.The path coefficient indicates how the combination of Path 1 and Path 3.Due to the longer distance amplitude and phase of the given path change with time. from the speaker to Mic 2,the theoretical delay difference Both the amplitude and the phase of the path coefficient between Path 2 and Path 4 is 0.34 ms(17 samples).As a result, are important for later movement measurement and touch the Mic 2 has two peaks with similar amplitude,which cor- detection algorithms. respond to the structure path (the first peak)and the LOS air One key challenge in path coefficient measurement is that path(the second peak),respectively.By locating the peaks cross-correlations are measured at low sampling rates.The of the IR estimation of the two microphones,we are able to basic cross-correlation algorithm presented in Section 5.1 separate different propagation paths. produces one IR estimation per frame of 1,024 samples.This We use the IR estimation of both microphones to identify converts to a sampling rate of 48,000/1,024 =46.875 Hz.The different propagation paths.On commercial mobile devices, low sampling rate may lead to ambiguity in fast movements the starting point of the auto-correlation function is random where the path coefficient changes quickly.Figure 6 shows due to the randomness in the hardware/system delay of the path coefficient of a finger movement with a speed of 10 sound playback and recording.The peaks corresponding to cm/s.We observe that there are only 2~3 samples in each the structure propagation may appear at random positions phase cycle of 2.As a phase difference of can be caused every time when the system restarts.Therefore,we need either by a phase increases ofπor a phase decreased byπ， to first locate the structure paths in the IR estimations.Our the direction of phase changing cannot be determined by key observation is that the two microphones are strictly such low rate measurements. synchronized so that their structure paths should appear We use the property of the circular cross-correlation to at the same position in the IR estimations.Based on this upsample the path coefficient measurements.For a given observation,we first locate the highest peak of Mic 1,which delay of n samples,the IR estimation at time t is given by corresponds to the combination of both Path 1 and Path 3. the circular cross-correlation of the received signal and the Then,we can locate the peaks of Path 2 and Path 4 in the IR transmitted sequence: estimation of Mic 2 as the position of Path 2 should be aligned with Path 1/Path 3.Since we focus on the movement around the mobile devices,the reflection air path is 5~15 samples hr[n] ZCR[t +I]x ZC[(l-n)mod Nzc](4) (3.5 10.7 cm)away from LOS path for both microphones. In this way,we get the delays of(i)combination of Path This is equivalent to take the summation of Nzc point of 1 and Path 3,(ii)Path 2,(iii)Path 4,and(iv)the range of the received signal multiplied by a conjugated ZC sequencesound in different materials and the propagation distance. Table 2 lists the theoretical propagation delays and ampli￾tude for the six different paths between the speaker and the two microphones on the example shown in Figure 2. Given the high speed of sound for the structure-borne sound, the two structure sound paths (Path 1 and Path 2) have similar delays even if their path lengths are slightly different. Since the acoustic attenuation coefficient of metal is close to air [26], the amplitude of structure sound path is close to the amplitude of the LOS air path. The LOS air paths (Path 3 and Path 4) have longer delays than the structure paths due to the slower speed of sound in the air. The reflection air paths (Path 5 and Path 6) arrive after the LOS air paths due to the longer path length. The amplitudes of reflection air paths are smaller than other two types of paths due to the attenuation along the reflection and propagation process. 5.3 Sound Propagation Separation Typical impulse response estimations of the two micro￾phones are shown in Figure 3. Although the theoretical delay difference between Path 1 and Path 3 is 0.13 ms (6 samples), the time resolution of the interpolated ZC sequence is not enough to separate Path 1 and Path 3 on Mic 1. Thus, the first peak in the IR estimation of the Mic 1 represents the combination of Path 1 and Path 3. Due to the longer distance from the speaker to Mic 2, the theoretical delay difference between Path 2 and Path 4 is 0.34ms (17 samples). As a result, the Mic 2 has two peaks with similar amplitude, which cor￾respond to the structure path (the first peak) and the LOS air path (the second peak), respectively. By locating the peaks of the IR estimation of the two microphones, we are able to separate different propagation paths. We use the IR estimation of both microphones to identify different propagation paths. On commercial mobile devices, the starting point of the auto-correlation function is random due to the randomness in the hardware/system delay of sound playback and recording. The peaks corresponding to the structure propagation may appear at random positions every time when the system restarts. Therefore, we need to first locate the structure paths in the IR estimations. Our key observation is that the two microphones are strictly synchronized so that their structure paths should appear at the same position in the IR estimations. Based on this observation, we first locate the highest peak of Mic 1, which corresponds to the combination of both Path 1 and Path 3. Then, we can locate the peaks of Path 2 and Path 4 in the IR estimation of Mic 2 as the position of Path 2 should be aligned with Path 1/Path 3. Since we focus on the movement around the mobile devices, the reflection air path is 5 ∼ 15 samples (3.5 ∼ 10.7 cm) away from LOS path for both microphones. In this way, we get the delays of (i) combination of Path 1 and Path 3, (ii) Path 2, (iii) Path 4, and (iv) the range of reflection air path (Path 5 and Path 6), respectively. We call this process as path delay calibration, which is performed once when the system starts transmitting and recording the sound signal. The path delay calibration is based on the first ten data segments (213 ms) of IR estimation. We use an 1- nearest neighbor algorithm to confirm the path delays based on the results of the ten segments. Note that the calibration time is 14.95 ms for one segment (21.3 ms). Thus, we can perform calibration for each seg￾ment in real-time. To save the computational cost, we only calibrate the LOS path and structure-borne path delays for the first ten segments (213 ms). The path delay calibration is only performed once after the system initialization because holding styles hardly change delays of the structure-borne path and the LOS path. For the reflection path delay, we adaptively estimate it as shown in Section 6.2 so that our system will be robust to different holding styles. 5.4 Path Coefficient Measurement After finding the delay of each propagation path, we mea￾sure the path coefficient of each path. For a path i with a delay of ni samples in the IR estimation, the path coef￾ficient is the complex value of ˆh[ni] on the correspond￾ing microphone. The path coefficient indicates how the amplitude and phase of the given path change with time. Both the amplitude and the phase of the path coefficient are important for later movement measurement and touch detection algorithms. One key challenge in path coefficient measurement is that cross-correlations are measured at low sampling rates. The basic cross-correlation algorithm presented in Section 5.1 produces one IR estimation per frame of 1,024 samples. This converts to a sampling rate of 48, 000/1, 024 = 46.875 Hz. The low sampling rate may lead to ambiguity in fast movements where the path coefficient changes quickly. Figure 6 shows the path coefficient of a finger movement with a speed of 10 cm/s. We observe that there are only 2∼3 samples in each phase cycle of 2π. As a phase difference of π can be caused either by a phase increases of π or a phase decreased by π, the direction of phase changing cannot be determined by such low rate measurements. We use the property of the circular cross-correlation to upsample the path coefficient measurements. For a given delay of n samples, the IR estimation at time t is given by the circular cross-correlation of the received signal and the transmitted sequence: ˆht[n] = N XZC −1 l=0 ZCR[t + l] × ZC∗ T [(l − n) mod NZC] (4) This is equivalent to take the summation of NZC point of the received signal multiplied by a conjugated ZC sequence