100 sound in different materials and the propagation distance. Table 2 lists the theoretical propagation delays and ampli- tude for the six different paths between the speaker and the two microphones on the example shown in Figure 2.Given the high speed of sound for the structure-borne sound,the two structure sound paths(Path 1 and Path 2)have similar delays even if their path lengths are slightly different.Since With sampling rate increasing the acoustic attenuation coefficient of metal is close to air Without sampling rate increasing [26],the amplitude of structure sound path is close to the 200 -100 0 100 I(normalized) amplitude of the LOS air path.The LOS air paths(Path 3 and Path 4)have longer delays than the structure paths due to Figure 6:Path coefficient at different sampling rate the slower speed of sound in the air.The reflection air paths reflection air path(Path 5 and Path 6),respectively.We call (Path 5 and Path 6)arrive after the LOS air paths due to the this process as path delay calibration,which is performed longer path length.The amplitudes of reflection air paths are once when the system starts transmitting and recording the smaller than other two types of paths due to the attenuation sound signal.The path delay calibration is based on the first along the reflection and propagation process. ten data segments(213 ms)of IR estimation.We use an 1- nearest neighbor algorithm to confirm the path delays based 5.3 Sound Propagation Separation on the results of the ten segments. Typical impulse response estimations of the two micro- Note that the calibration time is 14.95 ms for one segment phones are shown in Figure 3.Although the theoretical delay (21.3 ms).Thus,we can perform calibration for each seg- difference between Path 1 and Path 3 is 0.13 ms(6 samples). ment in real-time.To save the computational cost,we only the time resolution of the interpolated ZC sequence is not calibrate the LOS path and structure-borne path delays for enough to separate Path 1 and Path 3 on Mic 1.Thus,the the first ten segments(213 ms).The path delay calibration is first peak in the IR estimation of the Mic 1 represents the only performed once after the system initialization because combination of Path 1 and Path 3.Due to the longer distance holding styles hardly change delays of the structure-borne from the speaker to Mic 2,the theoretical delay difference path and the LOS path.For the reflection path delay,we between Path 2 and Path 4 is 0.34 ms(17 samples).As a result, adaptively estimate it as shown in Section 6.2 so that our the Mic 2 has two peaks with similar amplitude,which cor- system will be robust to different holding styles. respond to the structure path(the first peak)and the LOS air path(the second peak),respectively.By locating the peaks 5.4 Path Coefficient Measurement of the IR estimation of the two microphones,we are able to After finding the delay of each propagation path,we mea- separate different propagation paths. sure the path coefficient of each path.For a path i with We use the IR estimation of both microphones to identify a delay of ni samples in the IR estimation,the path coef- different propagation paths.On commercial mobile devices, ficient is the complex value of h[ni]on the correspond- the starting point of the auto-correlation function is random ing microphone.The path coefficient indicates how the due to the randomness in the hardware/system delay of amplitude and phase of the given path change with time. sound playback and recording.The peaks corresponding to Both the amplitude and the phase of the path coefficient the structure propagation may appear at random positions are important for later movement measurement and touch every time when the system restarts.Therefore,we need detection algorithms. to first locate the structure paths in the IR estimations.Our One key challenge in path coefficient measurement is that key observation is that the two microphones are strictly cross-correlations are measured at low sampling rates.The synchronized so that their structure paths should appear basic cross-correlation algorithm presented in Section 5.1 at the same position in the IR estimations.Based on this produces one IR estimation per frame of 1,024 samples.This observation,we first locate the highest peak of Mic 1,which converts to a sampling rate of 48,000/1,024 =46.875 Hz.The corresponds to the combination of both Path 1 and Path 3. low sampling rate may lead to ambiguity in fast movements Then,we can locate the peaks of Path 2 and Path 4 in the IR where the path coefficient changes quickly.Figure 6 shows estimation of Mic 2 as the position of Path 2 should be aligned the path coefficient of a finger movement with a speed of 10 with Path 1/Path 3.Since we focus on the movement around cm/s.We observe that there are only 2~3 samples in each the mobile devices,the reflection air path is 5~15 samples phase cycle of 2m.As a phase difference of m can be caused (3.5 10.7 cm)away from LOS path for both microphones. either by a phase increases ofπor a phase decreased byπ， In this way,we get the delays of(i)combination of Path the direction of phase changing cannot be determined by 1 and Path 3,(ii)Path 2,(iii)Path 4,and (iv)the range of such low rate measurements.sound in different materials and the propagation distance. Table 2 lists the theoretical propagation delays and ampli￾tude for the six different paths between the speaker and the two microphones on the example shown in Figure 2. Given the high speed of sound for the structure-borne sound, the two structure sound paths (Path 1 and Path 2) have similar delays even if their path lengths are slightly different. Since the acoustic attenuation coefficient of metal is close to air [26], the amplitude of structure sound path is close to the amplitude of the LOS air path. The LOS air paths (Path 3 and Path 4) have longer delays than the structure paths due to the slower speed of sound in the air. The reflection air paths (Path 5 and Path 6) arrive after the LOS air paths due to the longer path length. The amplitudes of reflection air paths are smaller than other two types of paths due to the attenuation along the reflection and propagation process. 5.3 Sound Propagation Separation Typical impulse response estimations of the two micro￾phones are shown in Figure 3. Although the theoretical delay difference between Path 1 and Path 3 is 0.13 ms (6 samples), the time resolution of the interpolated ZC sequence is not enough to separate Path 1 and Path 3 on Mic 1. Thus, the first peak in the IR estimation of the Mic 1 represents the combination of Path 1 and Path 3. Due to the longer distance from the speaker to Mic 2, the theoretical delay difference between Path 2 and Path 4 is 0.34ms (17 samples). As a result, the Mic 2 has two peaks with similar amplitude, which cor￾respond to the structure path (the first peak) and the LOS air path (the second peak), respectively. By locating the peaks of the IR estimation of the two microphones, we are able to separate different propagation paths. We use the IR estimation of both microphones to identify different propagation paths. On commercial mobile devices, the starting point of the auto-correlation function is random due to the randomness in the hardware/system delay of sound playback and recording. The peaks corresponding to the structure propagation may appear at random positions every time when the system restarts. Therefore, we need to first locate the structure paths in the IR estimations. Our key observation is that the two microphones are strictly synchronized so that their structure paths should appear at the same position in the IR estimations. Based on this observation, we first locate the highest peak of Mic 1, which corresponds to the combination of both Path 1 and Path 3. Then, we can locate the peaks of Path 2 and Path 4 in the IR estimation of Mic 2 as the position of Path 2 should be aligned with Path 1/Path 3. Since we focus on the movement around the mobile devices, the reflection air path is 5 ∼ 15 samples (3.5 ∼ 10.7 cm) away from LOS path for both microphones. In this way, we get the delays of (i) combination of Path 1 and Path 3, (ii) Path 2, (iii) Path 4, and (iv) the range of -200 -100 0 100 I (normalized) -200 -100 0 100 Q (normalized) With sampling rate increasing Without sampling rate increasing Figure 6: Path coefficient at different sampling rate reflection air path (Path 5 and Path 6), respectively. We call this process as path delay calibration, which is performed once when the system starts transmitting and recording the sound signal. The path delay calibration is based on the first ten data segments (213 ms) of IR estimation. We use an 1- nearest neighbor algorithm to confirm the path delays based on the results of the ten segments. Note that the calibration time is 14.95 ms for one segment (21.3 ms). Thus, we can perform calibration for each seg￾ment in real-time. To save the computational cost, we only calibrate the LOS path and structure-borne path delays for the first ten segments (213 ms). The path delay calibration is only performed once after the system initialization because holding styles hardly change delays of the structure-borne path and the LOS path. For the reflection path delay, we adaptively estimate it as shown in Section 6.2 so that our system will be robust to different holding styles. 5.4 Path Coefficient Measurement After finding the delay of each propagation path, we mea￾sure the path coefficient of each path. For a path i with a delay of ni samples in the IR estimation, the path coef￾ficient is the complex value of ˆh[ni] on the correspond￾ing microphone. The path coefficient indicates how the amplitude and phase of the given path change with time. Both the amplitude and the phase of the path coefficient are important for later movement measurement and touch detection algorithms. One key challenge in path coefficient measurement is that cross-correlations are measured at low sampling rates. The basic cross-correlation algorithm presented in Section 5.1 produces one IR estimation per frame of 1,024 samples. This converts to a sampling rate of 48, 000/1, 024 = 46.875 Hz. The low sampling rate may lead to ambiguity in fast movements where the path coefficient changes quickly. Figure 6 shows the path coefficient of a finger movement with a speed of 10 cm/s. We observe that there are only 2∼3 samples in each phase cycle of 2π. As a phase difference of π can be caused either by a phase increases of π or a phase decreased by π, the direction of phase changing cannot be determined by such low rate measurements