Gait Recognition Using WiFi Signals Wei Wang Alex X.Liu Muhammad Shahzad Nanjing University Michigan State University North Carolina State University Nanjing University GAN STATE NANJING UNIVERSITY 1902 Founded 1855
1/96 Gait Recognition Using WiFi Signals Wei Wang Alex X. Liu Muhammad Shahzad Nanjing University Michigan State University North Carolina State University Nanjing University
Gait Based Human Authentication ldentification Human Authentication Problem Statement:Are you Trump or not? Why need gait based human authentication? Multi-factor human authentication Human ldentification Problem Statement:Which one of the following people are you:Trump or Hillary? Why need gait based human identification? ·Customization 3/96
3/96 Gait Based Human Authentication & Identification § Human Authentication – Problem Statement: Are you Trump or not? – Why need gait based human authentication? • Multi-factor human authentication § Human Identification – Problem Statement: Which one of the following people are you: Trump or Hillary? – Why need gait based human identification? • Customization
Gait Based Human Authentication ldentification Why possible? Individually unique gait patterns -What have been done? Video camera ·Cannot work in dark ·Privacy concern Wearble sensors ·Inconvenient Floor sensors ·Expensive What do we propose? WiFi signals 4/96
4/96 Gait Based Human Authentication & Identification § Why possible? – Individually unique gait patterns § What have been done? – Video camera • Cannot work in dark • Privacy concern – Wearble sensors • Inconvenient – Floor sensors • Expensive § What do we propose? – WiFi signals
WiFiU:Gait Based Human Authentication Recognition Using WiFi Signals Using commercial off the shelf devices Woks in dark Privacy friendly NetGEAR JR6100 Lenovo Thinkpad X200 5/96
5/96 WiFiU: Gait Based Human Authentication & Recognition Using WiFi Signals § Using commercial off the shelf devices § Woks in dark § Privacy friendly NetGEAR JR6100 Lenovo Thinkpad X200
Key Insight Human body reflects wireless signals WiFi signal WiFi router WiFi signal reflection Laptop Signal analysis Chanel State Information (CSI)has huge amount of useful information about environmental changes 2,500 samples per second Complex valued samples with 8 bit accuracy On 30 subcarriers for each antenna pair ■ So,commercial WiFi devices can act as Doppler Radars to measure human activities 6/96
6/96 Key Insight § Human body reflects wireless signals § Chanel State Information (CSI) has huge amount of useful information about environmental changes – 2,500 samples per second – Complex valued samples with 8 bit accuracy – On 30 subcarriers for each antenna pair § So, commercial WiFi devices can act as Doppler Radars to measure human activities Signal analysis WiFi router Laptop WiFi signal reflection WiFi signal
System Architecture Data collection and Feature extraction Identification pre-processing Gait cycle time 1.Model generation 1.CSI data collection Torso speed Training data Footstep size collection Leg speed Spectrogram signature 2.PCA denoising 2.Prediction SVM based prediction 3.Spectrogram generation 204000100 40600 0406082 10 7196
7/96 System Architecture Data collection and pre-processing Feature extraction - Gait cycle time - Torso speed - Footstep size - Leg speed - Spectrogram signature 1. CSI data collection 2. PCA denoising 3. Spectrogram generation Identification 1. Model generation 2. Prediction Training data collection SVM based prediction
How to deal with noisy signals? Signals from commercial WiFi n 75 devices are very noisy 60 Time (Seconds) 12 12.5 Original 75 Traditional filter based denoising approaches not work "wwwpylo- 11.5 12 12.5 Time (seconds) Low-pass filter Analysis based approach in our MobiCom 15 paper 11.5 12 12.5 Time (seconds) PCA 8/96
8/96 How to deal with noisy signals? § Signals from commercial WiFi devices are very noisy § Traditional filter based denoising approaches not work § We proposed a Principal Component Analysis based approach in our MobiCom 15 paper 11 11.5 12 12.5 60 65 70 75 CSI Time (seconds) Original 11 11.5 12 12.5 65 70 75 CSI Time (seconds) Low-pass filter 11 11.5 12 12.5 −10 −5 0 5 10 Time (seconds) CSI PCA
PCA Based Noise Reduction Changes in CSI streams caused by human movementare correlated Theoretically proved Experimentally validated NxM×30 CSI streams 30 subcarriers …↑↑11 Phase changes by 2 Noises present in all streams 150 100 50 2.5 2.552.62.652.72.752.82.852.92.95 Time(seconds) 9/96
9/96 PCA Based Noise Reduction § Changes in CSI streams caused by human movement are correlated – Theoretically proved – Experimentally validated
How to extract gait information? Challenge:signal reflections of different body parts are mixed together in the wave form. Insight:different body parts move at different speeds Signal reflections of different parts have differentfrequencies. CSI fluctuations of differentfrequencies are separable in the frequency domain. How:convert waveforms into time-frequency domain - Use Short-Time Fourier Transform(STFT)to convert each slice of waveforms to a spectrogram. Spectrogram 3 dimensions:time,frequency,and FFT amplitude Window size:tradeoff between frequency and time resolutions Larger:higher frequency resolution,low time resolution Smaller:low frequency solution,high time resolution Our choice:FFT size=1024 samples,window size=32 samples Frequency resolution=2.44Hz,time resolution=12.8ms 10/96
10/96 How to extract gait information? § Challenge: signal reflections of different body parts are mixed together in the wave form. § Insight: different body parts move at different speeds – Signal reflections of different parts have different frequencies. – CSI fluctuations of different frequencies are separable in the frequency domain. § How: convert waveforms into time-frequency domain – Use Short-Time Fourier Transform (STFT) to convert each slice of waveforms to a spectrogram. – Spectrogram 3 dimensions: time, frequency, and FFT amplitude – Window size: tradeoff between frequency and time resolutions • Larger: higher frequency resolution, low time resolution • Smaller: low frequency solution, high time resolution – Our choice: FFT size=1024 samples, window size=32 samples • Frequency resolution=2.44Hz, time resolution=12.8ms
Spectrogram Enhancement We apply spectrogram enhancement techniques to further reduce noise. Insight:CSI spectrograms give similar information as the expensive Doppler radars Leg 80 Cycle time reflection Torso 60 reflection 40 20 2 3 4 5 Time(seconds) 11/96
11/96 Spectrogram Enhancement § We apply spectrogram enhancement techniques to further reduce noise. § Insight: CSI spectrograms give similar information as the expensive Doppler radars