AirContour 1:5 3 TECHNICAL CHALLENGES AND DEFINITIONS IN IN-AIR GESTURE RECOGNITION 3.1 Intra-class Variability in Sensor Data As shown in Fig.2,even when the user performs the same type of gestures (e.g.,writes 't'), the sensor data can be quite different,due to the variation of writing speeds(Fig.2(a)), gesture sizes (Fig.2(b)),writing directions (Fig.2(c)),etc.Directly using the extracted features from sensor data may fail to recognize in-air gestures accurately. To handle the intra-class variability of in-air gestures,e.g.,the variation of speed,amplitude, and orientation of gestures,we present the contour-based gesture model,which utilizes contours to correlate sensor data with human gestures.The 'contour'is represented with a sequence of coordinate points over time.Additionally,to avoid the differences caused by facing directions,we transform the sensor data from device coordinate system to human coordinate system shown in Fig.5(a),i.e.,we analyze the 3D contours in human coordinate system.In this paper,we take the instance of writing characters in the air to illustrate the contour-based gesture model.The characters are referred to the alphabet,i.e.,a'z',and we use the term“character'”,“letter'”interchangeably throughout the paper.It is worth mentioning that in-air writing letters can be different from printed letters,due to joined-up writing.In particular,we remove the point of 'i'and j',and use't'to represent the letter I' for simplification. 2420 25 nong eqene 5npg8net120 4 5nptnms9ne65we120 hamal 40的 Below 一 —6—上的 一泡将日 28 一dm6 -od- (0mgn 的mng0coI网e,0 (a)Different speeds (b)Different sizes (c)Different directions Fig.2.Linear acceleration of writing the same character 't' 3.2 Difference between 2D Contours and 3D Contours Usually,people get used to recognizing and reading handwritten characters in a 2D plane, e.g.,on a piece of paper.Therefore,we can map a 2D gesture contour with a 2D character for recognition.However,based on extensive observations and experimental study,we find that 3D contour recognition is quite different from 2D contour recognition.In fact,recognizing 3D contours as 2D characters can be very challenging,due to the contour distortion caused by viewing angles,contour difference caused by writing directions,and contour distribution across different planes,as described below. 3.2.1 Viewing Angles.There is a uniform viewing angle for a 2D character contour,while multiple viewing angles for a 3D character contour.In a predefined plane-coordinate system, ACM Trans.Sensor Netw.,Vol.1,No.1,Article 1.Publication date:January 2019.AirContour 1:5 3 TECHNICAL CHALLENGES AND DEFINITIONS IN IN-AIR GESTURE RECOGNITION 3.1 Intra-class Variability in Sensor Data As shown in Fig. 2, even when the user performs the same type of gestures (e.g., writes ‘t’), the sensor data can be quite different, due to the variation of writing speeds (Fig. 2(a)), gesture sizes (Fig. 2(b)), writing directions (Fig. 2(c)), etc. Directly using the extracted features from sensor data may fail to recognize in-air gestures accurately. To handle the intra-class variability of in-air gestures, e.g., the variation of speed, amplitude, and orientation of gestures, we present the contour-based gesture model, which utilizes contours to correlate sensor data with human gestures. The ‘contour’ is represented with a sequence of coordinate points over time. Additionally, to avoid the differences caused by facing directions, we transform the sensor data from device coordinate system to human coordinate system shown in Fig. 5(a), i.e., we analyze the 3D contours in human coordinate system. In this paper, we take the instance of writing characters in the air to illustrate the contour-based gesture model. The characters are referred to the alphabet, i.e., ‘a’ ∼ ‘z’, and we use the term “character”, “letter” interchangeably throughout the paper. It is worth mentioning that in-air writing letters can be different from printed letters, due to joined-up writing. In particular, we remove the point of ‘i’ and ‘j’, and use ‘ι’ to represent the letter ‘l’ for simplification. User13, D2-Front, D4-Right User6, Yinyafeng20, D3; 60-D5, 40-D2 YinyafengFast, D1-Fast， D5-Normal, Gezefan D2-Slow (a) Different speeds User13, D2-Front, D4-Right User6, Yinyafeng20, D3; 60-D5, 40-D2 YinyafengFast, D1-Fast， D5-Normal, Gezefan D2-Slow (b) Different sizes User13, D2-Front, D4-Right, D5-Down User6, Yinyafeng20, D3; 60-D4, 40-D2 YinyafengFast, D1-Fast
D5-Normal, Gezefan D2-Slow Below (c) Different directions Fig. 2. Linear acceleration of writing the same character ‘t’ 3.2 Difference between 2D Contours and 3D Contours Usually, people get used to recognizing and reading handwritten characters in a 2D plane, e.g., on a piece of paper. Therefore, we can map a 2D gesture contour with a 2D character for recognition. However, based on extensive observations and experimental study, we find that 3D contour recognition is quite different from 2D contour recognition. In fact, recognizing 3D contours as 2D characters can be very challenging, due to the contour distortion caused by viewing angles, contour difference caused by writing directions, and contour distribution across different planes, as described below. 3.2.1 Viewing Angles. There is a uniform viewing angle for a 2D character contour, while multiple viewing angles for a 3D character contour. In a predefined plane-coordinate system, ACM Trans. Sensor Netw., Vol. 1, No. 1, Article 1. Publication date: January 2019