AirContour:Building Contour-based Model for in-Air Writing Gesture Recognition YAFENG YIN,State Key Laboratory for Novel Software Technology,Nanjing University,China LEI XIE (corresponding author),State Key Laboratory for Novel Software Technology,Nanjing University,China TAO GU,RMIT University,Australia YIJIA LU,State Key Laboratory for Novel Software Technology,Nanjing University,China SANGLU LU,State Key Laboratory for Novel Software Technology,Nanjing University,China Recognizing in-air hand gestures will benefit a wide range of applications such as sign language recognition,remote control with hand gestures,and "writing"in the air as a new way of text input.This paper presents AirContour,which focuses on in-air writing gesture recognition with a wrist-worn device.We propose a novel contour-based gesture model which converts human gestures to contours in 3D space,and then recognize the contours as characters.Different from 2D contours, the 3D contours may have the problems such as contour distortion caused by different viewing angles,contour difference caused by different writing directions,and the contour distribution across different planes.To address the above problem,we introduce Principal Component Analysis(PCA) to detect the principal/writing plane in 3D space,and then tune the projected 2D contour in the principal plane through reversing,rotating and normalizing operations,to make the 2D contour in right orientation and normalized size under a uniform view.After that,we propose both an online approach AC-Vec and an offline approach AC-CNN for character recognition.The experimental results show that AC-Vec achieves an accuracy of 91.6%and AC-CNN achieves an accuracy of 94.3%for gesture/character recognition,both outperform the existing approaches. CCS Concepts:.Human-centered computing-Ubiquitous and mobile computing de- sign and evaluation methods;Empirical studies in ubiquitous and mobile computing. Additional Key Words and Phrases:AirContour,in-air writing,contour-based gesture model, principal component analysis(PCA),gesture recognition ACM Reference Format: Yafeng Yin,Lei Xie (corresponding author),Tao Gu,Yijia Lu,and Sanglu Lu.2019.AirContour: Building Contour-based Model for in-Air Writing Gesture Recognition.ACM Trans.Sensor Netw. 1,1,Article 1 (January 2019),26 pages.https://doi.org/10.1145/3343855 Authors'addresses:Yafeng Yin,State Key Laboratory for Novel Software Technology,Nanjing University, Nanjing,China,yafeng@nju.edu.cn;Lei Xie(corresponding author),State Key Laboratory for Novel Software Technology,Nanjing University,Nanjing,China,Ixie@nju.edu.cn;Tao Gu,RMIT University,Melbourne, Australia,tao.gu@rmit.edu.au;Yijia Lu,State Key Laboratory for Novel Software Technology,Nanjing University,Nanjing,China,lyj@smail.nju.edu.cn;Sanglu Lu,State Key Laboratory for Novel Software Technology,Nanjing University,Nanjing,China,sanglu@nju.edu.cn. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page.Copyrights for components of this work owned by others than ACM must be honored.Abstracting with credit is permitted.To copy otherwise, or republish,to post on servers or to redistribute to lists,requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. C)2019 Association for Computing Machinery. 1550-4859/2019/1-ART1$15.00 https://doi.org/10.1145/3343855 ACM Trans.Sensor Netw.,Vol.1,No.1,Article 1.Publication date:January 2019.1 AirContour: Building Contour-based Model for in-Air Writing Gesture Recognition YAFENG YIN, State Key Laboratory for Novel Software Technology, Nanjing University, China LEI XIE (corresponding author), State Key Laboratory for Novel Software Technology, Nanjing University, China TAO GU, RMIT University, Australia YIJIA LU, State Key Laboratory for Novel Software Technology, Nanjing University, China SANGLU LU, State Key Laboratory for Novel Software Technology, Nanjing University, China Recognizing in-air hand gestures will benefit a wide range of applications such as sign language recognition, remote control with hand gestures, and “writing” in the air as a new way of text input. This paper presents AirContour, which focuses on in-air writing gesture recognition with a wrist-worn device. We propose a novel contour-based gesture model which converts human gestures to contours in 3D space, and then recognize the contours as characters. Different from 2D contours, the 3D contours may have the problems such as contour distortion caused by different viewing angles, contour difference caused by different writing directions, and the contour distribution across different planes. To address the above problem, we introduce Principal Component Analysis (PCA) to detect the principal/writing plane in 3D space, and then tune the projected 2D contour in the principal plane through reversing, rotating and normalizing operations, to make the 2D contour in right orientation and normalized size under a uniform view. After that, we propose both an online approach AC-Vec and an offline approach AC-CNN for character recognition. The experimental results show that AC-Vec achieves an accuracy of 91.6% and AC-CNN achieves an accuracy of 94.3% for gesture/character recognition, both outperform the existing approaches. CCS Concepts: • Human-centered computing → Ubiquitous and mobile computing de￾sign and evaluation methods; Empirical studies in ubiquitous and mobile computing. Additional Key Words and Phrases: AirContour, in-air writing, contour-based gesture model, principal component analysis (PCA), gesture recognition ACM Reference Format: Yafeng Yin, Lei Xie (corresponding author), Tao Gu, Yijia Lu, and Sanglu Lu. 2019. AirContour: Building Contour-based Model for in-Air Writing Gesture Recognition. ACM Trans. Sensor Netw. 1, 1, Article 1 (January 2019), 26 pages. https://doi.org/10.1145/3343855 Authors’ addresses: Yafeng Yin, State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China, yafeng@nju.edu.cn; Lei Xie (corresponding author), State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China, lxie@nju.edu.cn; Tao Gu, RMIT University, Melbourne, Australia, tao.gu@rmit.edu.au; Yijia Lu, State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China, lyj@smail.nju.edu.cn; Sanglu Lu, State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China, sanglu@nju.edu.cn. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. © 2019 Association for Computing Machinery. 1550-4859/2019/1-ART1 $15.00 https://doi.org/10.1145/3343855 ACM Trans. Sensor Netw., Vol. 1, No. 1, Article 1. Publication date: January 2019