AirContour:Building Contour-based Model for In-Air Writing Gesture Recognition YAFENG YIN and LEI XIE,State Key Laboratory for Novel Software Technology,Nanjing University, China TAO GU,RMIT University,Australia YIJIA LU and SANGLU LU,State Key Laboratory for Novel Software Technology,Nanjing University, China Recognizing in-air hand gestures will benefit a wide range of applications such as sign-language recogni- tion,remote control with hand gestures,and "writing"in the air as a new way of text input.This article presents AirContour,which focuses on in-air writing gesture recognition with a wrist-worn device.We pro- pose a novel contour-based gesture model that converts human gestures to contours in 3D space and then recognizes the contours as characters.Different from 2D contours,the 3D contours may have the problems such as contour distortion caused by different viewing angles,contour difference caused by different writing directions,and the contour distribution across different planes.To address the above problem,we introduce Principal Component Analysis(PCA)to detect the principal/writing plane in 3D space,and then tune the projected 2D contour in the principal plane through reversing,rotating,and normalizing operations,to make the 2D contour in right orientation and normalized size under a uniform view.After that,we propose both an online approach,AC-Vec,and an offline approach,AC-CNN,for character recognition.The experimen- tal results show that AC-Vec achieves an accuracy of 91.6%and AC-CNN achieves an accuracy of 94.3%for gesture/character recognition,both outperforming the existing approaches. CCS Concepts:Human-centered computing-Ubiquitous and mobile computing design and eval- uation methods;Empirical studies in ubiquitous and mobile computing: Additional Key Words and Phrases:AirContour,in-air writing,contour-based gesture model,principal com- ponent analysis(PCA),gesture recognition ACM Reference format: Yafeng Yin,Lei Xie,Tao Gu,Yijia Lu,and Sanglu Lu.2019.AirContour:Building Contour-based Model for In-Air Writing Gesture Recognition.ACM Trans.Sen.Netw.15,4,Article 44(October 2019),25 pages. https:/doi.org/10.1145/3343855 This work is supported by National Natural Science Foundation of China under Grant Nos.61802169,61872174,61832008. 61321491:JiangSu Natural Science Foundation under Grant No.BK20180325;the Fundamental Research Funds for the Cen- tral Universities under Grant No.020214380049;Australian Research Council(ARC)Discovery Project Grants DP190101888 and DP180103932.This work is partially supported by Collaborative Innovation Center of Novel Software Technology and Industrialization Authors'addresses:Y.Yin,L.Xie (corresponding author).Y.Lu,and S.Lu,State Key Laboratory for Novel Software Technology,Nanjing University,Nanjing.210023,China;emails:(yafeng.Ixie@nju.edu.cn,lyj@smailnju.edu.cn,sanglu@ nju.edu cn;T.Gu,School of Computer Science and Information Technology,RMIT University.Melbourne VIC 3000,Aus- tralia;email:tao.gu@rmit.edu.au. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page.Copyrights for components of this work owned by others than ACM must be honored Abstracting with credit is permitted.To copy otherwise,or republish,to post on servers or to redistribute to lists.requires prior specific permission and/or a fee.Request permissions from permissions@acm.org. 2019 Association for Computing Machinery. 1550-4859/2019/10-ART44$15.00 https:/∥doi.org/10.1145/3343855 ACM Transactions on Sensor Networks,Vol.15,No.4.Article 44.Publication date:October 2019. 4444 AirContour: Building Contour-based Model for In-Air Writing Gesture Recognition YAFENG YIN and LEI XIE, State Key Laboratory for Novel Software Technology, Nanjing University, China TAO GU, RMIT University, Australia YIJIA LU and SANGLU LU, State Key Laboratory for Novel Software Technology, Nanjing University, China Recognizing in-air hand gestures will benefit a wide range of applications such as sign-language recogni￾tion, remote control with hand gestures, and “writing” in the air as a new way of text input. This article presents AirContour, which focuses on in-air writing gesture recognition with a wrist-worn device. We pro￾pose a novel contour-based gesture model that converts human gestures to contours in 3D space and then recognizes the contours as characters. Different from 2D contours, the 3D contours may have the problems such as contour distortion caused by different viewing angles, contour difference caused by different writing directions, and the contour distribution across different planes. To address the above problem, we introduce Principal Component Analysis (PCA) to detect the principal/writing plane in 3D space, and then tune the projected 2D contour in the principal plane through reversing, rotating, and normalizing operations, to make the 2D contour in right orientation and normalized size under a uniform view. After that, we propose both an online approach, AC-Vec, and an offline approach, AC-CNN, for character recognition. The experimen￾tal results show that AC-Vec achieves an accuracy of 91.6% and AC-CNN achieves an accuracy of 94.3% for gesture/character recognition, both outperforming the existing approaches. CCS Concepts: • Human-centered computing → Ubiquitous and mobile computing design and eval￾uation methods; Empirical studies in ubiquitous and mobile computing; Additional Key Words and Phrases: AirContour, in-air writing, contour-based gesture model, principal com￾ponent analysis (PCA), gesture recognition ACM Reference format: Yafeng Yin, Lei Xie, Tao Gu, Yijia Lu, and Sanglu Lu. 2019. AirContour: Building Contour-based Model for In-Air Writing Gesture Recognition. ACM Trans. Sen. Netw. 15, 4, Article 44 (October 2019), 25 pages. https://doi.org/10.1145/3343855 This work is supported by National Natural Science Foundation of China under Grant Nos. 61802169, 61872174, 61832008, 61321491; JiangSu Natural Science Foundation under Grant No. BK20180325; the Fundamental Research Funds for the Cen￾tral Universities under Grant No. 020214380049; Australian Research Council (ARC) Discovery Project Grants DP190101888 and DP180103932. This work is partially supported by Collaborative Innovation Center of Novel Software Technology and Industrialization. Authors’ addresses: Y. Yin, L. Xie (corresponding author), Y. Lu, and S. Lu, State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, 210023, China; emails: {yafeng, lxie}@nju.edu.cn, lyj@smail.nju.edu.cn, sanglu@ nju.edu.cn; T. Gu, School of Computer Science and Information Technology, RMIT University, Melbourne VIC 3000, Aus￾tralia; email: tao.gu@rmit.edu.au. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. © 2019 Association for Computing Machinery. 1550-4859/2019/10-ART44 $15.00 https://doi.org/10.1145/3343855 ACM Transactions on Sensor Networks, Vol. 15, No. 4, Article 44. Publication date: October 2019