Depth Aware Finger Tapping on Virtual Displays MobiSys'18,June 10-15,2018,Munich,Germany REFERENcES [33]Anthony Santella,Maneesh Agrawala,Doug DeCarlo,David Salesin,and Michael [1]Heba Abdelnasser,Moustafa Youssef,and Khaled A Harras.2015.WiGest:A Cohen.2006.Gaze-based interaction for semi-automatic photo cropping.In ubiquitous WiFi-based gesture recognition system.In Proceedings of IEEE INFO- Proceedings of ACM CHI COM [34]Hamed Sarbolandi.Damien Lefloch,and Andreas Kolb.2015.Kinect range [2]C Albitar,P Graebling,and C Doignon.2007.Robust Structured Light Coding for sensing:Structured-light versus Time-of-Flight Kinect aYE.Computer Vision and 3D Reconstruction.In Proceedings of IEEE ICCV. Image Understanding (2015). [3]Kamran Ali,Alex X.Liu,Wei Wang,and Muhammad Shahzad.2015.Keystroke [35]Jie Song.Gabor Soros,Fabrizio Pece,Sean Ryan Fanello,Shahram Izadi,Cem Recognition Using WiFi Signals.In Proceedings of ACM MobiCom. Keskin,and Otmar Hilliges.2014.In-air gestures around unmodified mobile [4]Amazon.2017.Alexa.https://developer.amazon.com/alexa.(2017). devices.In Proceedings of ACM UIST. [5]Gunilla Borgefors.1986.Distance transformations in digital images.Computer [36]Jue Wang.Deepak Vasisht,and Dina Katabi.2014.RF-IDraw: irtual to uch screen Vision Graphics Image Processing (1986). in the air using RF signals.In Proceedings of ACM SIGCOMM [6]Bo Chen,Vivek Yenamandra,and Kannan Srinivasan.2015.Tracking Keystrokes [37]Junjue Wang,Kaicher n Zhao,Xinyu Zhang,and Chunyi Peng.2014.Ubiquitous Using Wireless Signals.In Proceedings of ACM MobiSys. keyboard for small mobile devices:harnessing multipath fading for fine-grained [7]Ke-Yu Chen,Daniel Ashbrook,Mayank Goel,Sung-Hyuck Lee,and Shwetak keystroke localization.In Proceedings of ACM MobiSys. Patel 2014.AirLink:sharing files between multiple devices using in-air gestures. [38]Wei Wang.Alex X.Liu,and Ke Sun.2016.Device-free gesture tracking using In Proceedings of ACM UbiComp. acoustic signals.In Proceedings of ACM MobiCom. [8]Xiang Anthony Chen,Julia Schwarz,Chris Harrison,Jennifer Mankoff,and [39]Yan Wang.Jian Liu,Yingying Chen,Marco Gruteser,Jie Yan and Hongbo Liu Scott E.Hudson.2014. Air+touch:interweaving touch in-air gestures.In 2014.E-eyes:In-home Device-free Activity Identification Using Fine-grained Proceedings of ACM UIST. WiFi Signatures.In Proceedings of ACM MobiCom. [9]Patricia Ivette Cornelio Martinez,Silvana De Pirro,Chi Thanh Vi,and Sriram [40]Teng Wei and Xinyu Zhang.2015.mTrack:High-Precision Passive Tracking Subramanian.2017.Agency in mid-air interfaces.In Proceedings of ACM CHI. Using Millimeter Wave Radios.In Proceedings of ACM MobiCom. [10]Yan Cui,Sebastian Schuon,Derek Chan,Sebastian Thrun,and Christian Theobalt. [41]Frank Weichert.Daniel Bachmann,Bartholomaus Rudak,and Denis Fisseler.2013 2010 3D shape scanning with a time-of-flight camera.In Proceedings of IEER Analysis of the accuracy and robustness of the leap motion controller.Sensors CVPR 20131 [11]Google.2016.Google Project Soli.https://www.google.com/atap/project-soli/ [42]Xin Yi,Chun Yu,Mingrui Zhang.Sida Gao,Ke Sun,and Yuanchun Shi.2015. 20161. ATK:Enabling Ten-Finger Freehand Typing in Air Based on 3D Hand Tracking [12]Claire C Gordon,Cynthia L Blackwell,Bruce Bradtmiller.Joseph L Parha Data.In Proceedings of ACM UIST Jennifer Hotzman,Stephen P Paquette,Brian D Corner,and Belva M Hodge [43]Yafeng Yin,Oun Li,Lei Xie,Shanhe Yi,Ed Novak,and Sanglu Lu.2016.CamK:a 1989.2010 Anthropometric Survey of U.S.Marine Corps Personnel:Methods and Camera-based Keyboard for Small Mobile Devices.In Proceedings of IEEE INFO- Summary Statistics.Anthropometric Survey of U.s.army Personnel Methods and COM. Summary Statistics (1989). [44]Sangki Yun,Yi-Chao Chen,Huihuang Zheng.Lili Qi iu,and Wenguang Mao.2017. [13]Sidhant Gupta,Daniel Morris,Shwetak Patel,and Desney Tan.2012.Soundwave: Strata:Fine-Grained Acoustic-based Device-Free Tracking.In Proceedings of ACM using the doppler effect to sense gestures.In Proceedings of ACM CHI Mobisys. [14]Masakazu Higuchi and Takashi Komuro.2015.Multi-finger AR Typing Interface [45]Chi Zhang.Josh Tabor,Jialiang Zh and Xinyu Zhang.2015.Extending Mobile for Mobile Devices Using High-Speed Hand Motion Recognition.In Proceedings Interaction Through Near-Field Visible Light Sensing.In Proceedings of ACM of ACM CHL MobiCom [15]Intel.2017.Intel Realsense.http://www.intel.com/content/www/us/en/architecture [46]Guoming Zhang,Chen Yan,Xiaoyu Ji,Tianchen Zhang.Taimin Zhang.and and-technology/realsense-overview.html.(2017). Wenyuan Xu.2017.DolphinAttack:Inaudible voice commands.In Proceedings of [16]Bryce Kellogg.Vamsi Talla,and Shyamnath Gollakota.2014.Bringing gesture ACM CCS. recognition to all devices.In Proceedings of Usenix NSDI [47]Lide Zhang.Birjodh Tiwana,Zhiyun Oian,Zhaoguang Wang.Robert P.Dick. [17]Kourosh Khoshelham.2011.Accuracy analysis of kinect depth data.In ISPRS Zhuoqing Morley Mao,and Lei Yang.2010.Accurate online power estimation workshop Laser Scanning. and automatic battery behavior based power model generation for smartphones. [18]Jian Liu,Yan Wang,Gorkem Kar,Yingying Chen,Jie Yang.and Marco Gruteser. In Proceedings of IEEE CODES+ISSS. 2015.Snooping Keystrokes with mm-level Audio Ranging on a Single Phone.In [48]Chenyin Zhao and Songqing Fan.2002.Measurement of Fingers.Progress of Proceedings of ACM MobiCom. Anatomical Sciences (2002). [19]I Scott Mackenzie.1992.Fitts'Law as a Research and Design Tool in Human- Computer Interaction.Human-Computer Interaction(1992). [20]Pedro Melgarejo,Xinyu Zhang.Parameswaran Ramanathan,and David Chu.2014 Leveraging directional antenna capabilities for fine-grained gesture recognition In Proceedings of ACM UbiComp. [21]Microsoft.2014. Microsoft Kinect http://www.microsoft.com/en- us/kinectforwindows/.(2014). [22]Microsoft.2016.Microsoft Hololens.http://www.microsoft.com/microsoft- hololens/en-us.(2016). [23]Robert B Miller.1968.Response time in man-computer conversational transac- tions.In Proceedings of ACM AFIPS. [24]Mark R Mine,Frederick P Brooks Jr,and Carlo H Sequin.1997.Moving ob jects in space:exploiting proprioception in virtual-environment interaction.In Proceedings of ACM SIGGRAPH. [25]Leap Motion.2015.Leap Motion.https://www.leapmotion.com/.(2015). [26]Taichi Murase,Atsunori Moteki,Noriaki Ozawa,Nobuyuki Hara,Takehiro Nakai and Katsuhito Fujimoto.2011.Gesture keyboard requiring only one camera.In Proceedings of ACM UIST. [27]Rajalakshmi Nandakumar,Vikram lyer,Desney Tan,and Shyamnath Gollakota 2016.FingerlO:Using Active Sonar for Fine-Grained Finger Tracking.In Proceed- ings of ACM CHL [28]OptiTrack.2018.OptiTrack.http://www.optitrack.com.(2018) 29]Nobuyuki Otsu.1975.A threshold selection method from gray-level histograms. Automatica (1975). [30]S.L.Phung.A.Bouzerdoum,and D.Chai.2005.Skin segmentation using color pixel classification:analysis and comparison.IEEE Transactions on Pattern Analy sis Machine Intelligence (2005). [31]Qifan Pu,Sidhant Gupta,Shyamnath Gollakota,and Shwetak Patel.2013.Whole- home gesture recognition using wireless signals.In Proceedings ofACM MobiCom. [32]Rajesh PN Rao.2013.Brain-computer interfacing:an introduction.Cambridge iversity PresDepth Aware Finger Tapping on Virtual Displays MobiSys’18, June 10–15, 2018, Munich, Germany REFERENCES [1] Heba Abdelnasser, Moustafa Youssef, and Khaled A Harras. 2015. WiGest: A ubiquitous WiFi-based gesture recognition system. In Proceedings of IEEE INFOCOM. [2] C Albitar, P Graebling, and C Doignon. 2007. Robust Structured Light Coding for 3D Reconstruction. In Proceedings of IEEE ICCV. [3] Kamran Ali, Alex X. Liu, Wei Wang, and Muhammad Shahzad. 2015. Keystroke Recognition Using WiFi Signals. In Proceedings of ACM MobiCom. [4] Amazon. 2017. Alexa. https://developer.amazon.com/alexa. (2017). [5] Gunilla Borgefors. 1986. Distance transformations in digital images. Computer Vision Graphics & Image Processing (1986). [6] Bo Chen, Vivek Yenamandra, and Kannan Srinivasan. 2015. Tracking Keystrokes Using Wireless Signals. In Proceedings of ACM MobiSys. [7] Ke-Yu Chen, Daniel Ashbrook, Mayank Goel, Sung-Hyuck Lee, and Shwetak Patel. 2014. AirLink: sharing files between multiple devices using in-air gestures. In Proceedings of ACM UbiComp. [8] Xiang Anthony Chen, Julia Schwarz, Chris Harrison, Jennifer Mankoff, and Scott E. Hudson. 2014. Air+touch:interweaving touch & in-air gestures. In Proceedings of ACM UIST. [9] Patricia Ivette Cornelio Martinez, Silvana De Pirro, Chi Thanh Vi, and Sriram Subramanian. 2017. Agency in mid-air interfaces. In Proceedings of ACM CHI. [10] Yan Cui, Sebastian Schuon, Derek Chan, Sebastian Thrun, and Christian Theobalt. 2010. 3D shape scanning with a time-of-flight camera. In Proceedings of IEEE CVPR. [11] Google. 2016. Google Project Soli. https://www.google.com/atap/project-soli/. (2016). [12] Claire C Gordon, Cynthia L Blackwell, Bruce Bradtmiller, Joseph L Parham, Jennifer Hotzman, Stephen P Paquette, Brian D Corner, and Belva M Hodge. 1989. 2010 Anthropometric Survey of U.S. Marine Corps Personnel: Methods and Summary Statistics. Anthropometric Survey of U.s.army Personnel Methods and Summary Statistics (1989). [13] Sidhant Gupta, Daniel Morris, Shwetak Patel, and Desney Tan. 2012. Soundwave: using the doppler effect to sense gestures. In Proceedings of ACM CHI. [14] Masakazu Higuchi and Takashi Komuro. 2015. Multi-finger AR Typing Interface for Mobile Devices Using High-Speed Hand Motion Recognition. In Proceedings of ACM CHI. [15] Intel. 2017. Intel Realsense. http://www.intel.com/content/www/us/en/architectureand-technology/realsense-overview.html. (2017). [16] Bryce Kellogg, Vamsi Talla, and Shyamnath Gollakota. 2014. Bringing gesture recognition to all devices. In Proceedings of Usenix NSDI. [17] Kourosh Khoshelham. 2011. Accuracy analysis of kinect depth data. In ISPRS workshop Laser Scanning. [18] Jian Liu, Yan Wang, Gorkem Kar, Yingying Chen, Jie Yang, and Marco Gruteser. 2015. Snooping Keystrokes with mm-level Audio Ranging on a Single Phone. In Proceedings of ACM MobiCom. [19] I. Scott Mackenzie. 1992. Fitts’ Law as a Research and Design Tool in HumanComputer Interaction. Human-Computer Interaction (1992). [20] Pedro Melgarejo, Xinyu Zhang, Parameswaran Ramanathan, and David Chu. 2014. Leveraging directional antenna capabilities for fine-grained gesture recognition. In Proceedings of ACM UbiComp. [21] Microsoft. 2014. Microsoft Kinect. http://www.microsoft.com/enus/kinectforwindows/. (2014). [22] Microsoft. 2016. Microsoft Hololens. http://www.microsoft.com/microsofthololens/en-us. (2016). [23] Robert B Miller. 1968. Response time in man-computer conversational transactions. In Proceedings of ACM AFIPS. [24] Mark R Mine, Frederick P Brooks Jr, and Carlo H Sequin. 1997. Moving objects in space: exploiting proprioception in virtual-environment interaction. In Proceedings of ACM SIGGRAPH. [25] Leap Motion. 2015. Leap Motion. https://www.leapmotion.com/. (2015). [26] Taichi Murase, Atsunori Moteki, Noriaki Ozawa, Nobuyuki Hara, Takehiro Nakai, and Katsuhito Fujimoto. 2011. Gesture keyboard requiring only one camera. In Proceedings of ACM UIST. [27] Rajalakshmi Nandakumar, Vikram Iyer, Desney Tan, and Shyamnath Gollakota. 2016. FingerIO: Using Active Sonar for Fine-Grained Finger Tracking. In Proceedings of ACM CHI. [28] OptiTrack. 2018. OptiTrack. http://www.optitrack.com. (2018). [29] Nobuyuki Otsu. 1975. A threshold selection method from gray-level histograms. Automatica (1975). [30] S. L. Phung, A. Bouzerdoum, and D. Chai. 2005. Skin segmentation using color pixel classification: analysis and comparison. IEEE Transactions on Pattern Analysis & Machine Intelligence (2005). [31] Qifan Pu, Sidhant Gupta, Shyamnath Gollakota, and Shwetak Patel. 2013. Wholehome gesture recognition using wireless signals. In Proceedings of ACM MobiCom. [32] Rajesh PN Rao. 2013. Brain-computer interfacing: an introduction. Cambridge University Press. [33] Anthony Santella, Maneesh Agrawala, Doug DeCarlo, David Salesin, and Michael Cohen. 2006. Gaze-based interaction for semi-automatic photo cropping. In Proceedings of ACM CHI. [34] Hamed Sarbolandi, Damien Lefloch, and Andreas Kolb. 2015. Kinect range sensing: Structured-light versus Time-of-Flight Kinect âŸĘ. Computer Vision and Image Understanding (2015). [35] Jie Song, Gábor Sörös, Fabrizio Pece, Sean Ryan Fanello, Shahram Izadi, Cem Keskin, and Otmar Hilliges. 2014. In-air gestures around unmodified mobile devices. In Proceedings of ACM UIST. [36] Jue Wang, Deepak Vasisht, and Dina Katabi. 2014. RF-IDraw: virtual touch screen in the air using RF signals. In Proceedings of ACM SIGCOMM. [37] Junjue Wang, Kaichen Zhao, Xinyu Zhang, and Chunyi Peng. 2014. Ubiquitous keyboard for small mobile devices: harnessing multipath fading for fine-grained keystroke localization. In Proceedings of ACM MobiSys. [38] Wei Wang, Alex X. Liu, and Ke Sun. 2016. Device-free gesture tracking using acoustic signals. In Proceedings of ACM MobiCom. [39] Yan Wang, Jian Liu, Yingying Chen, Marco Gruteser, Jie Yang, and Hongbo Liu. 2014. E-eyes: In-home Device-free Activity Identification Using Fine-grained WiFi Signatures. In Proceedings of ACM MobiCom. [40] Teng Wei and Xinyu Zhang. 2015. mTrack: High-Precision Passive Tracking Using Millimeter Wave Radios. In Proceedings of ACM MobiCom. [41] Frank Weichert, Daniel Bachmann, Bartholomäus Rudak, and Denis Fisseler. 2013. Analysis of the accuracy and robustness of the leap motion controller. Sensors (2013). [42] Xin Yi, Chun Yu, Mingrui Zhang, Sida Gao, Ke Sun, and Yuanchun Shi. 2015. ATK: Enabling Ten-Finger Freehand Typing in Air Based on 3D Hand Tracking Data. In Proceedings of ACM UIST. [43] Yafeng Yin, Qun Li, Lei Xie, Shanhe Yi, Ed Novak, and Sanglu Lu. 2016. CamK: a Camera-based Keyboard for Small Mobile Devices. In Proceedings of IEEE INFOCOM. [44] Sangki Yun, Yi-Chao Chen, Huihuang Zheng, Lili Qiu, and Wenguang Mao. 2017. Strata: Fine-Grained Acoustic-based Device-Free Tracking. In Proceedings of ACM Mobisys. [45] Chi Zhang, Josh Tabor, Jialiang Zhang, and Xinyu Zhang. 2015. Extending Mobile Interaction Through Near-Field Visible Light Sensing. In Proceedings of ACM MobiCom. [46] Guoming Zhang, Chen Yan, Xiaoyu Ji, Tianchen Zhang, Taimin Zhang, and Wenyuan Xu. 2017. DolphinAttack: Inaudible voice commands. In Proceedings of ACM CCS. [47] Lide Zhang, Birjodh Tiwana, Zhiyun Qian, Zhaoguang Wang, Robert P. Dick, Zhuoqing Morley Mao, and Lei Yang. 2010. Accurate online power estimation and automatic battery behavior based power model generation for smartphones. In Proceedings of IEEE CODES+ISSS. [48] Chenyin Zhao and Songqing Fan. 2002. Measurement of Fingers. Progress of Anatomical Sciences (2002)