175 UbiComp/ISWC '20 Adjunct,September 12-16,2020,Virtual Event,Mexico Lingyu Zhang and Yafeng Yin,et al. 5 CONCLUSION In this paper,we propose a homework auto-checking system HmwkCheck focusing on the recognition of four arithmetic operations for low- grade pupils.We use the embedded camera of a smartphone to take 1+31=83 明-64=26 a picture of the homework,and then process the image to recog- nize characters.After that,we calculate the arithmetic expression 5+2454 to check the handwritten answer.We implement HmwkCheck in an Android smartphone.The experiment results show that we can 51+31=82(位c10】 recognize characters accurately. ACKNOWLEDGMENTS (a) (b) This work is supported by National Natural Science Foundation of Figure 6:System implementation China under Grant Nos.61802169,61872174,61832008,61902175 61906085;JiangSu Natural Science Foundation under Grant Nos. that of the homework for low-grade pupils.Each volunteer calcu- BK20180325,BK20190293;the Key R&D Program of Jiangsu Province lated all the 40 expressions in a piece of paper and wrote the answers. under Grant No.BE2018116.This work is partially supported by Unless otherwise specified,we use the A4-sized white paper and Collaborative Innovation Center of Novel Software Technology and Samsung Note 8 smartphone to take pictures.To train the 'printed- model',we print out numerals and operators,use smartphone to Industrialization.Yafeng Yin is the corresponding author. take pictures,and get 360 images for each printed character.To REFERENCES train the 'handwritten-model',we use the public USPS data set [1] [1]Jonathan J.Hull.1994.A database for handwritten text recognition research.IEEE and randomly select 2150 images for each handwritten character. Transactions on Pattern Analysis and Machine Intelligence 16,5(1994),550-554. Considering that the calculation of arithmetic expression has cer- [2]Y.Jiang.H.Dong.and A.El Saddik.2018.Baidu Meizu Deep Learning Competition: tain rules which are fixed.we focus on the evaluation of character etic Operation Re IEEE Access6(2018).60128-60136 cognition Using End-to-End Learning OCR Technologies. recognition [3]S.Khalighi,P.Tirdad,H.R.Rabiee,and M.Parviz.2009.A Novel OCR System for In Fig.7,we show the recognition performance for printed char- Calculating Handwritten Persian Arithmetic Expressions.In 2009 International Conference on Machine Learning and Applications.755-758. acters,handwritten characters,and both of the them.For printed [4]Xiaoshuo Li,Tiezhu Yue,Xuanping Huang.Zhe Yang.and Gang Xu.2019.BAGS: characters,they have fixed typefaces,thus can be recognized well. An automatic homework grading system using the pictures taken by smart phones. i.e.,the precision,recall and F1-score achieve 96.22%,97.01%and CoRR abs/,1906.03767(2019.arXiv:1906.03767htp:arxiv.org/abs/1906.03767 [5]Wang Meng,Bian Guang-rong.Zheng Kai,He Ninghui,and Zhang Yan-lei.2017. 96.61%,respectively.The failure of recognizing some printed char- Recognition Technology for Four Arithmetic Operations.TELKOMNIKA Telecom- acters may be caused by the interference in the input image,e.g., munication Computing Electronics and Control 15,1(2017),306-313. smudges around the printed characters.For handwritten characters, different people can write the same character in different styles, thus the recognition performance of handwritten characters is not as good as that of printed characters,i.e.,the precision,recall and F1-score are 88.24%,84.34%and 86.25%,respectively.In addition, some illegible handwritten characters also decrease the recogni- tion performance.For example,two handwritten characters having overlap may be segmented as one character,the strokes in one character away from each other may be segmented as two charac- ters,which can lead to the recognition error.Overall,HmwkCheck can recognize characters efficiently,the average precision,recall and F1-score of character recognition achieve 94.03%,93.41%and 93.72%,respectively. 0 70 50 40 10 Printed Handwritten A Figure 7:Character recognition performanceUbiComp/ISWC ’20 Adjunct, September 12–16, 2020, Virtual Event, Mexico Lingyu Zhang and Yafeng Yin, et al. Figure 6: System implementation that of the homework for low-grade pupils. Each volunteer calcu￾lated all the 40 expressions in a piece of paper and wrote the answers. Unless otherwise specified, we use the A4-sized white paper and Samsung Note 8 smartphone to take pictures. To train the ‘printed￾model’, we print out numerals and operators, use smartphone to take pictures, and get 360 images for each printed character. To train the ‘handwritten-model’, we use the public USPS data set [1] and randomly select 2150 images for each handwritten character. Considering that the calculation of arithmetic expression has cer￾tain rules which are fixed, we focus on the evaluation of character recognition. In Fig. 7, we show the recognition performance for printed char￾acters, handwritten characters, and both of the them. For printed characters, they have fixed typefaces, thus can be recognized well, i.e., the precision, recall and F1-score achieve 96.22%, 97.01% and 96.61%, respectively. The failure of recognizing some printed char￾acters may be caused by the interference in the input image, e.g., smudges around the printed characters. For handwritten characters, different people can write the same character in different styles, thus the recognition performance of handwritten characters is not as good as that of printed characters, i.e., the precision, recall and F1-score are 88.24%, 84.34% and 86.25%, respectively. In addition, some illegible handwritten characters also decrease the recogni￾tion performance. For example, two handwritten characters having overlap may be segmented as one character, the strokes in one character away from each other may be segmented as two charac￾ters, which can lead to the recognition error. Overall, HmwkCheck can recognize characters efficiently, the average precision, recall and F1-score of character recognition achieve 94.03%, 93.41% and 93.72%, respectively. Figure 7: Character recognition performance 5 CONCLUSION In this paper, we propose a homework auto-checking system HmwkCheck focusing on the recognition of four arithmetic operations for low￾grade pupils. We use the embedded camera of a smartphone to take a picture of the homework, and then process the image to recog￾nize characters. After that,we calculate the arithmetic expression to check the handwritten answer. We implement HmwkCheck in an Android smartphone. The experiment results show that we can recognize characters accurately. ACKNOWLEDGMENTS This work is supported by National Natural Science Foundation of China under Grant Nos. 61802169, 61872174, 61832008, 61902175, 61906085; JiangSu Natural Science Foundation under Grant Nos. BK20180325, BK20190293; the Key R&D Program of Jiangsu Province under Grant No. BE2018116. This work is partially supported by Collaborative Innovation Center of Novel Software Technology and Industrialization. Yafeng Yin is the corresponding author. REFERENCES [1] Jonathan J. Hull. 1994. A database for handwritten text recognition research. IEEE Transactions on Pattern Analysis and Machine Intelligence 16, 5 (1994), 550–554. [2] Y. Jiang, H. Dong, and A. El Saddik. 2018. Baidu Meizu Deep Learning Competition: Arithmetic Operation Recognition Using End-to-End Learning OCR Technologies. IEEE Access 6 (2018), 60128–60136. [3] S. Khalighi, P. Tirdad, H. R. Rabiee, and M. Parviz. 2009. A Novel OCR System for Calculating Handwritten Persian Arithmetic Expressions. In 2009 International Conference on Machine Learning and Applications. 755–758. [4] Xiaoshuo Li, Tiezhu Yue, Xuanping Huang, Zhe Yang, and Gang Xu. 2019. BAGS: An automatic homework grading system using the pictures taken by smart phones. CoRR abs/1906.03767 (2019). arXiv:1906.03767 http://arxiv.org/abs/1906.03767 [5] Wang Meng, Bian Guang-rong, Zheng Kai, He Ninghui, and Zhang Yan-lei. 2017. Recognition Technology for Four Arithmetic Operations. TELKOMNIKA Telecom￾munication Computing Electronics and Control 15, 1 (2017), 306–313. 175