工程科学学报.第42卷，第11期：1433-1448.2020年11月 Chinese Journal of Engineering,Vol.42,No.11:1433-1448,November 2020 https://doi.org/10.13374/j.issn2095-9389.2020.03.24.002;http://cje.ustb.edu.cn 自然场景文本检测技术研究综述 白志程2)，李擎1,2)区，陈鹏》，郭立晴) 1)北京科技大学自动化学院，北京1000832)工业过程知识自动化教育部重点实验室，北京1000833)中国邮政储蓄银行金融科技创新 部.北京100808 ☒通信作者，E-mail:liging@ies.ustb.edu.cn 摘要文本检测在自动驾驶和跨模态图像检索中具有极为广泛的应用.该技术也是基于光学字符的文本识别任务中重要 的前置环节.目前，复杂场景下的文本检测仍极具挑战性.本文对自然场景文本检测进行综述，回顾了针对该问题的主要技 术和相关研究进展，并对研究现状进行分析.首先对问题进行概述，分析了自然场景中文本检测的主要特点：接着，介绍了经 典的基于连通域分析、基于滑动检测窗的自然场景文本检测技术：在此基础上，综述了近年来较为常用的深度学习文本检测 技术：最后，对自然场景文本检测未来可能的研究方向进行展望. 关键词文本检测：场景文本；连通域分析：图像处理：统计学习：深度学习 分类号TP18 Text detection in natural scenes:a literature review BAI Zhi-cheng2),LI Qing CHEN Peng,GUO Li-qing 1)School of Automation and Electrical Engineering,University of Science and Technology Beijing,Beijing 100083,China 2)Key Laboratory of Knowledge Automation for Industrial Processes,Ministry of Education,Beijing 100083,China 3)FINTECH Innovation Division,Postal Savings Bank of China,Beijing 100808,China Corresponding author,E-mail:liqing @ies.ustb.edu.cn ABSTRACT Text detection is widely applied in the automatic driving and cross-modal image retrieval fields.This technique is also an important pre-procedure in optical character-based text recognition tasks.At present,text detection in complex natural scenes remains a challenging topic.Because text distribution and orientation are varied in different scenes and domains,there is still room for improvement in existing computer vision-based text detection methods.To complicate matters,natural scene texts,such as those in guideposts and shop signs,always contain words in different languages.Even characters are missing from some natural scene texts. These circumstances present more difficulties for feature extraction and feature description,thereby weakening the detectability of existing computer vision and image processing methods.In this context,text detection applications in natural scenes were summarized in this paper,the classical and newly presented techniques were reviewed,and the research progress and status were analyzed.First,the definitions of natural scene text detection and associated concepts were provided based on an analysis of the main characteristics of this problem.In addition,the classic natural scene text detection technologies,such as connected component analysis-based methods and sliding detection window-based methods,were introduced comprehensively.These methods were also compared and discussed. Furthermore,common deep learning models for scene text detection of the past decade were also reviewed.We divided these models into two main categories:region proposal-based models and segmentation-based models.Accordingly,the typical detection and semantic segmentation frameworks,including Faster R-CNN,SSD,Mask R-CNN,FCN,and FCIS,were integrated in the deep learning methods reviewed in this section.Moreover,hybrid algorithms that use region proposal ideas and segmentation strategies were also analyzed.As 收稿日期：2020-03-24 基金项目：国家自然科学基金资助项目(11296089)自然场景文本检测技术研究综述 白志程1,2)，李    擎1,2) 苣，陈    鹏3)，郭立晴1) 1) 北京科技大学自动化学院，北京 100083    2) 工业过程知识自动化教育部重点实验室，北京 100083    3) 中国邮政储蓄银行金融科技创新 部，北京 100808 苣通信作者，E-mail：liqing@ies.ustb.edu.cn 摘    要    文本检测在自动驾驶和跨模态图像检索中具有极为广泛的应用. 该技术也是基于光学字符的文本识别任务中重要 的前置环节. 目前，复杂场景下的文本检测仍极具挑战性. 本文对自然场景文本检测进行综述，回顾了针对该问题的主要技 术和相关研究进展，并对研究现状进行分析. 首先对问题进行概述，分析了自然场景中文本检测的主要特点；接着，介绍了经 典的基于连通域分析、基于滑动检测窗的自然场景文本检测技术；在此基础上，综述了近年来较为常用的深度学习文本检测 技术；最后，对自然场景文本检测未来可能的研究方向进行展望. 关键词    文本检测；场景文本；连通域分析；图像处理；统计学习；深度学习 分类号    TP18 Text detection in natural scenes: a literature review BAI Zhi-cheng1,2) ，LI Qing1,2) 苣 ，CHEN Peng3) ，GUO Li-qing1) 1) School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing 100083, China 2) Key Laboratory of Knowledge Automation for Industrial Processes, Ministry of Education, Beijing 100083, China 3) FINTECH Innovation Division, Postal Savings Bank of China, Beijing 100808, China 苣 Corresponding author, E-mail: liqing@ies.ustb.edu.cn ABSTRACT    Text detection is widely applied in the automatic driving and cross-modal image retrieval fields. This technique is also an important pre-procedure in optical character-based text recognition tasks. At present, text detection in complex natural scenes remains a challenging  topic.  Because  text  distribution  and  orientation  are  varied  in  different  scenes  and  domains,  there  is  still  room  for improvement  in  existing  computer  vision-based  text  detection  methods.  To  complicate  matters,  natural  scene  texts,  such  as  those  in guideposts  and  shop  signs,  always  contain  words  in  different  languages.  Even  characters  are  missing  from  some  natural  scene  texts. These  circumstances  present  more  difficulties  for  feature  extraction  and  feature  description,  thereby  weakening  the  detectability  of existing computer vision and image processing methods. In this context, text detection applications in natural scenes were summarized in this paper, the classical and newly presented techniques were reviewed, and the research progress and status were analyzed. First, the definitions of natural scene text detection and associated concepts were provided based on an analysis of the main characteristics of this problem. In addition, the classic natural scene text detection technologies, such as connected component analysis-based methods and sliding  detection  window-based  methods,  were  introduced  comprehensively.  These  methods  were  also  compared  and  discussed. Furthermore, common deep learning models for scene text detection of the past decade were also reviewed. We divided these models into two main categories: region proposal-based models and segmentation-based models. Accordingly, the typical detection and semantic segmentation frameworks, including Faster R-CNN, SSD, Mask R-CNN, FCN, and FCIS, were integrated in the deep learning methods reviewed in this section. Moreover, hybrid algorithms that use region proposal ideas and segmentation strategies were also analyzed. As 收稿日期: 2020−03−24 基金项目: 国家自然科学基金资助项目（11296089） 工程科学学报，第 42 卷，第 11 期：1433−1448，2020 年 11 月 Chinese Journal of Engineering, Vol. 42, No. 11: 1433−1448, November 2020 https://doi.org/10.13374/j.issn2095-9389.2020.03.24.002; http://cje.ustb.edu.cn