·400· 智能系统学报 第17卷 Net算法在CK+数据集上能够对害怕和惊讶两种 [4]ZHAO Guoying,PIETIKAINEN M.Dynamic texture re- 表情达到100%识别率：遮挡嘴巴后，能够对困 cognition using local binary patterns with an application 惑、快乐和惊讶3种表情达到100%识别率。而 to facial expressions[J].IEEE transactions on pattern ana- 轻蔑和恐惧表情的F1-score分别只达到0.76和 lysis and machine intelligence,2007,29(6):915-928. 0.75,说明这两种表情的有效特征大部分在于嘴 [5]WHITEHILL J,OMLIN C W.Haar features for FACS 巴部分。 AU recognition[C]//Proceedings of the 7th International 由图9和表9可知，遮挡眼睛情况下的悲伤 Conference on Automatic Face and Gesture Recognition. Southampton,UK,2006:5-101. 表情F1-score仅达到0.82，说明悲伤表情的有效 [6]BARTLETT MS,LITTLEWORT G,FRANK M.et al. 特征大部分在于眼睛部分，虽然该值达到最低， Recognizing facial expression:machine learning and ap- 但DMFA-ResNet在JAFFE数据集上也取得相当 plication to spontaneous behavior[C]//Proceedings of 不错的效果。由于该数据集样本间的差异较小， 2005 IEEE Computer Society Conference on Computer 导致算法仍出现较多误判情况，无法完全精准识 Vision and Pattern Recognition.San Diego,USA,2005: 别某一类表情。以上实验结果证明了DMFA-Res 568-573. Nt在应对遮挡图像问题上的优越性，更适用于 [7]LI Xiaobai,PFISTER T,HUANG Xiaohua,et al.A spon- 人脸表情识别任务。 taneous micro-expression database:inducement,collec- tion and baseline[C]//2013 10th IEEE International Con- 4结束语 ference and Workshops on Automatic Face and Gesture Recognition (FG).Shanghai,China,2013:1-6. 本文提出一种多尺度融合注意力残差网络 [8]RIVERA A R.CASTILLO J R.CHA E OO.Local direc- (DMFA-ResNet)。该网络主要提出一种新的注意 tional number pattern for face analysis:face and expres- 力残差模块，提高了网络对局部重点部位特征的 sion recognition[J].IEEE transactions on image pro- 提取，有利于学习到非遮挡部位的信息；提出多 cessing,.2013,22(5:1740-1752 尺度融合模块，将各残差模块的输出进行融合以 [9]KIM T H,YU C,LEE S W.Facial expression recogni- 提取更加丰富的人脸表情特征；为了减少参与网 tion using feature additive pooling and progressive fine- 络运算的参数量，在各个残差模块之间添加过渡 tuning of CNN[J].Electronics letters,2018,54(23): 层，主要进行下采样操作并使用全局平均池化+ 1326-1328. Dropout设计防止网络过拟合。在CK+、JAFFE [10]AN Fengping,LIU Zhiwen.Facial expression recogni- 和Oulu-CASIA数据集上进行实验均取得了不错 tion algorithm based on parameter adaptive initialization 的效果，注意力残差模块对局部区域的特征能够 of CNN and LSTM[J].The visual computer,2020. 36(3):483-498. 进行有效提取，实验验证本文算法具有优越性。 但所提算法为针对静态图像的表情识别算法，不 [11]XIE Siyue,HU Haifeng,WU Yongbo.Deep multi-path convolutional neural network joint with salient region at- 适用于动态连续的视频识别，在接下来的工作中， tention for facial expression recognition[J].Pattern re- 可以重点研究基于视频的动态表情识别技术。 cognition,2019,92:177-191 参考文献： [12]WANG Kai,PENG Xiaojiang,YANG Jianfei,et al. Suppressing uncertainties for large-scale facial expres- [1]BEN Xianye,REN Yi,ZHANG Junping,et al.Video- sion recognition[C]//Proceedings of 2020 IEEE/CVF based Facial micro-expression analysis:a survey of data- Conference on Computer Vision and Pattern Recogni- sets,features and algorithms[EB/OLl.(2021-03-19)[2021- tion.Seattle,USA,2020:6897-6906 05-01].https://arxiv.org/abs/2201.12728v1. [13]HE Kaiming,ZHANG Xiangyu,REN Shaoqing,et al. [2]CHEN Boyu,GUAN Wenlong,LI Peixia,et al.Residual Deep residual learning for image recognition[Cl//Pro- multi-task learning for facial landmark localization and ceedings of 2016 IEEE Conference on Computer Vision expression recognition [EB/OL].(2021-07-01)[2021-07- and Pattern Recognition.Las Vegas,USA,2016: 05].https://www.sciencedirect.com/science/article/pii/S00 770-778 31320321000807. [14]LI Yong,ZENG Jiabei,SHAN Shiguang,et al.Occlu- [3]LI Shan,DENG Weihong.Deep facial expression recog- sion aware facial expression recognition using CNN with nition:a survey [EB/OL].(2020-03-17)[2021-05-01].ht- attention mechanism[J].IEEE transactions on image pro- tps://ieeexplore.ieee.org/document/9039580. cessing,2019,28(5):2439-2450.Net 算法在 CK+数据集上能够对害怕和惊讶两种 表情达到 100% 识别率；遮挡嘴巴后，能够对困 惑、快乐和惊讶 3 种表情达到 100% 识别率。而 轻蔑和恐惧表情的 F1-score 分别只达到 0.76 和 0.75，说明这两种表情的有效特征大部分在于嘴 巴部分。 由图 9 和表 9 可知，遮挡眼睛情况下的悲伤 表情 F1-score 仅达到 0.82，说明悲伤表情的有效 特征大部分在于眼睛部分，虽然该值达到最低， 但 DMFA-ResNet 在 JAFFE 数据集上也取得相当 不错的效果。由于该数据集样本间的差异较小， 导致算法仍出现较多误判情况，无法完全精准识 别某一类表情。以上实验结果证明了 DMFA-Res￾Net 在应对遮挡图像问题上的优越性，更适用于 人脸表情识别任务。 4 结束语 本文提出一种多尺度融合注意力残差网络 (DMFA-ResNet)。该网络主要提出一种新的注意 力残差模块，提高了网络对局部重点部位特征的 提取，有利于学习到非遮挡部位的信息；提出多 尺度融合模块，将各残差模块的输出进行融合以 提取更加丰富的人脸表情特征；为了减少参与网 络运算的参数量，在各个残差模块之间添加过渡 层，主要进行下采样操作并使用全局平均池化+ Dropout 设计防止网络过拟合。在 CK+、JAFFE 和 Oulu-CASIA 数据集上进行实验均取得了不错 的效果，注意力残差模块对局部区域的特征能够 进行有效提取，实验验证本文算法具有优越性。 但所提算法为针对静态图像的表情识别算法，不 适用于动态连续的视频识别，在接下来的工作中， 可以重点研究基于视频的动态表情识别技术。 参考文献： BEN Xianye, REN Yi, ZHANG Junping, et al. Video￾based Facial micro-expression analysis: a survey of data￾sets, features and algorithms[EB/OL].(2021-03-19)[2021- 05-01].https://arxiv.org/abs/2201.12728v1. [1] CHEN Boyu, GUAN Wenlong, LI Peixia, et al. Residual multi-task learning for facial landmark localization and expression recognition[EB/OL].(2021-07-01)[2021-07- 05].https://www.sciencedirect.com/science/article/pii/S00 31320321000807. [2] LI Shan, DENG Weihong. Deep facial expression recog￾nition: a survey[EB/OL].（2020-03-17）[2021-05-01]. ht￾tps://ieeexplore.ieee.org/document/9039580. [3] ZHAO Guoying, PIETIKAINEN M. Dynamic texture re￾cognition using local binary patterns with an application to facial expressions[J]. IEEE transactions on pattern ana￾lysis and machine intelligence, 2007, 29(6): 915–928. [4] WHITEHILL J, OMLIN C W. Haar features for FACS AU recognition[C]//Proceedings of the 7th International Conference on Automatic Face and Gesture Recognition. Southampton, UK, 2006: 5−101. [5] BARTLETT M S, LITTLEWORT G, FRANK M, et al. Recognizing facial expression: machine learning and ap￾plication to spontaneous behavior[C]//Proceedings of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego, USA, 2005: 568−573. [6] LI Xiaobai, PFISTER T, HUANG Xiaohua, et al. A spon￾taneous micro-expression database: inducement, collec￾tion and baseline[C]//2013 10th IEEE International Con￾ference and Workshops on Automatic Face and Gesture Recognition (FG). Shanghai, China, 2013: 1−6. [7] RIVERA A R, CASTILLO J R, CHA E O O. Local direc￾tional number pattern for face analysis: face and expres￾sion recognition[J]. IEEE transactions on image pro￾cessing, 2013, 22(5): 1740–1752. [8] KIM T H, YU C, LEE S W. Facial expression recogni￾tion using feature additive pooling and progressive fine￾tuning of CNN[J]. Electronics letters, 2018, 54(23): 1326–1328. [9] AN Fengping, LIU Zhiwen. Facial expression recogni￾tion algorithm based on parameter adaptive initialization of CNN and LSTM[J]. The visual computer, 2020, 36(3): 483–498. [10] XIE Siyue, HU Haifeng, WU Yongbo. Deep multi-path convolutional neural network joint with salient region at￾tention for facial expression recognition[J]. Pattern re￾cognition, 2019, 92: 177–191. [11] WANG Kai, PENG Xiaojiang, YANG Jianfei, et al. Suppressing uncertainties for large-scale facial expres￾sion recognition[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recogni￾tion. Seattle, USA, 2020: 6897−6906. [12] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Pro￾ceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA, 2016: 770−778. [13] LI Yong, ZENG Jiabei, SHAN Shiguang, et al. Occlu￾sion aware facial expression recognition using CNN with attention mechanism[J]. IEEE transactions on image pro￾cessing, 2019, 28(5): 2439–2450. [14] ·400· 智 能 系 统 学 报 第 17 卷