第16卷第3期 智能系统学报 Vol.16 No.3 2021年5月 CAAI Transactions on Intelligent Systems May 2021 D0L:10.11992tis.202008034 基于迁移学习的无监督跨域人脸表情识别 莫宏伟，傅智杰 (哈尔滨工程大学自动化学院，黑龙江哈尔滨150001) 摘要：本文主要研究了基于迁移学习的无监督跨域人脸表情识别。在过去的几年里，提出的许多方法在人脸 表情识别方面取得了令人满意的识别效果。但这些方法通常认为训练和测试数据来自同一个数据集，因此其 具有相同的分布。而在实际应用中，这一假设通常并不成立，特别当训练集和测试集来自不同的数据集时，即 跨域人脸表情识别问题。为了解决这一问题，本文提出将一种基于联合分布对齐的迁移学习方法(domain align learning)应用于跨域人脸表情识别.该方法通过找到一个特征变换，将源域和目标域数据映射到一个公共子空 间中，在该子空间中联合对齐边缘分布和条件分布来减小域之间的分布差异，然后对变换后的特征进行训练得 到一个域适应分类器来预测目标域样本标签。为了验证提出算法的有效性，在CK+、Oulu-CASIA NIR和Oulu CASIA VIS这3个不同的数据库上做了大量实验，实验结果证明所提算法在跨域表情识别上是有效性的。 关键词：表情识别；无监督；跨域；迁移学习；特征变换：联合对齐；公共子空间：域适应 中图分类号：TP181文献标志码：A文章编号：1673-4785(2021)03-0397-10 中文引用格式：莫宏伟，傅智杰.基于迁移学习的无监督跨域人脸表情识别.智能系统学报，2021,16(3)：397-406. 英文引用格式：MO Hongwei,FU Zhijie.Unsupervised cross-domain expression recognition based on transfer learning.CAAI transactions on intelligent systems,2021,16(3):397-406. Unsupervised cross-domain expression recognition based on transfer learning MO Hongwei,FU Zhijie (Automation College,Harbin Engineering University,Harbin 150001,China) Abstract:This paper primarily studies unsupervised cross-domain facial expression recognition based on transfer learn- ing.In recent years,many proposed methods have achieved satisfactory results in facial expression recognition. However,these methods usually assume that the training and test data come from the same data set and therefore have the same distribution.In practical applications,this assumption is usually untrue,especially when the training and test sets come from different data sets,also known as the cross-domain facial expression recognition problem.To solve this problem,we propose a migration learning method(domain align learning)based on joint distributed alignment for cross- domain facial expression recognition.By determining a feature transform,the source and target domain data are mapped onto a common subspace,wherein edge distribution and conditional distribution are aligned jointly to reduce the distri- bution difference between domains,and then a domain adaptive classifier is obtained by training the transformed fea- tures to predict the target domain sample label.To verify the effectiveness of the proposed algorithm,many experiments are performed on three databases,CK+,Oulu-CASIA NIR,and Oulu-CASIA VIS.The experimental results show the ef- fectiveness of the proposed algorithm in cross-domain facial expression recognition. Keywords:expression recognition;unsupervised;cross-domain;transfer learning;feature transformation;joint align- ment;public subspace;domain adaptive 人脸表情自动识别是情感计算和计算机视觉 收稿日期：2020-08-28. 基金项目：国家自然科学基金项目(60035117). 领域的研究热点之一。近年来，随着人工智能 通信作者：莫宏伟.E-mail:honwei2004@126.com 的迅猛发展，人脸表情识别在多媒体娱乐、人机DOI: 10.11992/tis.202008034 基于迁移学习的无监督跨域人脸表情识别 莫宏伟，傅智杰 （哈尔滨工程大学 自动化学院，黑龙江 哈尔滨 150001） 摘 要：本文主要研究了基于迁移学习的无监督跨域人脸表情识别。在过去的几年里，提出的许多方法在人脸 表情识别方面取得了令人满意的识别效果。但这些方法通常认为训练和测试数据来自同一个数据集，因此其 具有相同的分布。而在实际应用中，这一假设通常并不成立，特别当训练集和测试集来自不同的数据集时，即 跨域人脸表情识别问题。为了解决这一问题，本文提出将一种基于联合分布对齐的迁移学习方法 (domain align learning) 应用于跨域人脸表情识别，该方法通过找到一个特征变换，将源域和目标域数据映射到一个公共子空 间中，在该子空间中联合对齐边缘分布和条件分布来减小域之间的分布差异，然后对变换后的特征进行训练得 到一个域适应分类器来预测目标域样本标签。为了验证提出算法的有效性，在 CK+、Oulu-CASIA NIR 和 Oulu￾CASIA VIS 这 3 个不同的数据库上做了大量实验，实验结果证明所提算法在跨域表情识别上是有效性的。 关键词：表情识别；无监督；跨域；迁移学习；特征变换；联合对齐；公共子空间；域适应 中图分类号：TP181 文献标志码：A 文章编号：1673−4785(2021)03−0397−10 中文引用格式：莫宏伟, 傅智杰. 基于迁移学习的无监督跨域人脸表情识别 [J]. 智能系统学报, 2021, 16(3): 397–406. 英文引用格式：MO Hongwei, FU Zhijie. Unsupervised cross-domain expression recognition based on transfer learning[J]. CAAI transactions on intelligent systems, 2021, 16(3): 397–406. Unsupervised cross-domain expression recognition based on transfer learning MO Hongwei，FU Zhijie (Automation College, Harbin Engineering University, Harbin 150001, China) Abstract: This paper primarily studies unsupervised cross-domain facial expression recognition based on transfer learn￾ing. In recent years, many proposed methods have achieved satisfactory results in facial expression recognition. However, these methods usually assume that the training and test data come from the same data set and therefore have the same distribution. In practical applications, this assumption is usually untrue, especially when the training and test sets come from different data sets, also known as the cross-domain facial expression recognition problem. To solve this problem, we propose a migration learning method (domain align learning) based on joint distributed alignment for cross￾domain facial expression recognition. By determining a feature transform, the source and target domain data are mapped onto a common subspace, wherein edge distribution and conditional distribution are aligned jointly to reduce the distri￾bution difference between domains, and then a domain adaptive classifier is obtained by training the transformed fea￾tures to predict the target domain sample label. To verify the effectiveness of the proposed algorithm, many experiments are performed on three databases, CK+, Oulu-CASIA NIR, and Oulu-CASIA VIS. The experimental results show the ef￾fectiveness of the proposed algorithm in cross-domain facial expression recognition. Keywords: expression recognition; unsupervised; cross-domain; transfer learning; feature transformation; joint align￾ment; public subspace; domain adaptive 人脸表情自动识别是情感计算和计算机视觉 领域的研究热点之一[1-5]。近年来，随着人工智能 的迅猛发展，人脸表情识别在多媒体娱乐、人机 收稿日期：2020−08−28. 基金项目：国家自然科学基金项目 (60035117). 通信作者：莫宏伟. E-mail：honwei2004@126.com. 第 16 卷第 3 期 智 能 系 统 学 报 Vol.16 No.3 2021 年 5 月 CAAI Transactions on Intelligent Systems May 2021