第17卷第5期 智能系统学报 Vol.17 No.5 2022年9月 CAAI Transactions on Intelligent Systems Sep.2022 D0:10.11992/tis.202109022 网络出版地址：https:/kns.cnki.net/kcms/detail/23.1538.tp.20220615.1700.002.html 融合关系特征的半监督图像分类方法研究 刘威2，王薪予日，刘光伟，王东‘，牛英杰 (1.辽宁工程技术大学理学院，辽宁阜新123000：2.辽宁工程技术大学智能工程与数学研究院，辽宁阜新 123000:3.辽宁工程技术大学数学与系统科学研究所，辽宁阜新123000：4.辽宁工程技术大学矿业学院，辽宁 阜新123000) 摘要：半监督深度学习模型具有泛化能力强.所需样本数较少等特点，经过10多年的发展，在理论和实际应 用方面都取得了巨大的进步，然而建模样本内部“隐含”关系时模型缺乏解释性以及构造无监督正则化项难度 较大等问题限制了半监督深度学习的进一步发展。针对上述问题，从丰富样本特征表示的角度出发，构造了一 种新的半监督图像分类模型一融合关系特征的半监督分类模型(semi-supervised classification model fused with relational features,.SCUTTLE),该模型在卷积神经网络模型(convolutional neural networks,CNN)基础上引入了图 卷积神经网络(graph convolutional networks,.GCN),尝试通过GCN模型来提取CNN模型各层的低、高级特征间 的关系，使得融合模型不仅具有特征提取能力，而且具有关系表示能力。通过对SCUTTLE模型泛化性能进行 分析，进一步说明了该模型在解决半监督相关问题时的有效性。数值实验结果表明，三层CNN与一层GCN的 融合模型在CIFAR10、CIFAR1O0、SVHN3种数据集上与CNN监督学习模型的精度相比均可提升5%6%的精 度值，在最先进的ResNet、DenseNet、WRN(wide residual networks)与GCN的融合模型上同样证明了本文所提模 型的有效性。 关键词：关系表示：特征提取：图卷积神经网络：融合模型：半监督学习：图像分类：视觉卷积：泛化性能 中图分类号：TP181文献标志码：A文章编号：1673-4785(2022)05-0886-14 中文引用格式：刘威，王薪予，刘光伟，等.融合关系特征的半监督图像分类方法研究.智能系统学报，2022,17(5)： 886-899. 英文引用格式：LIU Wei,,WANG Xinyu,LIU Guangwei,,ctal.Semi-supervised image classification method fused with relational featuresJ CAAI transactions on intelligent systems,2022,17(5):886-899. Semi-supervised image classification method fused with relational features LIU Wei2,WANG Xinyu'3,LIU Guangwei,WANG Dong',NIU Yingjie'3 (1.School of Sciences,Liaoning Technical University,Fuxin 123000,China;2.Institutes of Intelligent Engineering and Mathematics, Liaoning Technical University,Fuxin 123000,China;3.Institute of Mathematics and Systems Science,Liaoning Technical Uni- versity,Fuxin 123000,China;4.School of Mining,Liaoning Technical University,Fuxin 123000,China) Abstract:A semi-supervised deep learning model exhibits great generalization ability with minimal required samples and has made great progress in theory and practical application over the past ten years or so.However,the lack of the model's interpretability when modeling the internal "implicit"relationship of samples and the difficulty in constructing unsupervised regularization items have limited the further development of semi-supervised deep learning.To solve these problems and enrich the sample feature representation,this study has developed a novel semi-supervised model for im- age classification-semi-supervised classification model integrating the relational features(SCUTTLE).The model in- troduces the graph convolutional networks(GCN)based on the convolutional neural networks(CNN)and extracts the relationships between the low-and high-level features of each layer of the CNN model via the GCN model,thus extract- ing features and expressing relationships.By analyzing the generalization performance of the SCUTTLE model,the pa- per further illustrates its effectiveness in solving semi-supervised related problems.The numerical results indicate that the classification accuracy of the hybrid model with three layers of CNN and one layer of GCN can be improved by 5%-6%compared to that of the CNN model on the CIFAR10,CIFAR100,and SVHN datasets.The effectiveness of the model proposed in this paper is also proved in the most advanced fusion models of ResNet,DenseNet.WRN(wide re- sidual networks),and GCN. Keywords:relationship representation;feature extraction;graph convolutional neural network;hybrid model;semi-su- pervised learning;image classification;convolution in vision;generalization performance 收稿日期：2021-09-13.网络出版日期：2022-06-16. 基金项目：国家自然科学基金项目(51974144,51874160)：辽宁 半监督学习，是一种介于监督学习和无监 省教育厅项目(LJKZ0340):辽宁工程技术大学学科 创新团队项目(LNTU20TD-01,LNTU20TD-07). 督学习之间的学习方法，其基本思想是在训练少 通信作者：刘威.E-mail:v8218218@126.com. 量标签样本的过程中引入大量无标签样本来解决DOI: 10.11992/tis.202109022 网络出版地址: https://kns.cnki.net/kcms/detail/23.1538.tp.20220615.1700.002.html 融合关系特征的半监督图像分类方法研究 刘威1,2,3，王薪予1,3，刘光伟4 ，王东4 ，牛英杰1,3 （1. 辽宁工程技术大学 理学院，辽宁 阜新 123000; 2. 辽宁工程技术大学 智能工程与数学研究院，辽宁 阜新 123000; 3. 辽宁工程技术大学 数学与系统科学研究所，辽宁 阜新 123000; 4. 辽宁工程技术大学 矿业学院，辽宁 阜新 123000） 摘 要：半监督深度学习模型具有泛化能力强，所需样本数较少等特点，经过 10 多年的发展，在理论和实际应 用方面都取得了巨大的进步，然而建模样本内部“隐含”关系时模型缺乏解释性以及构造无监督正则化项难度 较大等问题限制了半监督深度学习的进一步发展。针对上述问题，从丰富样本特征表示的角度出发，构造了一 种新的半监督图像分类模型—融合关系特征的半监督分类模型（semi-supervised classification model fused with relational features，SCUTTLE），该模型在卷积神经网络模型 (convolutional neural networks，CNN) 基础上引入了图 卷积神经网络 (graph convolutional networks，GCN)，尝试通过 GCN 模型来提取 CNN 模型各层的低、高级特征间 的关系，使得融合模型不仅具有特征提取能力，而且具有关系表示能力。通过对 SCUTTLE 模型泛化性能进行 分析，进一步说明了该模型在解决半监督相关问题时的有效性。数值实验结果表明，三层 CNN 与一层 GCN 的 融合模型在 CIFAR10、CIFAR100、SVHN 3 种数据集上与 CNN 监督学习模型的精度相比均可提升 5%~6% 的精 度值，在最先进的 ResNet、DenseNet、WRN（wide residual networks）与 GCN 的融合模型上同样证明了本文所提模 型的有效性。 关键词：关系表示；特征提取；图卷积神经网络；融合模型；半监督学习；图像分类；视觉卷积；泛化性能 中图分类号：TP181 文献标志码：A 文章编号：1673−4785(2022)05−0886−14 中文引用格式：刘威, 王薪予, 刘光伟, 等. 融合关系特征的半监督图像分类方法研究 [J]. 智能系统学报, 2022, 17(5): 886–899. 英文引用格式：LIU Wei, WANG Xinyu, LIU Guangwei, et al. Semi-supervised image classification method fused with relational features[J]. CAAI transactions on intelligent systems, 2022, 17(5): 886–899. Semi-supervised image classification method fused with relational features LIU Wei1,2,3 ，WANG Xinyu1,3 ，LIU Guangwei4 ，WANG Dong4 ，NIU Yingjie1,3 (1. School of Sciences, Liaoning Technical University, Fuxin 123000, China; 2. Institutes of Intelligent Engineering and Mathematics, Liaoning Technical University, Fuxin 123000, China; 3. Institute of Mathematics and Systems Science, Liaoning Technical Uni￾versity, Fuxin 123000, China; 4. School of Mining, Liaoning Technical University, Fuxin 123000, China) Abstract: A semi-supervised deep learning model exhibits great generalization ability with minimal required samples and has made great progress in theory and practical application over the past ten years or so. However, the lack of the model’s interpretability when modeling the internal “implicit” relationship of samples and the difficulty in constructing unsupervised regularization items have limited the further development of semi-supervised deep learning. To solve these problems and enrich the sample feature representation, this study has developed a novel semi-supervised model for im￾age classification—semi-supervised classification model integrating the relational features (SCUTTLE). The model in￾troduces the graph convolutional networks (GCN) based on the convolutional neural networks (CNN) and extracts the relationships between the low- and high-level features of each layer of the CNN model via the GCN model, thus extract￾ing features and expressing relationships. By analyzing the generalization performance of the SCUTTLE model, the pa￾per further illustrates its effectiveness in solving semi-supervised related problems. The numerical results indicate that the classification accuracy of the hybrid model with three layers of CNN and one layer of GCN can be improved by 5%–6% compared to that of the CNN model on the CIFAR10, CIFAR100, and SVHN datasets. The effectiveness of the model proposed in this paper is also proved in the most advanced fusion models of ResNet, DenseNet, WRN (wide re￾sidual networks), and GCN. Keywords: relationship representation; feature extraction; graph convolutional neural network; hybrid model; semi-su￾pervised learning; image classification; convolution in vision; generalization performance 半监督学习[1,2] 是一种介于监督学习和无监 督学习之间的学习方法，其基本思想是在训练少 量标签样本的过程中引入大量无标签样本来解决 收稿日期：2021−09−13. 网络出版日期：2022−06−16. 基金项目：国家自然科学基金项目 (51974144, 51874160)；辽宁 省教育厅项目 ( LJKZ0340)；辽宁工程技术大学学科 创新团队项目 (LNTU20TD-01,LNTU20TD- 07). 通信作者：刘威. E-mail：lv8218218@126.com. 第 17 卷第 5 期 智 能 系 统 学 报 Vol.17 No.5 2022 年 9 月 CAAI Transactions on Intelligent Systems Sep. 2022