Liu YL,Xu XG,Guo YW et al.Pores-preserving face cleaning based on improved empirical mode decomposition.JOURNAL OF COMPUTER SCIENCE AND TECHNOLOGY 24(3):557-567 May 2009 Pores-Preserving Face Cleaning Based on Improved Empirical Mode Decomposition Yan-Li Liul,2(刘艳丽)，Xiao-Gang Xu(徐晓刚)2，Yam-Wen Guo3(郭延文)，Jin Wang2(王进) Xin Duan!(段鑫)，Xi Chen1(陈曦)，and Qun-Sheng Peng1,2(彭群生)，Senior Member,CCF State Key Lab of CAD CG,Zhejiang University,Hangzhou 310027,China 2Department of Mathematics,Zhejiang University,Hangzhou 310027,China 2Department of Equipment Automatization,Dalian Naval Academy,Dalian 116026,China 3State Key Lab of Novel Software Technology,Nanjing University,Nanjing 210000,China E-mail:liuyanli@cad.zju.edu.cn:ywguo@nju.edu.cn:jwang@cad.zju.edu.cn Received May 29,2008;revised February 28,2009 Abstract In this paper,we propose a novel method of cleaning up facial imperfections such as bumps and blemishes that may detract from a pleasing digital portrait.Contrasting with traditional methods which tend to blur facial details, our method fully retains fine scale skin textures (pores etc.)of the subject.Our key idea is to find a quantity,namely normalized local energy,to capture different characteristics of fine scale details and distractions,based on empirical mode decomposition,and then build a quantitative measurement of facial skin appearance which characterizes both imperfections and facial details in a unified framework.Finally,we use the quantitative measurement as a guide to enhance facial skin.We also introduce a few high-level,intuitive parameters for controlling the amount of enhancement.In addition,an adaptive local mean and neighborhood limited empirical mode decomposition algorithm is also developed to improve in two respects the performance of empirical mode decomposition.It can effectively avoid the gray spots effect commonly associated with traditional empirical mode decomposition when dealing with high-nonstationary images. Keywords image enhancement,empirical mode decomposition,normalized local energy Introduction to preserve the former and remove the latter make face cleaning a challenging problem.For approaches to Facial skin not only exhibits various fine scale de- eliminating distractions with denoising techniques-3, tails,e.g.,pores,but also sometimes exhibits distrac- which are widely adopted by commercial software,fa- tions such as blemishes,bumps,acne scarring etc.From cial details such as pores are often accounted as noise many existing photo enhancement softwares that can and thereby removed. be used to remove facial distractions,a common char- In this paper,based on empirical mode decomposi- acteristic is observed:the resulting face is often over- tion](EMD),we set a quantity,i.e.,normalized local smoothed,of which the natural pores disappear com- energy (NLE),to reveal the characteristics of fine scale pletely.However,human skin is perception sensitive, and distractions.Building upon NLE,we further de- in which fine scale texture accounts for an important velop a quantitative characterization of facial skin ap- part in overall appearance.Over-smoothed face not pearance,namely imperfect degree,to depict the visual only looks unnatural but also dissolves the individua- perception of facial skin.Finally,we employ imperfect lity of a person.In this paper,we focus our attention degree as a guide to enhance facial skin.In addition on retaining fine skin texture features,e.g.,pores struc- we also propose an algorithm called adaptive local mean ture of the skin,while simultaneously removing facial and neighborhood limited empirical mode decomposi- imperfections such as scars,bumps and blemishes. tion (ALNEMD)to reduce the gray spots effect com- Since the pores and some distractions of imperfect monly associated with traditional EMD when dealing faces all belong to high frequency component of images with high nonstationary images. Regular Paper This work is supported by the National Natural Science Foundation of China under Grant Nos.60403038 and 60703084,the Natural Science Foundation of Jiangsu Province under Grant No.BK2007571,and the Natural Science Foundation of Liaoning Province under Grant No.20082176.Liu YL, Xu XG, Guo YW et al. Pores-preserving face cleaning based on improved empirical mode decomposition. JOURNAL OF COMPUTER SCIENCE AND TECHNOLOGY 24(3): 557–567 May 2009 Pores-Preserving Face Cleaning Based on Improved Empirical Mode Decomposition Yan-Li Liu1,2 (刘艳丽), Xiao-Gang Xu (徐晓刚) 2 , Yan-Wen Guo3 (郭延文), Jin Wang1 (王 进) Xin Duan1 (段 鑫), Xi Chen1 (陈 曦), and Qun-Sheng Peng1,2 (彭群生), Senior Member, CCF 1State Key Lab of CAD & CG, Zhejiang University, Hangzhou 310027, China 2Department of Mathematics, Zhejiang University, Hangzhou 310027, China 2Department of Equipment Automatization, Dalian Naval Academy, Dalian 116026, China 3State Key Lab of Novel Software Technology, Nanjing University, Nanjing 210000, China E-mail: liuyanli@cad.zju.edu.cn; ywguo@nju.edu.cn; jwang@cad.zju.edu.cn Received May 29, 2008; revised February 28, 2009. Abstract In this paper, we propose a novel method of cleaning up facial imperfections such as bumps and blemishes that may detract from a pleasing digital portrait. Contrasting with traditional methods which tend to blur facial details, our method fully retains fine scale skin textures (pores etc.) of the subject. Our key idea is to find a quantity, namely normalized local energy, to capture different characteristics of fine scale details and distractions, based on empirical mode decomposition, and then build a quantitative measurement of facial skin appearance which characterizes both imperfections and facial details in a unified framework. Finally, we use the quantitative measurement as a guide to enhance facial skin. We also introduce a few high-level, intuitive parameters for controlling the amount of enhancement. In addition, an adaptive local mean and neighborhood limited empirical mode decomposition algorithm is also developed to improve in two respects the performance of empirical mode decomposition. It can effectively avoid the gray spots effect commonly associated with traditional empirical mode decomposition when dealing with high-nonstationary images. Keywords image enhancement, empirical mode decomposition, normalized local energy 1 Introduction Facial skin not only exhibits various fine scale de￾tails, e.g., pores, but also sometimes exhibits distrac￾tions such as blemishes, bumps, acne scarring etc. From many existing photo enhancement softwares that can be used to remove facial distractions, a common char￾acteristic is observed: the resulting face is often over￾smoothed, of which the natural pores disappear com￾pletely. However, human skin is perception sensitive, in which fine scale texture accounts for an important part in overall appearance. Over-smoothed face not only looks unnatural but also dissolves the individua￾lity of a person. In this paper, we focus our attention on retaining fine skin texture features, e.g., pores struc￾ture of the skin, while simultaneously removing facial imperfections such as scars, bumps and blemishes. Since the pores and some distractions of imperfect faces all belong to high frequency component of images, to preserve the former and remove the latter make face cleaning a challenging problem. For approaches to eliminating distractions with denoising techniques[1−3] , which are widely adopted by commercial software, fa￾cial details such as pores are often accounted as noise and thereby removed. In this paper, based on empirical mode decomposi￾tion[4] (EMD), we set a quantity, i.e., normalized local energy (NLE), to reveal the characteristics of fine scale and distractions. Building upon NLE, we further de￾velop a quantitative characterization of facial skin ap￾pearance, namely imperfect degree, to depict the visual perception of facial skin. Finally, we employ imperfect degree as a guide to enhance facial skin. In addition, we also propose an algorithm called adaptive local mean and neighborhood limited empirical mode decomposi￾tion (ALNEMD) to reduce the gray spots effect com￾monly associated with traditional EMD when dealing with high nonstationary images. Regular Paper This work is supported by the National Natural Science Foundation of China under Grant Nos. 60403038 and 60703084, the Natural Science Foundation of Jiangsu Province under Grant No. BK2007571, and the Natural Science Foundation of Liaoning Province under Grant No. 20082176