558 J.Comput.Sci.Technol.,May 2009,Vol.24.No.3 In the process of facial image cleaning,our algorithm In the field of face recognition,Lin et al.o]used only requests the user to tune a few high level.intuitive scale-invariant feature transform (SIFT)framework to parameters to interactively control the amount of en- detect irregular skin details.Later,Pierrard et al.] hancement.The advantage of our technique is that employed 3D morphable model reconstruction to rec- while effectively removing facial imperfections,it does ognize facial moles.Their techniques are effective in not blur fine scale facial details.Another important detecting moles with relatively fixed pattern,for exam- feature of our technique is:it is a general model which ple,circular shape.However,as these methods detect characterizes both imperfections and facial details in a and localize facial moles in spatial domain,they are unified framework,thereby it does not require user to not suitable for detecting and localizing general skin ir- interactively mark imperfections on facial images. regularities,due to a large number of possible spatial The rest of this paper is structured as follows.Sec- variations of distractions.Instead of explicitly detect- tion 2 reviews some related work about face cleaning ing and localizing distractions,our new approach sug- and EMD.Section 3 describes our improved EMD al- gests a quantitative characterization of both pores and gorithm.The algorithm of face cleaning is introduced distractions,and uses it to enhance facial skin. in detail in Section 4.Experimental results are demon- Among the current image filtering techniques,bilat- strated in Section 5.Finally.Section 6 concludes the eral filteringlil is one of the most powerful filters and whole paper and highlights future work. has been used in various fields.For the typical case in which the spatial and intensity weighting functions are 21 Related Work and Background Gaussian,there are two important parameters,namely geometric spread and photometric spread.While small 2.1 Related Work geometric spread corresponds to filtering small intensity Face Cleaning.The problem of facial image editing changes,large photometric spread would preserve edges has received much attention both from the computer with large discontinuity.For face cleaning,of which the graphics and image processing communities.As a re- aim is to smooth large discontinuity and preserve small sult,many techniques have been developed to achieve scale texture,the settings for the two parameters are various facial appearance effects.Leyvand et al.5]pro- difficult:when the photometric spread is set to a very posed an approach to enhanceing the facial attractive- large value in order to smooth imperfections,the bilate- ness by adjusting facial features.Nguyen et al.6]pre- ral filter nearly degenerates into a Gaussian filter.In sented a layer extraction method to remove and syn- such a case,the smoothing effect of bilateral filter only thesize beard.Using reflectance transfer,Peers et al7 relies on the geometric spread.Apparently,the geo- produced face relighting in the post-production process. metric spread cannot be set to a large value,otherwise Most recently,Bitouks introduced an algorithm that the pores of skin will be removed.On the other hand, can automatically replace faces in photographs.More- the edges of bumps or scars in the skin are left aside if over.the algorithm also allows the user to interactively it is set to a small value. edit the illumination and colors of face. To the best of our knowledge,the most related work However,little research work has been published on to ours is developed by Matsui et al.2.This work also face cleaning in literature.Nevertheless,in practice, aims at removing spots and at the same time preserv- several methods have been employed to clean or en- ing skin natural roughness.The method uses e-filters to hance facial images.While cleaning face with denois- decompose image into several different frequency com- ing techniques is popular,an alternative approach to ponents.It assumes spots are of medium frequency serving this problem is the so-called interactive cut- and pores of high frequency,and then discards medium and-paste method,of which poison image editing] frequency component and retains the high frequency and healing brush in Adobe Photoshop are well-known component.The major drawback of this approach is: This approach repairs imperfections on the facial im- since spots may exist also in low frequency and high ages by seamless cloning,with little effects on preserv frequency components,it cannot remove spots com- ing skin details.Moreover,for facial images with many pletely.Moreover,the discarded layers are determined imperfections,it would incur a lot of user interactions by several parameters,which are not intuitive,hence to mark out source and destination areas.Since a great requiring heavy user interactions.The users also need deal of skill is demanded to achieve satisfactory results to use unsharp mask to avoid image blurring,which it is a suitable tool only for trained designers.Our goal might magnify image noise. is to develop a technique with a few high level,intuitive EMD.Traditional energy-frequency analysis parameters that can be easily adopted by naive users. are based on Fourier transformation and wavelet558 J. Comput. Sci. & Technol., May 2009, Vol.24, No.3 In the process of facial image cleaning, our algorithm only requests the user to tune a few high level, intuitive parameters to interactively control the amount of en￾hancement. The advantage of our technique is that while effectively removing facial imperfections, it does not blur fine scale facial details. Another important feature of our technique is: it is a general model which characterizes both imperfections and facial details in a unified framework, thereby it does not require user to interactively mark imperfections on facial images. The rest of this paper is structured as follows. Sec￾tion 2 reviews some related work about face cleaning and EMD. Section 3 describes our improved EMD al￾gorithm. The algorithm of face cleaning is introduced in detail in Section 4. Experimental results are demon￾strated in Section 5. Finally, Section 6 concludes the whole paper and highlights future work. 2 Related Work and Background 2.1 Related Work Face Cleaning. The problem of facial image editing has received much attention both from the computer graphics and image processing communities. As a re￾sult, many techniques have been developed to achieve various facial appearance effects. Leyvand et al.[5] pro￾posed an approach to enhanceing the facial attractive￾ness by adjusting facial features. Nguyen et al. [6] pre￾sented a layer extraction method to remove and syn￾thesize beard. Using reflectance transfer, Peers et al. [7] produced face relighting in the post-production process. Most recently, Bitouk[8] introduced an algorithm that can automatically replace faces in photographs. More￾over, the algorithm also allows the user to interactively edit the illumination and colors of face. However, little research work has been published on face cleaning in literature. Nevertheless, in practice, several methods have been employed to clean or en￾hance facial images. While cleaning face with denois￾ing techniques is popular, an alternative approach to serving this problem is the so-called interactive cut￾and-paste method, of which poison image editing[9] and healing brush in Adobe Photoshop are well-known. This approach repairs imperfections on the facial im￾ages by seamless cloning, with little effects on preserv￾ing skin details. Moreover, for facial images with many imperfections, it would incur a lot of user interactions to mark out source and destination areas. Since a great deal of skill is demanded to achieve satisfactory results, it is a suitable tool only for trained designers. Our goal is to develop a technique with a few high level, intuitive parameters that can be easily adopted by naive users. In the field of face recognition, Lin et al.[10] used scale-invariant feature transform (SIFT) framework to detect irregular skin details. Later, Pierrard et al.[11] employed 3D morphable model reconstruction to rec￾ognize facial moles. Their techniques are effective in detecting moles with relatively fixed pattern, for exam￾ple, circular shape. However, as these methods detect and localize facial moles in spatial domain, they are not suitable for detecting and localizing general skin ir￾regularities, due to a large number of possible spatial variations of distractions. Instead of explicitly detect￾ing and localizing distractions, our new approach sug￾gests a quantitative characterization of both pores and distractions, and uses it to enhance facial skin. Among the current image filtering techniques, bilat￾eral filtering[1] is one of the most powerful filters and has been used in various fields. For the typical case in which the spatial and intensity weighting functions are Gaussian, there are two important parameters, namely geometric spread and photometric spread. While small geometric spread corresponds to filtering small intensity changes, large photometric spread would preserve edges with large discontinuity. For face cleaning, of which the aim is to smooth large discontinuity and preserve small scale texture, the settings for the two parameters are difficult: when the photometric spread is set to a very large value in order to smooth imperfections, the bilate￾ral filter nearly degenerates into a Gaussian filter. In such a case, the smoothing effect of bilateral filter only relies on the geometric spread. Apparently, the geo￾metric spread cannot be set to a large value, otherwise the pores of skin will be removed. On the other hand, the edges of bumps or scars in the skin are left aside if it is set to a small value. To the best of our knowledge, the most related work to ours is developed by Matsui et al.[12]. This work also aims at removing spots and at the same time preserv￾ing skin natural roughness. The method uses ε-filters to decompose image into several different frequency com￾ponents. It assumes spots are of medium frequency and pores of high frequency, and then discards medium frequency component and retains the high frequency component. The major drawback of this approach is: since spots may exist also in low frequency and high frequency components, it cannot remove spots com￾pletely. Moreover, the discarded layers are determined by several parameters, which are not intuitive, hence requiring heavy user interactions. The users also need to use unsharp mask to avoid image blurring, which might magnify image noise. EMD. Traditional energy-frequency analysis are based on Fourier transformation and wavelet