Pacific Graphics 2012 Volume 31 (2012),Number 7 C.Bregler,P.Sander,and M.Wimmer (Guest Editors) Improving Photo Composition Elegantly: Considering Image Similarity During Composition Optimization Y.W.Guo!t,M.Liu',T.T.Gu',and W.P.Wang? National Key Lab for Novel Software Technology,Nanjing University 2Department of Computer Science.The University of Hong Kong Abstract Optimization of images with bad compositions has attracted increasing attention in recent years.Previous methods however seldomly consider image similarity when improving composition aesthetics.This may lead to significant content changes or bring large distortions,resulting in an unpleasant user experience.In this paper.we present a new algorithm for improving image composition aesthetics,while retaining faithful,as much as possible,to the original image content.Our method computes an improved image using a unified model of composition aesthetics and image similarity.The term of composition aesthetics obeys the rule of thirds and aims to enhance image composition.The similarity term in contrast penalizes image difference and distortion caused by composition adjustment.We use an edge-based measure of structure similarity which nearly coincides with human visual perception to compare the optimized image with the original one.We describe an effective scheme to generate the optimized image with the objective model.Our algorithm is able to produce the recomposed images with minimal visual distortions in an elegant and user controllable manner.We show the superiority of our algorithm by comparing our results with those by previous methods. Categories and Subject Descriptors (according to ACM CCS):1.3.3 [Computer Graphics]:Picture/Image Generation-Display algorithms 1.Introduction To evaluate composition aesthetics and optimize photo With the continuous performance improvement of digital composition automatically,the pioneering work is given by Bhattacharya et al.[BSS10]which learns a support vector re- cameras,humans can capture high quality photographs with- gression model for capturing aesthetics.Image quality is en- out suffering from the traditional factors such as noises,low hanced by recomposing user selected salient object onto the contrast,and blur that may degrade photo quality,more eas- inpainted background or by using a visual weight balancing ily than before.However,image composition as a crucial as- pect influencing visual aesthetics is often ignored by most technique.Liu et al.[LCWCO10]develop a computational means for evaluating composition aesthetics according to the amateur photographers.Taking a high quality photograph rule of thirds and other visual cues.A compound operator of with a good composition generally needs professional pho- tography knowledge.A simple,yet intuitive guideline is the crop-and-retarget is used to modify the composition and to produce a maximally-aesthetic image. rule of thirds which means that an image should be imaged as divided into nine equal parts by two equally-spaced hor- izontal lines and two equally-spaced vertical lines,and im- When improving photo aesthetics by using automatic portant compositional elements should be placed along these composition optimization techniques,the user may expect lines or their intersections [Pet04]. to maintain consistent visual perception over the resulting image as to the original one he shot.To account for this, the recomposed image should faithfully represent the orig- inal visual appearance as much as possible,rather than just Comresponding author:ywguo@nju.cdu.cn visually pleasing.Previous methods on photo optimization ©2012 The Author(s) Computer Graphies Forum2012 The Eurographics Association and Blackwell Publish ing Ltd.Published by Blackwell Publishing.9600 Garsington Road,Oxford OX4 2DQ. UK and 350 Main Street,Malden,MA 02148,USA.Pacific Graphics 2012 C. Bregler, P. Sander, and M. Wimmer (Guest Editors) Volume 31 (2012), Number 7 Improving Photo Composition Elegantly: Considering Image Similarity During Composition Optimization Y. W. Guo1 †, M. Liu1, T. T. Gu1, and W. P. Wang2 1National Key Lab for Novel Software Technology, Nanjing University 2Department of Computer Science, The University of Hong Kong Abstract Optimization of images with bad compositions has attracted increasing attention in recent years. Previous methods however seldomly consider image similarity when improving composition aesthetics. This may lead to significant content changes or bring large distortions, resulting in an unpleasant user experience. In this paper, we present a new algorithm for improving image composition aesthetics, while retaining faithful, as much as possible, to the original image content. Our method computes an improved image using a unified model of composition aesthetics and image similarity. The term of composition aesthetics obeys the rule of thirds and aims to enhance image composition. The similarity term in contrast penalizes image difference and distortion caused by composition adjustment. We use an edge-based measure of structure similarity which nearly coincides with human visual perception to compare the optimized image with the original one. We describe an effective scheme to generate the optimized image with the objective model. Our algorithm is able to produce the recomposed images with minimal visual distortions in an elegant and user controllable manner. We show the superiority of our algorithm by comparing our results with those by previous methods. Categories and Subject Descriptors (according to ACM CCS): I.3.3 [Computer Graphics]: Picture/Image Generation—Display algorithms 1. Introduction With the continuous performance improvement of digital cameras, humans can capture high quality photographs with￾out suffering from the traditional factors such as noises, low contrast, and blur that may degrade photo quality, more eas￾ily than before. However, image composition as a crucial as￾pect influencing visual aesthetics is often ignored by most amateur photographers. Taking a high quality photograph with a good composition generally needs professional pho￾tography knowledge. A simple, yet intuitive guideline is the rule of thirds which means that an image should be imaged as divided into nine equal parts by two equally-spaced hor￾izontal lines and two equally-spaced vertical lines, and im￾portant compositional elements should be placed along these lines or their intersections [Pet04]. † Corresponding author: ywguo@nju.edu.cn To evaluate composition aesthetics and optimize photo composition automatically, the pioneering work is given by Bhattacharya et al. [BSS10] which learns a support vector re￾gression model for capturing aesthetics. Image quality is en￾hanced by recomposing user selected salient object onto the inpainted background or by using a visual weight balancing technique. Liu et al. [LCWCO10] develop a computational means for evaluating composition aesthetics according to the rule of thirds and other visual cues. A compound operator of crop-and-retarget is used to modify the composition and to produce a maximally-aesthetic image. When improving photo aesthetics by using automatic composition optimization techniques, the user may expect to maintain consistent visual perception over the resulting image as to the original one he shot. To account for this, the recomposed image should faithfully represent the orig￾inal visual appearance as much as possible, rather than just visually pleasing. Previous methods on photo optimization
c 2012 The Author(s) Computer Graphics Forum
c 2012 The Eurographics Association and Blackwell Publish￾ing Ltd. Published by Blackwell Publishing, 9600 Garsington Road, Oxford OX4 2DQ, UK and 350 Main Street, Malden, MA 02148, USA