Table 1:Computational Time http://www.emarketer.com/newsroom/index.php/apparel- drives-retail-ecommerce-sales-growth,2012. Client Time(ms) Person Detection 138 [2]A.C.Gallagher and T.Chen,"Clothing cosegmentation Clothing Segmentation 6040 for recognizing people,"in CVPR 2008.IEEE,2008, Feature Extraction 411 Pp.1-8. Feature Quantization 35 [3]B.Hasan and D.Hogg,"Segmentation using De- Server Time (ms) formable Spatial Priors with Application to Clothing," Search and re-ranking 19 in BMVC,2010,pp.1-11. [4]N.Wang and H.Ai,"Who Blocks Who:Simultaneous truth.We achieve an average F-score over this random sam- Clothing Segmentation for Grouping Images,"in /CCV, ple of 0.857.Since the F-score reaches its best value at 1 and Nov.2011. worst at 0,our approach shows reasonable accuracy.Also, [5]M.Yang and K.Yu,"Real-time clothing recognition in this is favourable considering the baseline(GrabCut only)re- surveillance videos,"in IEEE ICIP,2011,pp.2937- sults in an F-score of 0.740 and with the skin elimination rou- 2940. tine of Chai rather than our own.0.808 is achieved.Addi- tionally,by visual inspection of Figure 3,we can see that our [6]X.Wang and T.Zhang,"Clothes search in consumer approach can segment clothing of persons in various difficult photos via color matching and attribute learning,"in uncontrolled scenes. MM.2011,Pp.1353-1356,ACM. Our retrieval results are reported qualitatively in Figure [7]X.Chao,M.J.Huiskes,T.Gritti,and C.Ciuhu,"A 3.We can see that when the clothing is segmented accurately, framework for robust feature selection for real-time the system appears promising with relevant clothing results of fashion style recommendation,"in Workshop on IMCE. a similar colour and shape retrieved.The clothing segmenta- ACM.2009. tion stage is important since if it is inaccurate,errors are prop- agated forward to rest of the system.Segmentation inaccura- [8]K.Yamaguchi,M.H.Kiapour,L.E.Ortiz,and T.L. cies appear to generally be caused by inherent issues such as Berg."Parsing clothing in fashion photographs,"in when scenes contain a garment that is a very similar colour to CVPR.IEEE.2012 the background or skin,or there is poor illumination present, [9]H.Chen,A.Gallagher,and B.Girod,"Describing or excessive long hair covering the clothing.However,when Clothing by Semantic Attributes,"in ECCV.2012, clothing segmentation fails and our algorithm decides to in- Springer. stead use the unsegmented image to establish features,such as in Figure 3g.we see that the results can still be reasonably [10]J.Sivic,C.L.Zitnick,and R.Szeliski,"Finding people relevant although may not be the most accurate. in repeated shots of the same scene,"in BMVC,2006, vol.3,pp.909-918. 7.CONCLUSIONS [11]M.W.Lee and I.Cohen,"A model-based approach for estimating human 3D poses in static images,"IEEE In this paper,we present a novel mobile client-server frame- TPAMI,Pp.905-916,2006. work for automatic visual clothes searching.Our system em- [12]G.A.Cushen and M.S.Nixon,"Real-Time Semantic ploys a Bag of Words(BoW)model and proposes an exten- Clothing Segmentation,"in ISVC.2012,pp.272-281, sion of GrabCut for clothing segmentation and a colour de- Springer. scriptor optimized for clothing.We demonstrate a novel ap- plication of combining a photo captured on a smart phone [13]R.Carsten,K.Vladimir,and B.Andrew,"GrabCut: (or from social networking)with GPS data to locate cloth- interactive foreground extraction using iterated graph ing of interest at nearby retailers.For future work,we aim cuts,"ACM Trans.Graph.,vol.23,no.3,pp.309-314. to perform a more comprehensive evaluation and integrate Aug.2004. more features to train clothing classifiers and re-rank domi- nant colour results by predicted clothing labels. [14]D.Chai and K.N.Ngan,"Face segmentation using skin- color map in videophone applications,"CSVT.IEEE Trans on,vol.9,no.4,pp.551-564,1999. 8.REFERENCES [15]T.Sikora,"The MPEG-7 visual standard for content description-an overview,"CSVT,IEEE Trans on,vol. [1]eMarketer, “Apparel Drives US 11,no.6.pp.696-702,2001. Retail Ecommerce Sales Growth."Table 1: Computational Time Client Time (ms) Person Detection 138 Clothing Segmentation 6040 Feature Extraction 411 Feature Quantization 35 Server Time (ms) Search and re-ranking 19 truth. We achieve an average F-score over this random sam￾ple of 0.857. Since the F-score reaches its best value at 1 and worst at 0, our approach shows reasonable accuracy. Also, this is favourable considering the baseline (GrabCut only) re￾sults in an F-score of 0.740 and with the skin elimination rou￾tine of Chai rather than our own, 0.808 is achieved. Addi￾tionally, by visual inspection of Figure 3, we can see that our approach can segment clothing of persons in various difficult uncontrolled scenes. Our retrieval results are reported qualitatively in Figure 3. We can see that when the clothing is segmented accurately, the system appears promising with relevant clothing results of a similar colour and shape retrieved. The clothing segmenta￾tion stage is important since if it is inaccurate, errors are prop￾agated forward to rest of the system. Segmentation inaccura￾cies appear to generally be caused by inherent issues such as when scenes contain a garment that is a very similar colour to the background or skin, or there is poor illumination present, or excessive long hair covering the clothing. However, when clothing segmentation fails and our algorithm decides to in￾stead use the unsegmented image to establish features, such as in Figure 3q, we see that the results can still be reasonably relevant although may not be the most accurate. 7. CONCLUSIONS In this paper, we present a novel mobile client-server frame￾work for automatic visual clothes searching. Our system em￾ploys a Bag of Words (BoW) model and proposes an exten￾sion of GrabCut for clothing segmentation and a colour de￾scriptor optimized for clothing. We demonstrate a novel ap￾plication of combining a photo captured on a smart phone (or from social networking) with GPS data to locate cloth￾ing of interest at nearby retailers. For future work, we aim to perform a more comprehensive evaluation and integrate more features to train clothing classifiers and re-rank domi￾nant colour results by predicted clothing labels. 8. REFERENCES [1] eMarketer, “Apparel Drives US Retail Ecommerce Sales Growth,” http://www.emarketer.com/newsroom/index.php/apparel￾drives-retail-ecommerce-sales-growth, 2012. [2] A. C. Gallagher and T. Chen, “Clothing cosegmentation for recognizing people,” in CVPR 2008. IEEE, 2008, pp. 1–8. [3] B. Hasan and D. Hogg, “Segmentation using De￾formable Spatial Priors with Application to Clothing,” in BMVC, 2010, pp. 1–11. [4] N. Wang and H. Ai, “Who Blocks Who: Simultaneous Clothing Segmentation for Grouping Images,” in ICCV, Nov. 2011. [5] M. Yang and K. Yu, “Real-time clothing recognition in surveillance videos,” in IEEE ICIP, 2011, pp. 2937– 2940. [6] X. Wang and T. Zhang, “Clothes search in consumer photos via color matching and attribute learning,” in MM. 2011, pp. 1353–1356, ACM. [7] X. Chao, M. J. Huiskes, T. Gritti, and C. Ciuhu, “A framework for robust feature selection for real-time fashion style recommendation,” in Workshop on IMCE. ACM, 2009. [8] K. Yamaguchi, M. H. Kiapour, L. E. Ortiz, and T. L. Berg, “Parsing clothing in fashion photographs,” in CVPR. IEEE, 2012. [9] H. Chen, A. Gallagher, and B. Girod, “Describing Clothing by Semantic Attributes,” in ECCV. 2012, Springer. [10] J. Sivic, C. L. Zitnick, and R. Szeliski, “Finding people in repeated shots of the same scene,” in BMVC, 2006, vol. 3, pp. 909–918. [11] M. W. Lee and I. Cohen, “A model-based approach for estimating human 3D poses in static images,” IEEE TPAMI, pp. 905–916, 2006. [12] G. A. Cushen and M. S. Nixon, “Real-Time Semantic Clothing Segmentation,” in ISVC. 2012, pp. 272–281, Springer. [13] R. Carsten, K. Vladimir, and B. Andrew, “GrabCut: interactive foreground extraction using iterated graph cuts,” ACM Trans. Graph., vol. 23, no. 3, pp. 309–314, Aug. 2004. [14] D. Chai and K. N. Ngan, “Face segmentation using skin￾color map in videophone applications,” CSVT, IEEE Trans on, vol. 9, no. 4, pp. 551–564, 1999. [15] T. Sikora, “The MPEG-7 visual standard for content description-an overview,” CSVT, IEEE Trans on, vol. 11, no. 6, pp. 696–702, 2001