IEEE TRANSACTIONS ON IMAGE PROCESSING.VOL.XX,NO.X.XXX 2019 12 [22]J.Wang,T.Zhang,J.Song,N.Sebe,and H.T.Shen,"A survey on learning to hash,"TPAMI,vol.40.no.4,pp.769-790,2018. [23]X.Lu,X.Zheng,and X.Li."Latent semantic minimal hashing for image retrieval,"TIP,vol.26,no.1,pp.355-368.2017 [24]F.Shen,C.Shen,W.Liu,and H.T.Shen,"Supervised discrete hashing," 4 48132 84 48182 in CVPR,2015,Pp.37-45. (25]F.Shen.X.Zhou,Y.Yang,J.Song,H.T.Shen.and D.Tao,"A fast (a)IAPR-TC12. (b)MIRFLICKR-25K. (c)NUS-WIDE optimization method for general binary code learning."TIP,vol.25. Fig.10.MAP values with different number of sampled points m on three no.12,Pp.5610-5621,2016. datasets. [26]D.Zhang and W.-J.Li,"Large-scale supervised multimodal hashing with semantic correlation maximization,"in AAA/,2014,pp.2177-2183. [27]G.Ding,Y.Guo,and J.Zhou,"Collective matrix factorization hashing VI.CONCLUSION for multimodal data,"in CVPR.2014,pp.2083-2090. In this paper,we propose a novel cross-modal hashing (28]X.Liu,L.Huang.C.Deng.B.Lang,and D.Tao,"Query-adaptive hash code ranking for large-scale multi-view visual search."TIP.vol.25 method,called discrete latent factor model based cross- n0.10,Pp.4514-4524,2016. modal hashing (DLFH),for cross-modal similarity search in [29]J.Song.Y.Yang.Z.Huang.H.T.Shen,and R.Hong,"Multiple feature large-scale datasets.DLFH is a discrete method which can hashing for real-time large scale near-duplicate video retrieval,"in MM. ACM2011,pp.423-432 directly learn the binary hash codes,and at the same time [30]D.Zhang.F.Wang.and L.Si,"Composite hashing with multiple it is efficient.Experiments show that DLFH can significantly information sources,"in S/GIR,2011,pp.225-234. outperform relaxation-based continuous methods in terms of 31]Y.Kang.S.Kim,and S.Choi."Deep learning to hash with multiple representations,"in ICDM.2012.pp.930-935. accuracy,but with a comparable training speed.Furthermore, [32]Y.Zhen and D.Yeung,"Co-regularized hashing for multimodal data," DLFH can significantly outperform existing discrete methods inW1PS,2012,pp.1385-1393. in terms of both accuracy and training speed. [33]B.Dai,R.Guo,S.Kumar,N.He,and L.Song."Stochastic generative hashing,"in /CML,2017,pp.913-922. [34]Q.-Y.Jiang and W.-J.Li,"Deep cross-modal hashing,"in CVPR,2017, REFERENCES pp.3232-3240. [35]G.Irie,H.Arai,and Y.Taniguchi,"Alternating co-quantization for cross- [1]M.Datar,N.Immorlica,P.Indyk,and V.S.Mirrokni,"Locality-sensitive modal hashing,"in /CCV,2015,pp.1886-1894. hashing scheme based on p-stable distributions,"in SCG,2004,pp.253- [36]J.Zhang.Y.Peng,and M.Yuan."Unsupervised generative adversarial 262. cross-modal hashing."in AAAl.2018.pp.539-546. [2]A.Andoni and P.Indyk."Near-optimal hashing algorithms for approxi- [37]Y.Zhen and D.Yeung."A probabilistic model for multimodal hash mate nearest neighbor in high dimensions,"in FOCS,2006,pp.459-468. function learning."in KDD,2012,pp.940-948. [3]A.Andoni and I.P.Razenshteyn,"Optimal data-dependent hashing for [38]Z.Lin,G.Ding,M.Hu,and J.Wang,"Semantics-preserving hashing approximate near neighbors,"in STOC,2015,pp.793-801 for cross-view retrieval,"in CVPR,2015,pp.3864-3872. [4]B.Kulis and K.Grauman,"Kernelized locality-sensitive hashing for 39]H.Liu,R.Ji,Y.Wu,and G.Hua,"Supervised matrix factorization for scalable image search,"in /CCV.2009.pp.2130-2137 cross-modality hashing,"in I/CAl,2016,pp.1767-1773. [5]J.Wang,O.Kumar,and S.Chang,"Semi-supervised hashing for scalable [40]M.Yu,L.Liu,and L.Shao,"Binary set embedding for cross-modal image retrieval,"in CVPR,2010,pp.3424-3431 retrieval,"TNNLS,2016. [6]Y.Gong and S.Lazebnik,"Iterative quantization:A procrustean ap- [41]Y.Cao,B.Liu,M.Long,and J.Wang,"Cross-modal hamming hashing." proach to learning binary codes,"in CVPR,2011,pp.817-824. in ECCV.2018.pp.207-223 [7]W.Liu,J.Wang.R.Ji,Y.Jiang.and S.Chang. “Supervised hashing (42]D.Mandal,K.N.Chaudhury,and S.Biswas,"Generalized semantic with kernels."in CVPR.2012.pp.2074-2081. preserving hashing for n-label cross-modal retrieval."in CVPR.2017. [8]G.Lin,C.Shen,Q.Shi,A.van den Hengel,and D.Suter,"Fast supervised hashing with decision trees for high-dimensional data."in PP.4076-4084. (43]S.Kumar and R.Udupa,"Learning hash functions for cross-view CVPR,2014,Pp.1971-1978. similarity search,"in I/CAl.2011,pp.1360-1365. [9]C.Da,S.Xu,K.Ding,G.Meng,S.Xiang,and C.Pan,"Amvh: [44]M.Rastegari,J.Choi,S.Fakhraei,H.D.III.and L.S.Davis."Pre- Asymmetric multi-valued hashing."in CVPR,2017,pp.736-744. dictable dual-view hashing,"in ICML,2013,pp.1328-1336. [10]L.Liu,L.Shao,F.Shen,and M.Yu,"Discretely coding semantic rank (45]R.Raziperchikolaei and M.A.Carreira-Perpinan,"Learning hashing orders for supervised image hashing,"in CVPR.2017,pp.1425-1434. with affinity-based loss functions using auxiliary coordinates,"in N/PS, [11]Z.Hu.J.Chen,H.Lu,and T.Zhang,"Bayesian supervised hashing," 2016. in CVPR,2017,pp.6348-6355. [46]Y.Gong.S.Lazebnik.A.Gordo,and F.Perronnin,"Iterative quantiza- [12]Y.Weiss,A.Torralba,and R.Fergus,"Spectral hashing,"in N/PS,2008 tion:A procrustean approach to learning binary codes for large-scale Pp.1753-1760. image retrieval,"TPAMI,vol.35,no.12,pp.2916-2929,2013. [13]W.Liu,J.Wang.S.Kumar,and S.Chang."Hashing with graphs,"in (47]X.He,D.Cai,S.Yan,and H.Zhang,"Neighborhood preserving 1CML.2011,pp.1-8. embedding."in /CCV,2005.pp.1208-1213. [14]R.Xia,Y.Pan.H.Lai,C.Liu,and S.Yan,"Supervised hashing for [48]K.Lange,D.R.Hunter,and I.Yang."Optimization transfer using image retrieval via image representation learning,"in AAAl.2014,pp surrogate objective functions,"/CGS,vol.9,no.1,pp.1-20,2000. 2156-2162. [49]D.Bohning and B.G.Lindsay,"Monotonicity of quadratic. [15]D.Wang,P.Cui,M.Ou,and W.Zhu,"Deep multimodal hashing with approximation algorithms,"A/SM.vol.40.no.4.pp.641-663.1988. orthogonal regularization."in I/CAl.2015.pp.2291-2297. [50]P.Zhang.W.Zhang.W.-J.Li,and M.Guo."Supervised hashing with [16]Y.Cao,M.Long,J.Wang,Q.Yang,and P.S.Yu,"Deep visual-semantic latent factor models,"in S/G/R,2014,pp.173-182. hashing for cross-modal retrieval,"in KDD,2016,pp.1445-1454. 51]H.J.Escalante,C.A.Hernandez,J.A.Gonzalez.A.Lopez-Lopez. [17]Y.Cao.M.Long.J.Wang.and S.Liu,"Collective deep quantization M.Montes-y-Gomez,E.F.Morales,L.E.Sucar,L.V.Pineda,and for efficient cross-modal retrieval,"in AAA/,2017,pp.3974-3980. M.Grubinger,"The segmented and annotated IAPR TC-12 benchmark," [18]D.Tian and D.Tao,"Global hashing system for fast image search,"TIP. CVIU,vol.114.no.4,pp.419-428.2010. vol.26,no.1,pp.79-89,2017. [52]M.J.Huiskes and M.S.Lew,"The MIR flickr retrieval evaluation,"in [19]J.Wang,H.T.Shen,J.Song,and J.Ji,"Hashing for similarity search: ACM SIGMM,2008.pp.39-43. A survey,"CoRR,vol.abs/1408.2927,2014. (53]T.Chua,J.Tang.R.Hong,H.Li,Z.Luo,and Y.Zheng."NUS-WIDE: [20]J.Wang.W.Liu,S.Kumar,and S.Chang,"Learning to hash for indexing a real-world web image database from national university of singapore," big data-A survey,"Proceedings of the IEEE.vol.104.no.1.pp.34-57 in C/VR,2009. 2016. [21]Y.Guo,G.Ding.L.Liu,J.Han,and L.Shao,"Learning to hash with optimized anchor embedding for scalable retrieval,"TIP,vol.26,no.3 Pp.1344-1354.2017.IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. XX, NO. X, XXX 2019 12 48 16 32 64 96 M A P 0.35 0.4 0.45 0.5 0.55 0.6 (a) IAPR-TC12. 48 16 32 64 96 M A P 0.65 0.7 0.75 0.8 0.85 (b) MIRFLICKR-25K. 48 16 32 64 96 M A P 0.6 0.65 0.7 0.75 0.8 (c) NUS-WIDE. Fig. 10. MAP values with different number of sampled points m on three datasets. VI. CONCLUSION In this paper, we propose a novel cross-modal hashing method, called discrete latent factor model based cross￾modal hashing (DLFH), for cross-modal similarity search in large-scale datasets. DLFH is a discrete method which can directly learn the binary hash codes, and at the same time it is efficient. Experiments show that DLFH can significantly outperform relaxation-based continuous methods in terms of accuracy, but with a comparable training speed. Furthermore, DLFH can significantly outperform existing discrete methods in terms of both accuracy and training speed. REFERENCES [1] M. Datar, N. Immorlica, P. Indyk, and V. S. Mirrokni, “Locality-sensitive hashing scheme based on p-stable distributions,” in SCG, 2004, pp. 253– 262. [2] A. Andoni and P. Indyk, “Near-optimal hashing algorithms for approxi￾mate nearest neighbor in high dimensions,” in FOCS, 2006, pp. 459–468. [3] A. Andoni and I. P. Razenshteyn, “Optimal data-dependent hashing for approximate near neighbors,” in STOC, 2015, pp. 793–801. [4] B. Kulis and K. Grauman, “Kernelized locality-sensitive hashing for scalable image search,” in ICCV, 2009, pp. 2130–2137. [5] J. Wang, O. Kumar, and S. Chang, “Semi-supervised hashing for scalable image retrieval,” in CVPR, 2010, pp. 3424–3431. [6] Y. Gong and S. Lazebnik, “Iterative quantization: A procrustean ap￾proach to learning binary codes,” in CVPR, 2011, pp. 817–824. [7] W. Liu, J. Wang, R. Ji, Y. Jiang, and S. Chang, “Supervised hashing with kernels,” in CVPR, 2012, pp. 2074–2081. [8] G. Lin, C. Shen, Q. Shi, A. van den Hengel, and D. Suter, “Fast supervised hashing with decision trees for high-dimensional data,” in CVPR, 2014, pp. 1971–1978. [9] C. Da, S. Xu, K. Ding, G. Meng, S. Xiang, and C. Pan, “Amvh: Asymmetric multi-valued hashing,” in CVPR, 2017, pp. 736–744. [10] L. Liu, L. Shao, F. Shen, and M. Yu, “Discretely coding semantic rank orders for supervised image hashing,” in CVPR, 2017, pp. 1425–1434. [11] Z. Hu, J. Chen, H. Lu, and T. Zhang, “Bayesian supervised hashing,” in CVPR, 2017, pp. 6348–6355. [12] Y. Weiss, A. Torralba, and R. Fergus, “Spectral hashing,” in NIPS, 2008, pp. 1753–1760. [13] W. Liu, J. Wang, S. Kumar, and S. Chang, “Hashing with graphs,” in ICML, 2011, pp. 1–8. [14] R. Xia, Y. Pan, H. Lai, C. Liu, and S. Yan, “Supervised hashing for image retrieval via image representation learning,” in AAAI, 2014, pp. 2156–2162. [15] D. Wang, P. Cui, M. Ou, and W. Zhu, “Deep multimodal hashing with orthogonal regularization,” in IJCAI, 2015, pp. 2291–2297. [16] Y. Cao, M. Long, J. Wang, Q. Yang, and P. S. Yu, “Deep visual-semantic hashing for cross-modal retrieval,” in KDD, 2016, pp. 1445–1454. [17] Y. Cao, M. Long, J. Wang, and S. Liu, “Collective deep quantization for efficient cross-modal retrieval,” in AAAI, 2017, pp. 3974–3980. [18] D. Tian and D. Tao, “Global hashing system for fast image search,” TIP, vol. 26, no. 1, pp. 79–89, 2017. [19] J. Wang, H. T. Shen, J. Song, and J. Ji, “Hashing for similarity search: A survey,” CoRR, vol. abs/1408.2927, 2014. [20] J. Wang, W. Liu, S. Kumar, and S. Chang, “Learning to hash for indexing big data - A survey,” Proceedings of the IEEE, vol. 104, no. 1, pp. 34–57, 2016. [21] Y. Guo, G. Ding, L. Liu, J. Han, and L. Shao, “Learning to hash with optimized anchor embedding for scalable retrieval,” TIP, vol. 26, no. 3, pp. 1344–1354, 2017. [22] J. Wang, T. Zhang, J. Song, N. Sebe, and H. T. Shen, “A survey on learning to hash,” TPAMI, vol. 40, no. 4, pp. 769–790, 2018. [23] X. Lu, X. Zheng, and X. Li, “Latent semantic minimal hashing for image retrieval,” TIP, vol. 26, no. 1, pp. 355–368, 2017. [24] F. Shen, C. Shen, W. Liu, and H. T. Shen, “Supervised discrete hashing,” in CVPR, 2015, pp. 37–45. [25] F. Shen, X. Zhou, Y. Yang, J. Song, H. T. Shen, and D. Tao, “A fast optimization method for general binary code learning,” TIP, vol. 25, no. 12, pp. 5610–5621, 2016. [26] D. Zhang and W.-J. Li, “Large-scale supervised multimodal hashing with semantic correlation maximization,” in AAAI, 2014, pp. 2177–2183. [27] G. Ding, Y. Guo, and J. Zhou, “Collective matrix factorization hashing for multimodal data,” in CVPR, 2014, pp. 2083–2090. [28] X. Liu, L. Huang, C. Deng, B. Lang, and D. Tao, “Query-adaptive hash code ranking for large-scale multi-view visual search,” TIP, vol. 25, no. 10, pp. 4514–4524, 2016. [29] J. Song, Y. Yang, Z. Huang, H. T. Shen, and R. Hong, “Multiple feature hashing for real-time large scale near-duplicate video retrieval,” in MM. ACM, 2011, pp. 423–432. [30] D. Zhang, F. Wang, and L. Si, “Composite hashing with multiple information sources,” in SIGIR, 2011, pp. 225–234. [31] Y. Kang, S. Kim, and S. Choi, “Deep learning to hash with multiple representations,” in ICDM, 2012, pp. 930–935. [32] Y. Zhen and D. Yeung, “Co-regularized hashing for multimodal data,” in NIPS, 2012, pp. 1385–1393. [33] B. Dai, R. Guo, S. Kumar, N. He, and L. Song, “Stochastic generative hashing,” in ICML, 2017, pp. 913–922. [34] Q.-Y. Jiang and W.-J. Li, “Deep cross-modal hashing,” in CVPR, 2017, pp. 3232–3240. [35] G. Irie, H. Arai, and Y. Taniguchi, “Alternating co-quantization for cross￾modal hashing,” in ICCV, 2015, pp. 1886–1894. [36] J. Zhang, Y. Peng, and M. Yuan, “Unsupervised generative adversarial cross-modal hashing,” in AAAI, 2018, pp. 539–546. [37] Y. Zhen and D. Yeung, “A probabilistic model for multimodal hash function learning,” in KDD, 2012, pp. 940–948. [38] Z. Lin, G. Ding, M. Hu, and J. Wang, “Semantics-preserving hashing for cross-view retrieval,” in CVPR, 2015, pp. 3864–3872. [39] H. Liu, R. Ji, Y. Wu, and G. Hua, “Supervised matrix factorization for cross-modality hashing,” in IJCAI, 2016, pp. 1767–1773. [40] M. Yu, L. Liu, and L. Shao, “Binary set embedding for cross-modal retrieval,” TNNLS, 2016. [41] Y. Cao, B. Liu, M. Long, and J. Wang, “Cross-modal hamming hashing,” in ECCV, 2018, pp. 207–223. [42] D. Mandal, K. N. Chaudhury, and S. Biswas, “Generalized semantic preserving hashing for n-label cross-modal retrieval,” in CVPR, 2017, pp. 4076–4084. [43] S. Kumar and R. Udupa, “Learning hash functions for cross-view similarity search,” in IJCAI, 2011, pp. 1360–1365. [44] M. Rastegari, J. Choi, S. Fakhraei, H. D. III, and L. S. Davis, “Pre￾dictable dual-view hashing,” in ICML, 2013, pp. 1328–1336. [45] R. Raziperchikolaei and M. A. Carreira-Perpi ´ n˜an, “Learning hashing ´ with affinity-based loss functions using auxiliary coordinates,” in NIPS, 2016. [46] Y. Gong, S. Lazebnik, A. Gordo, and F. Perronnin, “Iterative quantiza￾tion: A procrustean approach to learning binary codes for large-scale image retrieval,” TPAMI, vol. 35, no. 12, pp. 2916–2929, 2013. [47] X. He, D. Cai, S. Yan, and H. Zhang, “Neighborhood preserving embedding,” in ICCV, 2005, pp. 1208–1213. [48] K. Lange, D. R. Hunter, and I. Yang, “Optimization transfer using surrogate objective functions,” JCGS, vol. 9, no. 1, pp. 1–20, 2000. [49] D. Bohning and B. G. Lindsay, “Monotonicity of quadratic- ¨ approximation algorithms,” AISM, vol. 40, no. 4, pp. 641–663, 1988. [50] P. Zhang, W. Zhang, W.-J. Li, and M. Guo, “Supervised hashing with latent factor models,” in SIGIR, 2014, pp. 173–182. [51] H. J. Escalante, C. A. Hernandez, J. A. Gonz ´ alez, A. L ´ opez-L ´ opez, ´ M. Montes-y-Gomez, E. F. Morales, L. E. Sucar, L. V. Pineda, and ´ M. Grubinger, “The segmented and annotated IAPR TC-12 benchmark,” CVIU, vol. 114, no. 4, pp. 419–428, 2010. [52] M. J. Huiskes and M. S. Lew, “The MIR flickr retrieval evaluation,” in ACM SIGMM, 2008, pp. 39–43. [53] T. Chua, J. Tang, R. Hong, H. Li, Z. Luo, and Y. Zheng, “NUS-WIDE: a real-world web image database from national university of singapore,” in CIVR, 2009