Skip to main content
SpringerLink
Log in

Flexible Discrete Multi-view Hashing with Collective Latent Feature Learning

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Multi-view hashing has gained considerable research attention in efficient multimedia studies due to its promising performance on heterogeneous data from various sources. However, its application in discriminative hash codes learning remains challenging as it fails to efficiently capture preferable components from multiple representations. In this work, we propose a novel discriminative multi-view hashing framework, dubbed flexible discrete multi-view hashing, in conjunction with collective latent feature learning by combining multiple views of data and consistent hash codes learning by fusing visual features and flexible semantics. Specifically, an adaptive multi-view analysis dictionary learning model is developed to skillfully combine diverse representations into an established common latent feature space where the complementary properties of different views are well explored based on an automatic multi-view weighting strategy. Moreover, we introduce a collaborative learning scheme to jointly encode the visual and semantic embeddings into an aligned consistent Hamming space, which can effectively mitigate the visual-semantic gap. Particularly, we employ the correntropy induced regularization to improve the robustness of the formulated flexible semantics. An efficient learning algorithm is proposed to solve the optimization problem. Extensive experiments show the state-of-art performance of the proposed method on several benchmark datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Notes

  1. The codes of this work has been released at https://github.com/DarrenZZhang/FDMH.

  2. https://www.wikipedia.org/.

  3. http://www.cs.toronto.edu/kriz/cifar.html.

  4. http://lear.inrialpes.fr/people/guillaumin/data.php.

  5. https://www.flickr.com/.

References

  1. Aharon M, Elad M, Bruckstein A (2006) K-svd: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans Signal Process 54(11):4311–4322

    Article  Google Scholar 

  2. Ahonen T, Hadid A, Pietikäinen M (2004) Face recognition with local binary patterns. In: European conference on computer vision, pp 469–481

  3. Blei DM, Ng AY, Jordan MI (2003) Latent dirichlet allocation. J. Mach. Learn. Res. 3:993–1022

    MATH  Google Scholar 

  4. Chen H, Wang Y (2018) Kernel-based sparse regression with the correntropy-induced loss. Appl Comput Harmon Anal 44(1):144–164

    Article  MathSciNet  Google Scholar 

  5. Csurka G, Dance CR, Fan L, Willamowski J, Bray C (2004) Visual categorization with bags of keypoints. In: In ECCV workshop on statistical learning in computer vision, pp 1–22

  6. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: IEEE computer society conference on computer vision and pattern recognition, pp 886–893

  7. Gionis A, Indyk P, Motwani, R (1999) Similarity search in high dimensions via hashing. In: the 25th international conference on very large data bases, pp 518–529

  8. Gong Y, Lazebnik S, Gordo A, Perronnin F (2013) Iterative quantization: a procrustean approach to learning binary codes for large-scale image retrieval. IEEE Trans Pattern Anal Mach Intell 35(12):2916–2929

    Article  Google Scholar 

  9. Gui J, Liu T, Sun Z, Tao D, Tan T (2018) Fast supervised discrete hashing. IEEE Trans Pattern Anal Mach Intell 40(2):490–496

    Article  Google Scholar 

  10. Guillaumin M, Verbeek J, Schmid C (2010) Multiple instance metric learning from automatically labeled bags of faces. Eur Conf Comput Vis 6311:634–647

    Google Scholar 

  11. Guo J, Guo Y, Kong X, Zhang M, He R (2016) Discriminative analysis dictionary learning. In: The thirtieth AAAI conference on artificial intelligence, pp 1617–1623

  12. He R, Zheng WS, Hu BG (2010) Maximum correntropy criterion for robust face recognition. IEEE Trans Pattern Anal Mach Intell 33(8):1561–1576

    Google Scholar 

  13. Hu R, Zhu X, Zhu Y, Gan J (2019) Robust SVM with adaptive graph learning. World Wide Web. https://doi.org/10.1007/s11280-019-00766-x

  14. Huiskes MJ, Lew MS (2008) The MIR flickr retrieval evaluation. In: ACM international conference on multimedia information retrieval, pp 39–43

  15. Jiang Q, Li W (2015) Scalable graph hashing with feature transformation. In: The twenty-fourth international joint conference on artificial intelligence, pp 2248–2254

  16. Kim S, Choi S (2013) Multi-view anchor graph hashing. In: IEEE international conference on acoustics, speech and signal processing, pp 3123–3127

  17. Krizhevsky A (2012) Learning multiple layers of features from tiny images. University of Toronto

  18. Li J, Zhang B, Lu G, Zhang D (2019) Dual asymmetric deep hashing learning. IEEE Access

  19. Li Z, Zhang Z, Qin j, Zhang Z, Shao L (2019) Discriminative fisher embedding dictionary learning algorithm for object recognition. IEEE Trans Neural Netw Learn Syst. https://doi.org/10.1109/TNNLS.2019.2910146

  20. Liu L, Du X, Zhu L, Shen F, Huang Z (2018) Discrete binary hashing towards efficient fashion recommendation. In: Database systems for advanced applications—23rd international conference, DASFAA 2018, Gold Coast, QLD, Australia, May 21–24, 2018, Proceedings, Part I, pp 116–132

  21. Liu L, Du X, Zhu L, Shen F, Huang Z (2018) Learning discrete hashing towards efficient fashion recommendation. Data Sci Eng 3(4):307–322

    Article  Google Scholar 

  22. Liu L, Yu M, Shao L (2015) Multiview alignment hashing for efficient image search. IEEE Trans Image Process 24(3):956–966

    Article  MathSciNet  Google Scholar 

  23. Liu L, Zhu L, Li Z (2017) Learning robust graph hashing for efficient similarity search. In: Databases theory and applications—28th Australasian database conference, ADC 2017, Brisbane, QLD, Australia, September 25–28, 2017, Proceedings, pp 110–122

  24. Liu W, Pokharel PP, Príncipe JC (2007) Correntropy: properties and applications in non-gaussian signal processing. IEEE Trans Signal Process 55(11):5286–5298

    Article  MathSciNet  Google Scholar 

  25. Liu W, Wang J, Kumar S, Chang S (2011) Hashing with graphs. In: the 28th international conference on machine learning, pp 1–8

  26. Liu X, He J, Liu D, Lang B (2012) Compact kernel hashing with multiple features. In: The 20th ACM conference on multimedia, pp 881–884

  27. Lu X, Zhu L, Cheng Z, Nie L, Zhang H (2019) Online multi-modal hashing with dynamic query-adaption. In: The 42nd international ACM SIGIR conference on research and development in information retrieval, pp 715–724

  28. Manning C, Raghavan P, Schütze H (2010) Introduction to information retrieval. Nat Lang Eng 16(1):100–103

    Article  Google Scholar 

  29. Oliva A, Torralba A (2001) Modeling the shape of the scene: a holistic representation of the spatial envelope. Int J Comput Vis 42(3):145–175

    Article  Google Scholar 

  30. Pokharel R, Príncipe JC (2012) Kernel classifier with correntropy loss. In: The 2012 international joint conference on neural networks (IJCNN), Brisbane, Australia, June 10–15, 2012, pp 1–6

  31. Rasiwasia N, Pereira JC, Coviello E, Doyle G, Lanckriet GRG, Levy R, Vasconcelos N (2010) A new approach to cross-modal multimedia retrieval. In: The 18th international conference on multimedia retrieval, pp 251–260

  32. Shen F, Mu Y, Yang Y, Liu W, Liu L, Song J, Shen HT (2017) Classification by retrieval: Binarizing data and classifiers. In: Proceedings of the 40th international ACM SIGIR conference on research and development in information retrieval, pp 595–604. ACM

  33. Shen F, Shen C, Liu W, Shen HT (2015) Supervised discrete hashing. In: IEEE conference on computer vision and pattern recognition, pp. 37–45

  34. Shen F, Zhou X, Yu J, Yang Y, Liu L, Shen HT (2019) Scalable zero-shot learning via binary visual-semantic embeddings. IEEE Trans Image Process 28(7):3662–3674

    Article  MathSciNet  Google Scholar 

  35. Shen X, Shen F, Liu L, Yuan Y, Liu W, Sun Q (2018) Multiview discrete hashing for scalable multimedia search. ACM Trans Intell Syst Technol 9(5):53:1–53:21

    Article  Google Scholar 

  36. Shen XB, Shen F, Sun Q, Yuan Y (2015) Multi-view latent hashing for efficient multimedia search. In: The 23rd ACM conference on multimedia, pp 831–834

  37. Song J, Yang Y, Huang Z, Shen HT, Luo J (2013) Effective multiple feature hashing for large-scale near-duplicate video retrieval. IEEE Trans Multimed 15(8):1997–2008

    Article  Google Scholar 

  38. Wang H, Nie F, Huang H (2013) Multi-view clustering and feature learning via structured sparsity. In: The 30th international conference on machine learning, pp 352–360

  39. Wang J, Zhang T, Song J, Sebe N, Shen HT (2018) A survey on learning to hash. IEEE Trans Pattern Anal Mach Intell 40(4):769–790

    Article  Google Scholar 

  40. Wang Z, Zhang Z, Luo Y, Huang Z (2019) Deep collaborative discrete hashing with semantic-invariant structure. In: The 42nd international ACM SIGIR conference on research and development in information retrieval, pp 905–908

  41. Weiss Y, Torralba A, Fergus R (2008) Spectral hashing. In: Advances in neural information processing systems, pp 1753–1760

  42. Yang R, Shi Y, Xu XS (2017) Discrete multi-view hashing for effective image retrieval. In: Proceedings of the 2017 ACM on international conference on multimedia retrieval, pp. 175–183

  43. Zhang D, Wang F, Si L (2011) Composite hashing with multiple information sources. In: The 34th international ACM SIGIR conference on research and development in information retrieval, pp 225–234

  44. Zhang P, Zhang W, Li W, Guo M (2014) Supervised hashing with latent factor models. In: The 37th international ACM SIGIR conference on research and development in information retrieval, pp 173–182

  45. Zhang Z, Lai Z, Huang Z, Wong WK, Xie GS, Liu L, Shao L (2019) Scalable supervised asymmetric hashing with semantic and latent factor embedding. IEEE Trans Image Process 28(10):4803–4818

    Article  MathSciNet  Google Scholar 

  46. Zhang Z, Liu L, Qin J, Zhu F, Shen F, Xu Y, Shao L, Shen HT (2018) Highly-economized multi-view binary compression for scalable image clustering. In: European conference on computer vision, pp 717–732

  47. Zhang Z, Liu L, Shen F, Shen HT, Shao L (2019) Binary multi-view clustering. IEEE Trans Pattern Anal Mach Intell 41(7):1774–1782

    Article  Google Scholar 

  48. Zheng C, Zhu L, Lu X, Li J, Cheng Z, Zhang H (2019) Fast discrete collaborative multi-modal hashing for large-scale multimedia retrieval. IEEE Trans Knowl Data Eng. https://doi.org/10.1109/TKDE.2019.2913388

  49. Zhu L, Huang Z, Li Z, Xie L, Shen HT (2018) Exploring auxiliary context: discrete semantic transfer hashing for scalable image retrieval. IEEE Trans Neural Netw Learn Syst 29(11):5264–5276

    Article  MathSciNet  Google Scholar 

  50. Zhu L, Huang Z, Liu X, He X, Sun J, Zhou X (2017) Discrete multimodal hashing with canonical views for robust mobile landmark search. IEEE Trans Multimed 19(9):2066–2079

    Article  Google Scholar 

  51. Zhu L, Shen J, Xie L, Cheng Z (2017) Unsupervised visual hashing with semantic assistant for content-based image retrieval. IEEE Trans Knowl Data Eng 29(2):472–486

    Article  Google Scholar 

  52. Zhu X, Yang J, Zhang C, Zhang S (2019) Efficient utilization of missing data in cost-sensitive learning. IEEE Trans Knowl Data Eng. https://doi.org/10.1109/TKDE.2019.2956530

    Article  Google Scholar 

  53. Zhu X, Zhang S, Hu R, He W, Lei C, Zhu P (2019) One-step multi-view spectral clustering. IEEE Trans Knowl Data Eng 31(10):2022–2034

    Article  Google Scholar 

  54. Zhu X, Zhu Y, Zheng W (2019) Spectral rotation for deep one-step clustering. Pattern Recognit. https://doi.org/10.1016/j.patcog.2019.107175

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Information Technology and Electrical Engineering, University of Queensland, Brisbane, Australia

    Luyao Liu & Zi Huang

  2. Bio-Computing Research Center, Harbin Institute of Technology, Shenzhen, 518055, China

    Zheng Zhang

Authors
  1. Luyao Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zheng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zi Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zheng Zhang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, L., Zhang, Z. & Huang, Z. Flexible Discrete Multi-view Hashing with Collective Latent Feature Learning. Neural Process Lett 52, 1765–1791 (2020). https://doi.org/10.1007/s11063-020-10221-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-020-10221-y

Search

Navigation