Skip to main content
SpringerLink
Log in

Tensorized Multi-view Subspace Representation Learning

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Self-representation based subspace learning has shown its effectiveness in many applications. In this paper, we promote the traditional subspace representation learning by simultaneously taking advantages of multiple views and prior constraint. Accordingly, we establish a novel algorithm termed as Tensorized Multi-view Subspace Representation Learning. To exploit different views, the subspace representation matrices of different views are regarded as a low-rank tensor, which effectively models the high-order correlations of multi-view data. To incorporate prior information, a constraint matrix is devised to guide the subspace representation learning within a unified framework. The subspace representation tensor equipped with a low-rank constraint models elegantly the complementary information among different views, reduces redundancy of subspace representations, and then improves the accuracy of subsequent tasks. We formulate the model with a tensor nuclear norm minimization problem constrained with \(\ell _{2,1}\)-norm and linear equalities. The minimization problem is efficiently solved by using an Augmented Lagrangian Alternating Direction Minimization method. Extensive experimental results on diverse multi-view datasets demonstrate the effectiveness of our algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. http://cvc.yale.edu/projects/yalefaces/yalefaces.html.

  2. http://cvc.yale.edu/projects/yalefacesB/yalefacesB.html.

  3. http://www.uk.research.att.com/facedatabase.html.

  4. http://www.cs.columbia.edu/CAVE/software/softlib/.

  5. http://mlg.ucd.ie/datasets/.

  6. http://mlg.ucd.ie/aggregation/.

  7. http://mlg.ucd.ie/aggregation/.

References

  • Abavisani, M., & Patel, V. M. (2018). Multimodal sparse and low-rank subspace clustering. Information Fusion, 39, 168–177.

    Google Scholar 

  • Andrew, G., Arora, R., Bilmes, J., & Livescu, K. (2013). Deep canonical correlation analysis. In ICML (pp. 1247–1255).

  • Bartels, R. H., & Stewart, G. W. (1972). Solution of the matrix equation AX + XB = C. Communications of the ACM, 15(9), 820–826.

    MATH  Google Scholar 

  • Basu, S., Bilenko, M., & Mooney, R. J. (2004). A probabilistic framework for semi-supervised clustering. In ACM SIGKDD (pp. 59–68).

  • Bickel, S., & Scheffer, T. (2004). Multi-view clustering. In ICDM.

  • Blaschko, M. B., & Lampert, C. H. (2008). Correlational spectral clustering. In CVPR.

  • Cai, J. F., Cand, E. J. S., & Shen, Z. (2010). A singular value thresholding algorithm for matrix completion. Philadelphia: Society for Industrial and Applied Mathematics.

    Google Scholar 

  • Cao, X., Wei, X., Han, Y., Yang, Y., & Lin, D. (2013). Robust tensor clustering with non-greedy maximization. In AAAI.

  • Cao, X., Zhang, C., Fu, H., Liu, S., & Zhang, H. (2015a). Diversity-induced multi-view subspace clustering. In CVPR.

  • Cao, X., Zhang, C., Zhou, C., Fu, H., & Foroosh, H. (2015b). Constrained multi-view video face clustering. TIP, 24(11), 4381–4393.

    MathSciNet  MATH  Google Scholar 

  • Chaudhuri, K., Kakade, S. M., Livescu, K., & Sridharan, K. (2009). Multi-view clustering via canonical correlation analysis. In ICML.

  • Cheng, B., Liu, G., Wang, J., Huang, Z., & Yan, S. (2011). Multi-task low-rank affinity pursuit for image segmentation. In ICCV.

  • Cheng, M., Jing, L., & Ng, M. K. (2018). Tensor-based low-dimensional representation learning for multi-view clustering. IEEE Transactions on Image Processing, 28(5), 2399–2414.

    MathSciNet  Google Scholar 

  • Christopher, D. M., Raghavan, P., & Schtze, H. (2008). Introduction to information retrieval (Vol. 1). Cambridge: Cambridge University Press.

    Google Scholar 

  • Collins, M. D., Liu, J., Xu, J., Mukherjee, L., & Singh, V. (2014). Spectral clustering with a convex regularizer on millions of images. In ECCV.

  • Cortes, C., Mohri, M., & Rostamizadeh, A. (2009). Learning non-linear combination of kernels. In NIPS.

  • Davies, D. L., & Bouldin, D. W. (1979). A cluster separation measure. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2, 224–227.

    Google Scholar 

  • De Lathauwer, L., De Moor, B., & Vandewalle, J. (2000). On the best rank-1 and rank-(\(r_1, r_2,., r_n\)) approximation of higher-order tensors. SIAM Journal on Matrix Analysis and Applications, 21(4), 1324–1342.

    MathSciNet  MATH  Google Scholar 

  • de Sa, V. R. (2005). Spectral clustering with two views. In ICML.

  • Ding, Z., Zhao, H., & Fu, Y. (2018). Learning representation for multi-view data analysis: Models and applications. Berlin: Springer.

    Google Scholar 

  • Elhamifar, E., & Vidal, R. (2013). Sparse subspace clustering: Algorithm, theory, and applications. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(11), 2765–2781.

    Google Scholar 

  • Elkahky, A. M., Song, Y., & He, X. (2015). A multi-view deep learning approach for cross domain user modeling in recommendation systems. In WWW (pp. 278–288).

  • Gao, H., Nie, F., Li, X., & Huang, H. (2015). Multi-view subspace clustering. In ICCV (pp. 4238–4246).

  • Greene, D., & Cunningham, P. (2009). A matrix factorization approach for integrating multiple data views. In Joint European conference on machine learning and knowledge discovery in databases (pp. 423–438). Springer.

  • Grigorios, T., & Aristidis, L. (2012). Kernel-based weighted multi-view clustering. In ICDM.

  • Guo, Y. (2013). Convex subspace representation learning from multi-view data. In AAAI.

  • Hu, H., Lin, Z., Feng, J., & Zhou, J. (2014). Smooth representation clustering. In CVPR (pp. 3834–3841).

  • Jing, W., Xiao, W., Feng, T., Chang, H. L., & Yu, H. (2016). Constrained low-rank representation for robust subspace clustering. IEEE Transactions on Cybernetics, 47(99), 1–13.

    Google Scholar 

  • Kamvar, K., Sepandar, S., Klein, K., Dan, D., Manning, M., & Christopher, C. (2003). Spectral learning. In IJCAI (pp. 561–566).

  • Kumar, A., & Daumé III, H. (2011). A co-training approach for multi-view spectral clustering. In ICML.

  • Kumar, A., Rai, P., & Daumé III, H. (2011). Co-regularized multi-view spectral clustering. In NIPS.

  • Lades, M., Vorbruggen, J. C., Buhmann, J., Lange, J., von der Malsburg, C., Wurtz, R. P., et al. (1993). Distortion invariant object recognition in the dynamic link architecture. IEEE Transactions on Computers, 42(3), 300–311.

    Google Scholar 

  • Lathauwer, L. D., Moor, B. D., & Vandewalle, J. (2000). A multilinear singular value decomposition. SIAM Journal on Matrix Analysis and Applications, 21(4), 1253–1278.

    MathSciNet  MATH  Google Scholar 

  • Lawrence, H., & Phipps, A. (1985). Comparing partitions. Journal of Classification, 2(1), 193–218.

    MATH  Google Scholar 

  • Lin, Z., Chen, M., & Ma, Y. (2010). The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. arXiv preprint arXiv:1010.0789.

  • Lin, Z., Liu, R., & Li, H. (2015). Linearized alternating direction method with parallel splitting and adaptive penalty for separable convex programs in machine learning. Machine Learning, 99(2), 287–325.

    MathSciNet  MATH  Google Scholar 

  • Li, Z., & Tang, J. (2016). Weakly supervised deep matrix factorization for social image understanding. IEEE Transactions on Image Processing, 26(1), 276–288.

    MathSciNet  MATH  Google Scholar 

  • Li, Z., Tang, J., & Mei, T. (2018). Deep collaborative embedding for social image understanding. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(9), 2070–2083.

    Google Scholar 

  • Liu, G., Lin, Z., Yan, S., Sun, J., Yu, Y., & Ma, Y. (2013a). Robust recovery of subspace structures by low-rank representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(1), 171–184.

    Google Scholar 

  • Liu, J., Musialski, P., Wonka, P., & Ye, J. (2013b). Tensor completion for estimating missing values in visual data. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(1), 208–220.

    Google Scholar 

  • Liu, H., Wu, Z., Cai, D., & Huang, T. S. (2012). Constrained nonnegative matrix factorization for image representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(7), 1299–1311.

    Google Scholar 

  • Liu, X., Zhu, X., Li, M., Wang, L., Tang, C., Yin, J., et al. (2019). Late fusion incomplete multi-view clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41, 2410–2423.

    Google Scholar 

  • Lu, Z., & Leen, T. K. (2007). Penalized probabilistic clustering. Neural Computation, 19(6), 1528–1567.

    MathSciNet  MATH  Google Scholar 

  • Ng, A. Y., Jordan, M. I., & Weiss, Y. (2001). On spectral clustering: Analysis and an algorithm. In NIPS.

  • Ngiam, J., Khosla, A., Kim, M., Nam, J., Lee, H., & Ng, A. Y. (2011). Multimodal deep learning. In ICML (pp. 689–696).

  • Oh, T.-H., Matsushita, Y., Tai, Y.-W., & Kweon, I. S. (2015). Fast randomized singular value thresholding for nuclear norm minimization. In CVPR.

  • Ojala, T., Pietikainen, M., & Maenpaa, T. (2002). Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(7), 971–987.

    MATH  Google Scholar 

  • Patel, V. M., & Nguyen, H. V. (2014). Latent space sparse subspace clustering. In ICCV (pp. 225–232).

  • Quintana, E. (2001). A note on parallel matrix inversion. SIAM Journal on Scientific Computing, 22(5), 1762–1771.

    MathSciNet  MATH  Google Scholar 

  • Rangapuram, S. S., & Hein, M. (2012). Constrained 1-spectral clustering. In AISTATS.

  • Soleymani, F. (2013). A fast convergent iterative solver for approximate inverse of matrices. Numerical Linear Algebra with Applications, 21(3), 439–452.

    MathSciNet  MATH  Google Scholar 

  • Sui, J., Qi, S., van Erp, T. G., Bustillo, J., Jiang, R., Lin, D., et al. (2018). Multimodal neuromarkers in schizophrenia via cognition-guided mri fusion. Nature Communications, 9(1), 1–14.

    Google Scholar 

  • Tang, W., Lu, Z., & Dhillon, I. S. (2009). Clustering with multiple graphs. In ICDM.

  • Tomioka, R., Hayashi, K., & Kashima, H. (2010). Estimation of low-rank tensors via convex optimization. arXiv preprint arXiv:1010.0789.

  • Tong, H., He, J., Li, M., Zhang, C., & Ma, W.-Y. (2005). Graph based multi-modality learning. In ACM MM (pp. 862–871). ACM.

  • Wagstaff, K., Cardie, C., Rogers, S., Schrödl, S., et al. (2001). Constrained k-means clustering with background knowledge. In ICML (vol. 1, pp. 577–584).

  • Wang, H., Weng, C., & Yuan, J. (2014). Multi-feature spectral clustering with minimax optimization. In CVPR.

  • Wang, H., Yang, Y., & Liu, B. (2019). GMC: Graph-based multi-view clustering. IEEE Transactions on Knowledge and Data Engineering. https://doi.org/10.1109/TKDE.2019.2903810.

  • Wang, W., Arora, R., Livescu, K., & Bilmes, J. (2015). On deep multi-view representation learning, pp. 1083–1092.

  • White, M., Zhang, X., Schuurmans, D., & Yu, Y.-l. (2012). Convex multi-view subspace learning. In NIPS.

  • Wu, B., Zhang, Y., Hu, B., & Ji, Q. (2013). Constrained clustering and its application to face clustering in videos. In CVPR (pp. 3507–3514).

  • Xia, R., Pan, Y., Du, L., & Yin, J. (2014a). Robust multi-view spectral clustering via low-rank and sparse decomposition. In AAAI.

  • Xia, R., Pan, Y., Du, L., & Yin, J. (2014b). Robust multi-view spectral clustering via low-rank and sparse decomposition. In AAAI (pp. 2149–2155).

  • Xie, Y., Tao, D., Zhang, W., Liu, Y., Zhang, L., & Qu, Y. (2018). On unifying multi-view self-representations for clustering by tensor multi-rank minimization. International Journal of Computer Vision, 126(11), 1157–1179.

    MathSciNet  Google Scholar 

  • Yang, E., Deng, C., Li, C., Liu, W., Li, J., & Tao, D. (2018). Shared predictive cross-modal deep quantization. IEEE Transactions on Neural Networks and Learning Systems, 29(11), 5292–5303.

    Google Scholar 

  • Zhang, C., Adeli, E., Zhou, T., Chen, X., & Shen, D. (2018a). Multi-layer multi-view classification for alzheimers disease diagnosis. In AAAI.

  • Zhang, C., Fu, H., Hu, Q., Cao, X., Xie, Y., Tao, D., et al. (2020). Generalized latent multi-view subspace clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42, 86–99.

    Google Scholar 

  • Zhang, C., Fu, H., Hu, Q., Zhu, P., & Cao, X. (2017a). Flexible multi-view dimensionality co-reduction. IEEE Transactions on Image Processing, 26(2), 648–659.

    MathSciNet  MATH  Google Scholar 

  • Zhang, C., Hu, Q., Fu, H., Zhu, P., & Cao, X. (2017b). Latent multi-view subspace clustering. In CVPR (pp. 4333–4341).

  • Zhang, T., Ghanem, B., Liu, S., Xu, C., & Ahuja, N. (2013). Low-rank sparse coding for image classification. In ICCV.

  • Zhang, T., Liu, S., Ahuja, N., Yang, M.-H., & Ghanem, B. (2014). Robust visual tracking via consistent low-rank sparse learning. International Journal of Computer Vision, 111(2), 171–190.

    MATH  Google Scholar 

  • Zhang, Z., Liu, L., Shen, F., Shen, H. T., & Shao, L. (2018b). Binary multi-view clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(7), 1774–1782.

    Google Scholar 

  • Zhan, K., Zhang, C., Guan, J., & Wang, J. (2017). Graph learning for multiview clustering. IEEE Transactions on Cybernetics, 48(10), 2887–2895.

    Google Scholar 

  • Zhao, H., Ding, Z., & Fu, Y. (2017). Multi-view clustering via deep matrix factorization. In AAAI (pp. 2921–2927).

  • Zhao, X., Evans, N., & Dugelay, J.-L. (2014). A subspace co-training framework for multi-view clustering. Pattern Recognition Letters, 41, 73–82.

    Google Scholar 

  • Zhou, C., Zhang, C., Li, X., Shi, G., & Cao, X. (2014). Video face clustering via constrained sparse representation. In ICME (pp. 1–6).

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China (Nos. 61976151, 61732011, 61925602 and U1636214), Beijing Natural Science Foundation (No. 4172068).

Author information

Authors and Affiliations

  1. College of Intelligence and Computing, Tianjin Key Lab of Machine Learning, Tianjin University, Tianjin, 300350, China

    Changqing Zhang & Qinghua Hu

  2. Inception Institute of Artificial Intelligence, Abu Dhabi, UAE

    Huazhu Fu

  3. School of Computing and Mathematical Sciences, University of Greenwich, London, SE109LS, UK

    Jing Wang

  4. School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, 611731, China

    Wen Li

  5. State Key Laboratory of Information Security, Institute of Information Engineering, Chinese Academy of Sciences, Beijing, 100093, China

    Xiaochun Cao

Authors
  1. Changqing Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huazhu Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jing Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wen Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiaochun Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Qinghua Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaochun Cao.

Additional information

Communicated by Li Liu, Matti Pietikäinen, Jie Qin, Jie Chen, Wanli Ouyang, Luc Van Gool.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, C., Fu, H., Wang, J. et al. Tensorized Multi-view Subspace Representation Learning. Int J Comput Vis 128, 2344–2361 (2020). https://doi.org/10.1007/s11263-020-01307-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-020-01307-0

Keywords

Search

Navigation