Skip to main content
SpringerLink
Log in

Multi-view low rank sparse representation method for three-way clustering

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

During the past years, multi-view clustering algorithms have demonstrated satisfactory clustering results by fusing the multiple views of the dataset. Nowadays, the researches of dimensionality reduction and learning discriminative features from multi-view data have soared in the literatures. As for clustering, generating the suitable subspace of the high dimensional multi-view data is crucial to boost the clustering performance. In addition, the relationship between the original data and the clusters still remains uncovered. In this article, we design a new multi-view low rank sparse representation method based on three-way clustering to tackle these challenges, which derive the common consensus low dimensional representation from the multi-view data and further proceed to get the relationship between the data items and clusters. Specifically, we accomplish this goal by taking advantage of the low-rank and the sparse factor on the data representation matrix. The \(L_{2,1}\) norm is imposed on error matrix to reduce the impact of noise contained in the data. Finally, a new objective function is constructed to preserve the consistency between the views by using the low-rank sparse representation technique. The weighted low-rank matrix is utilized to build the consensus low rank matrix. Then, the whole objective function is optimized by using the Augmented Lagrange’s Multiplier algorithm. Further, to find the uncertain relationship between the data items and the clusters, we pursue the neighborhood based three-way clustering technique to reflect the data items into core and fringe regions. Experiments conducted on the real-world datasets show the superior performance of the proposed method compared with the state-of-the-art algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. http://mlg.ucd.ie/datasets/3sources.html.

  2. http://lig-membres.imag.fr/grimal/data.html.

  3. http://mlg.ucd.ie/datasets/bbc.html.

  4. http://mlg.ucd.ie/datasets/segment.html.

  5. http://archive.ics.uci.edu/ml/datasets/Multiple+Features.

  6. http://lig-membres.imag.fr/grimal/data.html.

  7. https://linqs.soe.ucsc.edu/data.

  8. http://www.svcl.ucsd.edu/projects/crossmodal/.

References

  1. Yang Y, Wang H (2018) Multi-view clustering: a survey. Big Data Min Anal 1(2):83–107

    Article  Google Scholar 

  2. Ye W, Wang H, Yan S, Li T, Yang Y (2019) Nonnegative matrix factorization for clustering ensemble based on dark knowledge. Knowl-Based Syst 163:624–631

    Article  Google Scholar 

  3. Zhao J, Xie X, Xu X, Sun S (2017) Multi-view learning overview: recent progress and new challenges. Inform Fusion 38:43–54

    Article  Google Scholar 

  4. Mitra S, Hasanuzzaman M, Saha S (2020) A unified multi-view clustering algorithm using multi-objective optimization coupled with generative model. ACM Trans Knowl Discov Data (TKDD) 14(1):1–31

    Article  Google Scholar 

  5. Peng X, Huang Z, Lv J, Zhu H, Zhou JT (2019) Comic: Multi-view clustering without parameter selection. In: Proc. of 36th international conference on machine learning, ICML, pp 5092–5101

  6. Huang S, Kang Z, Xu Z (2018) Self-weighted multi-view clustering with soft capped norm. Knowl-Based Syst 158:1–8

    Article  Google Scholar 

  7. Chen MS, Huang L, Wang CD, Huang D (2020) Multi-view clustering in latent embedding space. In: Proc. of 34th AAAI conference on artificial intelligence, AAAI, vol 34, no 4, pp 3513–3520

  8. Kumar A, Daumé H (2011) A co-training approach for multi-view spectral clustering. In: Proc. of 28th international conference on machine learning, ICML, pp 393–400

  9. Tao H, Hou C, Liu X, Liu T, Yi D, Zhu J (2018) Reliable multi-view clustering. In: Proc. of 32nd AAAI conference on artificial intelligence, AAAI, vol 32, no 1, pp 4123–4130

  10. de Morsier F, Borgeaud M, Gass V, Thiran J-P, Tuia D (2016) Kernel low-rank and sparse graph for unsupervised and semi-supervised classification of hyperspectral images. IEEE Trans Geosci Remote Sens 54:3410–3420

    Article  Google Scholar 

  11. Yu H, Zhang H (2016) A three-way decision clustering approach for high dimensional data. In: Proc. of international joint conference on rough sets, IJCRS, pp 229–239

  12. Hu Z, Nie F, Tian L, Wang R, Li X (2018) A comprehensive survey for low rank regularization. arXiv:1808.04521

  13. Elhamifar E, Vidal R (2013) Sparse subspace clustering: algorithm, theory, and applications. IEEE Trans Pattern Anal Mach Intell 35(11):2765–2781

    Article  Google Scholar 

  14. Parsons L, Haque E, Liu H (2004) Subspace clustering for high dimensional data: a review. ACM SIGKDD Explor Newslett 6(1):90–105

    Article  Google Scholar 

  15. Wang Y, Zhang W, Wu L, Lin X, Fang M, Pan S (2016) Iterative views agreement: an iterative low-rank based structured optimization method to multi-view spectral clustering. In: Proc. of 25th International Joint Conference on Artificial Intelligence, IJCAI, pp 2153–2159

  16. Ding Z, Fu Y (2014) Low-rank common subspace for multi-view learning. In: Proc. of 14th IEEE international conference on data mining, ICDM, pp 110–119

  17. Wang Y, Wu L, Lin X, Gao J (2018) Multiview spectral clustering via structured low-rank matrix factorization. IEEE Trans Neural Netw Learn Syst 29:4833–4843

    Article  Google Scholar 

  18. Brbic M, Kopriva I (2018) Multi-view low-rank sparse subspace clustering. Pattern Recogn 73:247–258

    Article  Google Scholar 

  19. Zhao Y, Dou Y, Liu X, Li T (2016) A novel multi-view clustering method via low-rank and matrix-induced regularization. Neurocomputing 216:342–350

    Article  Google Scholar 

  20. Yu H, Wang X, Wang G, Zeng X (2020) An active three-way clustering method via low-rank matrices for multi-view data. Inform Sci 507:823–839

    Article  Google Scholar 

  21. Liu G, Lin Z, Yan S, Sun J, Yu Y, Ma Y (2013) Robust recovery of subspace structures by low-rank representation. IEEE Trans Pattern Anal Mach Intell 35:171–184

    Article  Google Scholar 

  22. Huang S, Kang Z, Tsang IW, Xu Z (2019) Auto-weighted multi-view clustering via kernelized graph learning. Pattern Recogn 88:174–184

    Article  Google Scholar 

  23. Brbic M, Kopriva I (2020) \({l_0}\)-motivated low-rank sparse subspace clustering. IEEE Trans Cybern 50:1711–1725

    Article  Google Scholar 

  24. Ding Z, Fu Y (2016) Robust multi-view subspace learning through dual low-rank decompositions. In: Proc. of 30th AAAI conference on artificial intelligence, AAAI, vol 30, no 1, pp 1181–1187

  25. Xia R, Pan Y, Du L, Yin J (2014) Robust multi-view spectral clustering via low-rank and sparse decomposition. In: Proc. of 28th AAAI conference on artificial intelligence, AAAI, vol 28, no 1, pp 2149–2155

  26. Zhan K, Niu C, Chen C, Nie F, Zhang C (2018) Graph structure fusion for multi-view clustering. IEEE Trans Knowl Data Eng 31(10):1984–1993

    Article  Google Scholar 

  27. Zhan K, Zhang C, Guan J, Nie F, Wang J (2017) Graph learning for multi-view clustering. IEEE Trans Cybern 48(10):2887–2895

    Article  Google Scholar 

  28. Zhan K, Nie F, Wang J, Yang Y (2018) Multiview consensus graph clustering. IEEE Trans Image Process 28(3):1261–1270

    Article  MathSciNet  Google Scholar 

  29. Jun W, Zhouchen L, Daming S, Dansong C, Yongqiang Z, Junbin G (2016) LRSR: low-rank-sparse representation for subspace clustering. Neurocomputing 214:1026–1037

    Article  Google Scholar 

  30. Xiujun Z, Chen X, Xiaoli S, George B (2016) Schatten-\(q\) regularizer constrained low rank subspace clustering model. Neurocomputing 182:36–47

    Article  Google Scholar 

  31. Qian Y, Yin X, Kong J, Wang J, Gao W (2019) Low-rank graph optimization for multi-view dimensionality reduction. PLoS One 14(12):2765–2781

    Article  Google Scholar 

  32. Hong T, Chenping H, Yuhua Q, Jubo Z, Dongyun Y (2020) Latent complete row space recovery for multi-view subspace clustering. IEEE Trans Image Process 29(11):8083–8096

    MathSciNet  Google Scholar 

  33. Miaomiao C, Liping J, Michael KN (2018) Tensor-based low-dimensional representation learning for multi-view clustering. IEEE Trans Image Process 28(5):2399–2414

    MathSciNet  Google Scholar 

  34. Yu H, Chang Z, Wang G, Chen X (2020) An efficient three-way clustering algorithm based on gravitational search. Int J Mach Learn Cybern 11(5):1003–1016

    Article  Google Scholar 

  35. Chengming Z, Lin M, Panhong W, Duoqian M (2020) Multi-view and multi-label method with three-way decision-based clustering. In: Proc. of 3rd Chinese conference on pattern recognition and computer vision, PRCV, pp 69–80

  36. Yao Y (2012) An outline of a theory of three-way decisions. In: Proc. of 8th international conference on rough sets and current trends in computing, RSCTC, pp 1–17

  37. Wang P, Shi H, Yang X, Mi J-S (2019) Three-way \(k\)-means: integrating \(k\)-means and three-way decision. Int J Mach Learn Cybern 10:1–11

    Article  Google Scholar 

  38. Wang P, Yao Y (2018) Ce3: a three-way clustering method based on mathematical morphology. Knowl-Based Syst 155:54–65

    Article  Google Scholar 

  39. Yao Y (2016) Three-way decisions and cognitive computing. Cogn Comput 8:543–554

    Article  Google Scholar 

  40. Yu H, Wang X, Wang G (2017) A semi-supervised three-way clustering framework for multi-view data. In: Proc. of international joint conference on rough sets, IJCRS, pp 313–325

  41. Yu H (2017) A framework of three-way cluster analysis. In: Proc. of international joint conference on rough sets, IJCRS, pp 300–312

  42. Shi H, Liu Q, Wang P (2018) Three-way spectral clustering. In: Proc. of 24th international symposium on methodologies for intelligent systems, ISMIS, pp 389–398

  43. Yang L, Hou K (2018) A method of incomplete data three-way clustering based on density peaks. In: Proc. of American Institute of Physics conference series, AIP, vol 1967, no 1, p 020008

  44. Yang M-S, Sinaga KP (2019) A feature-reduction multi-view \(k\)-means clustering algorithm. IEEE Access 7(114):472 (114–486)

    Google Scholar 

  45. Khan GA, Hu J, Li T, Diallo B, Wang H (2021) Multi-view data clustering via non-negative matrix factorization with manifold regularization. Int J Mach Learn Cybern 1–13

  46. Ye F, Chen Z, Qian H, Li R, Chen C, Zheng Z (2018) New approaches in multi-view clustering. In: Recent applications in data clustering, p 195

  47. Wen J, Fang X, Xu Y, Tian C, Fei L (2018) Low-rank representation with adaptive graph regularization. Neural Netw 108:83–96

    Article  Google Scholar 

  48. Elhamifar E, Vidal R (2009) Sparse subspace clustering. In: 2009 IEEE conference on computer vision and pattern recognition, CVPR, pp 2790–2797

  49. Shen Y, Wen Z, Zhang Y (2014) Augmented Lagrangian alternating direction method for matrix separation based on low-rank factorization. Optim Methods Softw 29:239–263

    Article  MathSciNet  Google Scholar 

  50. Cai J-F, Candès EJ, Shen Z (2010) A singular value thresholding algorithm for matrix completion. SIAM J Optim 20(4):1956–1982

    Article  MathSciNet  Google Scholar 

  51. Ng AY, Jordan MI, Weiss Y (2002) On spectral clustering: analysis and an algorithm. In: NIPS’01: Proc. of the 14th international conference on neural information processing systems: natural and synthetic, vol 14, pp 849–856

  52. Kumar A, Rai P, Daume H (2011) Co-regularized multi-view spectral clustering. In: NIPS’11: Proc. of the 24th international conference on neural information processing systems, vol 24, pp 1413–1421

  53. Liu J, Wang C, Gao J, Han J (2013) Multi-view clustering via joint nonnegative matrix factorization. In: Proc. of 13th SIAM international conference on data mining, SMD, pp 252–260

  54. Tzortzis G, Likas A (2012) Kernel-based weighted multi-view clustering. In: Proc. of 12th international conference on data mining, ICDM, pp 675–684

  55. Jianlong W, Lin Z, Zha H (2019) Essential tensor learning for multi-view spectral clustering. IEEE Trans Image Process 28(12):5910–5922

    Article  MathSciNet  Google Scholar 

  56. Zhang C, Fu H, Liu S, Cao G (2015) Low-rank tensor constrained multiview subspace clustering. In: Proc. of 15th international conference on computer vision, ICCV, pp 1582–1590

  57. Zhang C, hu Q, Fu H, Zhu P, Cao X (2017) Latent multi-view subspace clustering. In: Proc. of IEEE conference on computer vision and pattern recognition, CVPR, pp 4279–4287

  58. Zhao K, Xinjia Z, Chong P, Hongyuan Z, Tianyi ZJ, Xi P, Wenyu C, Zenglin X (2020) A partition level multi-view subspace clustering. Neural Netw 122:279–288

    Article  Google Scholar 

  59. Zhao K, Wangtao Z, Zhitong Z, Junming S, Meng H, Zenglin X (2020) Large-scale multi-view subspace clustering in linear time. In: Proc. of 34th AAAI conference on artificial intelligence, AAAI, vol 34, no 4, pp 4412–4419

  60. Maaten LVD, Hinton G (2008) Visualizing data using t-SNE. J Mach Learn Res 9:2579–2605

    MATH  Google Scholar 

Download references

Acknowledgements

This work is supported by the National Science Foundation of China (Nos. 61772435, 61976182, 61876157) and Sichuan Key R&D project (Nos. 2020YFG0035, 2021YFG0312).

Author information

Authors and Affiliations

  1. School of Computing and Artificial Intelligence, Southwest Jiaotong University, Chengdu, 611756, China

    Ghufran Ahmad Khan, Jie Hu, Tianrui Li, Bassoma Diallo & Yimiao Zhao

Authors
  1. Ghufran Ahmad Khan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jie Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tianrui Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bassoma Diallo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yimiao Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jie Hu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Khan, G.A., Hu, J., Li, T. et al. Multi-view low rank sparse representation method for three-way clustering. Int. J. Mach. Learn. & Cyber. 13, 233–253 (2022). https://doi.org/10.1007/s13042-021-01394-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-021-01394-6

Keywords

Search

Navigation