SRAGL-AWCL: A two-step multi-view clustering via sparse representation and adaptive weighted cooperative learning
Introduction
Multi-view clustering (MVC) has become a hot topic due to the needs of analyzing the ever-rising ubiquitous multi-view data. MVC expresses an intuitive meaning for clustering from multiple angles and it fuses these angles to obtain an optimal model that is more effective than single-view clustering. With its development, MVC can be regarded as using multiple features of samples extracted from different ways, such as local binary patterns (LBP) [1], 2D Gabor Wavelets (GABOR) [2] and histograms of oriented gradients (HOG) [3] etc. Features from different views take into account the characteristics of different aspects of the same sample, which reflects its diversity. The diversity can then be used among these features to complement the information in order to better characterize the sample, hence MVC is currently one of the most used methods for tackling this problem. MVC can be divided into supervised MVC [4], semi-supervised MVC [5] and unsupervised MVC [6] according to the need of the associated class labels. Since unsupervised MVC does not need to annotate sample data, it is more convenient than other two types of clustering methods, therefore, it saves time and effort. The most representative unsupervised MVC methods are sparse representation (SR) [7], spectral clustering (SC) [8], graph learning (GL) [9] and consensus graph clustering [10]. Hybrid methods have also been used recently by combining multiple multi-view learning methods to obtain better results. Zhang et al. [11] introduced an unsupervised clustering method which simultaneously learned fuzzy k-means and non-negative SC with side information. This method considers the internal structural information of each view and the diversity information between views. Preserving the structural information of each view and discriminating the diversity between them is another important research topic in MVC task. In the early research of MVC algorithm, researchers paid much attention to this. Cai et al. [12] proposed a SR method based on non-negative matrix factorization (NMF). By adding the manifold regularization (MR) to preserve the local structural features of the data, it can improve the properties of the clustering within the class and keep its structural integrity within the class. Unfortunately, due to the way NMF handles the original data, some useful negative information is discarded. Hence, NMF has some limitations in data processing, which is also one of the challenges our paper aims to tackle with.
Further studies show that views are not independent and often inseparable in the process of MVC. Therefore, exploring the information connection between views is also a research hotspot. Feng et al. [13] proposed an unsupervised multi-view method, called adaptive unsupervised multi-view feature selection (AUMFS). This approach only considers the similarity learning among views on a cluster label matrix. Zhan et al. [14] introduced a new method to leverage the problem of multi-view clusters by a joint approach which uses both adaptive structure concept factorization and optimization of the similarity matrix to deal with the data and the relationship information among views. Zhao et al. [15] and Wang et al. [16] suggested two models of adaptive similarity structure by using adaptive weighted decomposition of each view. In general, these methods simply weight each view to merge the various views without considering their complementarity and diversity comprehensively. Consequently, these approaches lead to uncertainty in term of the useful information inevitably. As a matter of fact, each view can be trained collaboratively to get a global matrix in which the diversity information of each view is contained. Hence, it ensures the effectiveness of clustering and makes full use of the diversity information between views. This is our motivation employing cooperative learning (CL) to solve the multi-view fusion problem.
The MVC based on CL approach has become an important direction of research in this field recently. You et al. [39] introduced a global discriminant analysis method to handle the differences between views. Kumar et al. [17] proposed a method that utilizes the MR to preserve the local structural features of each view, and transformed each view into a global matrix through model learning. Although it used the idea of cooperative representation, they did not consider the redundancy of data, nor the diversity of similarity matrix between views during process. Consequently, the existence of noise in each view can easily cause misjudgment of information in the fusion process. To solve the problem of data redundancy and cooperative representation, Wen et al. [40] used sparse matrix as the learning graph for adaptive cooperative graph learning (ACGL). The method has some robustness to the noise in each view, and in particular it is helpful for discovering the internal structure of the noise data. Brbic et al. [18] proposed a method that learned a joint subspace representation by constructing an affinity matrix shared between views, using the importance of low-rank and sparse constraints in affinity matrix construction. Even though this method embodied its robustness at a certain level, it failed to fully preserve the internal structural integrity of each view.
To conclude, the MVC algorithms based on SR and CL have solved some problems that exist in features fusion. However, further study is still required in the usage of optimal extraction and fusion of specific information in the entire view. The issues associated with the existing methods detailed above can be summarized as follows:
- (1)
In the SR of the original data, NMF requires non-negative input data and non-negative constraints on the basis matrix, which causes some useful information contained in negative input data being filtered out during decomposition.
- (2)
It lacks effective methods to extract view-specific information. Existing methods have difficulties to obtain the optimal internal structural features, as the data often contains some redundant noise.
- (3)
The diversity of views and the globality among view cannot be fully considered by independently using SR, adaptive graph learning (AGL) or CL.
In order to address these three issues, we propose an adaptive multi-view spectral clustering (MVSC) method. As shown in Fig. 1, the proposed method can be divided into two steps: 1) sparse representation with adaptive graph learning (SRAGL); and 2) adaptive weighted cooperative learning (AWCL). Fig. 1 also illustrates the specific operation process of each step. The purpose of the first step (SRAGL) is to extract the optimal sparse matrix by SR and AGL. Redundant information from the original matrix is removed and the geometry of each view is preserved to the maximum possible extent. Therefore, the first step solves issues (1) and (2). The objective of the second step (AWCL) is to merge the diversity of views with the globality between views. In this step, it is crucial that the view with specific structure is effectively fused with the correlations between views. As shown on the right hand side of Fig. 1, step 2 will provide more discriminative representation. This is because AWCL fully considers the different advantages of each view, and then obtains an optimal global matrix. The above two steps closely link with SR, AGL and CL to form a final solution to tackle issue (3).
The main contributions of this paper can be summarized as follows:
- (1)
A unique sparse matrix decomposition method is proposed to relax the non-negative constraint of the NMF basis matrix, so that the sparse matrix obtained by matrix decomposition contains more useful information.
- (2)
A direct derivative method is introduced to efficiently optimize the basis matrix and the sparse matrix. Consequently, it increases the similarity discrimination between samples, in the meantime, a quadratic processing is proposed for the coefficient matrix of the SR.
- (3)
A new two-steps algorithm is proposed to solve view specific information learning and fusion simultaneously. Firstly, we applied AGL to optimize the similarity matrix and preserve the internal structural features of each view by combining manifold learning. The integration of AGL and SR can extract the specific information of each view. After that, an AWCL fusion method is proposed to effectively fuse the view's diversity information. The specific information for different views is obtained by using adaptive weighted method to learn a global matrix for fusion. In order to make a complete global matrix data structure, a manifold learning is further applied to the global matrix.
The remaining sections of the paper are organized as follows: related works will be introduced in Section 2; the proposed method based on the adaptive SR and AWCL method is introduced in Section 3; the proposed optimization process is outlined in Section 4; Section 5 details the experimental results and analysis; and finally, Section 6 concludes the paper with prospective future work.
Section snippets
Graph learning
Many GL based methods are used to improve MVC algorithms [38] for MVC [20]. Among these methods, the similarity matrix to evaluate the similarity between samples can often be solved by applying simplest distance metric. However, it is sensitive to the initial graph input and therefore, the initialization process can critically impact the entire MVC performance. Assuming all the elements in the similarity matrix are non-negative, we can get the relevant property of the Laplacian matrix
The proposed method
In this section, the related symbols of this paper will firstly be briefed. For multi-view data, we can represent the input dataset as , where is the number of input data views, donates the -th view data, and , where and represent the number of features in each sample data and the number of samples, respectively. has the same data dimension for all views. is used to represent the -th view basis matrix of the SR, and
Optimization
In this section, effective iterative methods to solve Eqs. (5) and (7) are proposed, including specific details of the optimization of these two parts.
Multi-view datasets
- 1)
UCI_Digits:1 It includes ten classes of handwritten numbers in this dataset, which consists of ten numbers from 0 to 9. In this experiment, five views are used for each point: the first view is the 216-D profile-correlation feature; the second is the 76-D Fourier-coefficient feature; the third is 64-D Karhunen-Loeve-coefficient feature; the fourth is 240-D intensity-averaged feature in windows; and the last view is 47-D morphological
Conclusion
In this paper, a novel method has been proposed to integrate SR with CL adaptively for MVSC. Both theoretical derivations and the pseudocode of algorithms have been given in detail. The convergence of the proposed method has also been theoretically proved.
The performance of the proposed algorithm has been tested on a wide range of benchmark datasets including both multi-view datasets and single-view datasets against evaluation index ACC, NMI, F1-score, precision, ARI and computational time. The
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgements
This work is supported in part by the National Nature Science Foundation of China (no. U1701266), Guangdong Natural Science Foundation (no. 2021A1515011341), Guangzhou Science and Technology Plan Project (no. 202002030386), Guangdong Graduate Education Innovation Project (no. 2020XSLT16), and Guangdong Provincial Key Laboratory of Intellectual Property and Big Data (no. 2018B030322016).
Junpeng Tan: Junpeng Tan received the B.S. degree from College of Electrical Engineering and Automation at Luoyang Institute of Technology, Henan, China, in 2018. Currently, he is pursuing his master degree in school of information engineering at Guangdong University of Technology. His research interests are Machine Learning and Pattern Recognition. He currently focuses on the application of machine learning in feature fusion of multi-view images.
References (40)
- et al.
A novel dimensionality reduction method with discriminative generalized eigen-decomposition
Neurocomputing
(2016) - et al.
Multi-view manifold learning with locality alignment
Pattern Recognit
(2018) - et al.
Multi-view subspace clustering with intactness-aware similarity
Pattern Recognit
(2019) - et al.
Multi-view low-rank sparse subspace clustering
Pattern Recognit
(2018) - et al.
Learning generative visual models from few training examples: an incremental Bayesian approach tested on 101 object categories
Comput. Vis. Image Understand.
(2007) - et al.
Robust unsupervised feature selection via dual self-representation and manifold regularization
Knowledge-Based Syst.
(2018) - et al.
Unsupervised feature selection for visual classification via feature-representation property
Neurocomputing
(2017) - et al.
Global and local structure preserving sparse subspace learning: an iterative approach to unsupervised feature selection
Pattern Recognit
(2016) - et al.
Unsupervised feature selection by regularized self-representation
Pattern Recognit
(2015) - et al.
Joint graph based embedding and feature weighting for image classification
Pattern Recognit
(2019)
Multi-view global component discriminant analysis for cross-view classification
Pattern Recognit
Face description with local binary patterns: application to face recognition
IEEE Trans. Pattern Anal. Mach. Intell.
Image representation using 2D Gabor Wavelets
IEEE Trans. Pattern Anal. Mach. Intell.
Histograms of oriented gradients for human detection
Multi-view supervision for single-view reconstruction via differentiable ray consistency
Auto-weighted multi-view learning for image clustering and semi-supervised classification
IEEE Trans. on Image Processing
Large-margin predictive latent subspace learning for multiview data analysis
IEEE Trans. Pattern Anal. Mach. Intell.
Multiview spectral clustering via structured low-rank matrix factorization
IEEE Trans. on Neural Networks and Learning Systems
Graph structure fusion for multiview clustering
IEEE Trans. on Knowledge and Data Engineering
Multiview consensus graph clustering
IEEE trans. on image process.
Cited by (19)
Asymmetric low-rank double-level cooperation for scalable discrete cross-modal hashing
2024, Expert Systems with ApplicationsClean affinity matrix learning with rank equality constraint for multi-view subspace clustering
2023, Pattern RecognitionCitation Excerpt :Yu et al. [25] proposed an anchor auto-weighted sample level fusion method for incomplete multi-view clustering (ASA-IC) to effectively explore the relationship between each instance and the anchor. Tan et al. [26] proposed a two-step MVC method based on sparse representation and adaptive weighted cooperative learning (SRAGL-AWCL) to effectively extract the features of multiple views. By introducing an exclusivity constraint and a clustering structure constraint into the objective function, Si et al. [27] proposed an MVC method which has shown promising performance.
Cross-view graph matching for incomplete multi-view clustering
2023, NeurocomputingCitation Excerpt :The main idea of MVC is to cluster the data coming from multiple channels or with multiple modalities [5–7]. Many methods have been researched for solving MVC, for example, the multi-view k-means clustering [8–10], the co-regularized multi-view spectral clustering [11–14], and the matrix factorization-based clustering [15–17]. These studies on multi-view learning mainly focus on the clustering of complete multi-view data.
Adaptive multi-view multiple-means clustering via subspace reconstruction
2022, Engineering Applications of Artificial IntelligenceCitation Excerpt :Therefore, abundant multi-view learning approaches have been put forward to deal with multi-perspective data, which involve different application areas, such as classification (Kan et al., 2016; Wang et al., 2020c), pattern recognition (Wang et al., 2020a,b, 2021), image retrieval (Li et al., 2009; Karthik and Kamath, 2020; Li et al., 2021), clustering (Zhang et al., 2018a; Li et al., 2018; Wu et al., 2020), etc. In particular, multi-view clustering (Wang et al., 2015a; Yang et al., 2019; Tan et al., 2021) has attracted enormous research enthusiasm, aims to accumulate alike samples in the same cluster and unalike samples into distinct collections, and obtain satisfactory performance by integrating the commonality and personality information between different views. A great deal of multi-view clustering algorithms have been widely presented in recent years.
Multi-view spectral clustering by simultaneous consensus graph learning and discretization
2022, Knowledge-Based SystemsCitation Excerpt :For instance, a document can be represented by translations in several different languages. Multi-view clustering [2] aims to group multi-view data samples according to their similarities and has attracted more and more attention in a variety of multidisciplinary fields, such as data mining [3,4], pattern recognition [5,6], and machine learning [7,8]. Each view of multi-view objects corresponds to one source of information and has different characteristics, prompting scholars to exploit complementary and consensus information hidden in different views.
A Time-Series-Based Sample Amplification Model for Data Stream with Sparse Samples
2024, Neural Processing Letters
Junpeng Tan: Junpeng Tan received the B.S. degree from College of Electrical Engineering and Automation at Luoyang Institute of Technology, Henan, China, in 2018. Currently, he is pursuing his master degree in school of information engineering at Guangdong University of Technology. His research interests are Machine Learning and Pattern Recognition. He currently focuses on the application of machine learning in feature fusion of multi-view images.
Zhijing Yang: Dr. Zhijing Yang received the B.S and Ph.D. degrees from the Mathematics and Computing Science, Sun Yat-sen University, Guangzhou China, in 2003 and 2008, respectively. He was a Visiting Research Scholar in the School of Computing, Informatics and Media, University of Bradford, U.K, between July-Dec, 2009, and a Research Fellow in the School of Engineering, University of Lincoln, U.K, between Jan. 2011 to Jan. 2013. He is currently a Professor and Vice Dean at the School of Information Engineering, Guangdong University of Technology, China. He has published over 60 peer-review journal and conference papers. His research interests include time-frequency analysis, signal processing, machine learning, and pattern recognition.
Yongqiang Cheng: Dr. Yongqiang Cheng received the Ph.D. degree from the School of Engineering, Design and Technology, University of Bradford, Bradford, U.K. Dr. Yongqiang Cheng is currently a Senior Lecturer with the Department of Computer Science and Technology, University of Hull, Hull, U.K. His research interest includes digital healthcare technologies, embedded systems, control theory and applications, and AI and data mining.
Jielin Ye: Jielin Ye received the B.S. degree from school of information engineering at Xiangtan University, Xiangtan, China, in 2018. Now, she is pursuing her master degree in school of information engineering at Guangdong University of Technology. Her research interests are Image Retrieval and Machine Learning. She currently focuses on product quantization for approximate k-nearest neighbor search in high-dimensional space.
Bing Wang: Dr. Bing Wang received the Ph.D. degree from the University of York, U.K. Dr. Bing Wang completed his Postdoctoral Research with the Engineering Department, University of Cambridge, where he designed and developed a multi-task multimedia interface, now widely used by the British Petroleum Company plc, U.K. He currently lectures at the Computer Science and Technology Department, University of Hull, specializing in semantic data modeling, hypermedia systems, and medical related technologies. He is a leading figure in the research areas focusing on database theories, and mark-up languages semantics and design.
Qingyun Dai: Qingyun Dai received the Ph.D. degree from South China University of Technology. She is a Full Professor in the School of Information Engineering, the Director of Intelligent Signal Processing and Big Data Research Center and the Director of Science and Technology Research Office of the Guangdong University of Technology. Her research interests mainly include wavelets, image processing, image retrieval, pattern recognition, manufacturing engineering systems, RFID, cloud computing, and big data, etc.