Abstract

Hyperspectral images (HSIs) contain large amounts of spectral and spatial information, and this provides the possibility for ground object classification. However, when using the traditional method, achieving a satisfactory classification result is difficult because of the insufficient labeling of samples in the training set. In addition, parameter adjustment during HSI classification is time-consuming. This paper proposes a novel fusion method based on the maximum noise fraction (MNF) and adaptive random multigraphs for HSI classification. Considering the overall spectrum of the object and the correlation of adjacent bands, the MNF was utilized to reduce the spectral dimension. Next, a multiscale local binary pattern (LBP) analysis was performed on the MNF dimension-reduced data to extract the spatial features of different scales. The obtained multiscale spatial features were then stacked with the MNF dimension-reduced spectral features to form multiscale spectral-spatial features (SSFs), which were sent into the RMG for HSI classification. Optimal performance was obtained by fusion. For all three real datasets, our method achieved competitive results with only 10 training samples. More importantly, the classification parameters corresponding to different hyperspectral data can be automatically optimized using our method.

1. Introduction

Due to the advancements in remote sensing technology, hyperspectral images (HSIs) are containing increasing spectral and spatial information (SSI), resulting in their extensive use in domains, such as forest inventory [1], urban area monitoring [2], road extraction [3], geological surveys [4], precision agriculture [5], environmental protection [6], military applications [7], hydrocarbon detection [8], oil reservoir exploration [9], and lake sediment analysis [10]. HSI classification is a crucial research topic related to these applications. In contrast to those of Synthetic Aperture Radar (SAR) [11] or RGB images [12], the two main challenges associated with HSI classification are the high dimensionality of the dataset and the redundancy of spectral information.

Many approaches for HSI dimension reduction have been proposed. The most frequently used methods are principal component analysis (PCA) [13, 14], independent component analysis (ICA) [15], maximum noise fraction (MNF) [16], linear discriminant analysis [17], and the deep learning approach [18]. Uddin et al. [13] proposed an information-theoretic normalized-based PCA for HSI classification. Fu et al. [14] proposed a segmented PCA-based algorithm for HSI classification.

However, because the noise in hyperspectral data can easily mask subtle hyperspectral features, careful noise removal is required to extract useful information. This phenomenon is problematic for PCA, which maximizes the variance of the orthogonal set of the projection sample and vector. Unlike PCA, MNF aims to maximize the signal-to-noise ratio (SNR) rather than the variance. MNF is worth considering in HSI classification because removing the noise is effective during dimensionality reduction.

HSI classification can be significantly improved by employing an appropriate object-based [19], spectral-spatial fusion-based approach [20], decision fusion-based method [21], or deep learning-based method [22]. However, two challenges are unavoidable for most researchers. On the one hand, the traditional depth learning-based approaches rely heavily on a large amount of labeled data to achieve competitive results, whereas HSI annotation is expensive and time-consuming because it requires expert knowledge and skills. On the other hand, many methods require the use of a large number of parameter settings during the experiment, which involves expert experience [23]. HSI classification has been widely studied, and various classification methods have been adopted, namely, support vector machine (SVM) [24], random forests [25], neural networks [26], low-rank representation [27], sparse representation [28], deep learning methods [29, 30], and meta-learning methods [31]. Xu et al. [29] directly used the random patches extracted from the HSI image as the convolution kernel, without any training, to improve the classification efficiency. Yin et al. [30] proposed a CapsNet-based alternative data-driven HSI classification model. The aforementioned results demonstrate that the proposed method can achieve ideal results when the training samples are sufficient.

To integrate the spatial information, researchers have proposed many spectral-spatial feature extraction (FE) techniques, such as the extinction profile (EP) [32] and local binary patterns (LBP) [33]. In contrast with the EP features, LBP features facilitate the mining of the HSI texture information, such as global contrast information and texture depth [34]. In this study, we adopted the LBP features as spatial features.

However, when the labeled samples in the training set are insufficient, the classification accuracy achieved by the traditional methods significantly reduces [35] because of the so-called Hughes effect or the curse of dimensionality [36]. Moreover, many methods require a series of manual parameter settings. For instance, to extract spatial features, researchers must select an appropriate window size for the neighborhood. The selection is time-consuming [37] and requires expertise [38]. Since 2010, ensemble learning methods for HSI classification have received significant attention because of their dependence on limited training samples [39]. Many methods have been developed, such as support vector machines [40], boosting [41], segmentation-based methods [19], unsupervised methods [42], and semisupervised methods [43]. However, the graph-based semisupervised method is rarely considered in HSI classification [44, 45].

Motivated by the aforementioned discussions, by combining the SSI, we proposed a new spectral-spatial HSI semisupervised classifier based on MNF and adaptive random multigraphs (SS-MNF-ARMG). The primary contributions of this study are as follows:(1)A novel spectral-spatial HSI semisupervised classification framework was developed. Because of the adaptive properties, the optimal parameters can be determined without artificial auxiliaries.(2)By introducing the MNF, the noise in the HSI can be removed more efficiently during dimensionality reduction. On the basis of dimension-reduced HSI, the SSI is combined, which can degrade the curse of dimensionality.(3)In contrast with several studies, SS-MNF-ARMG can achieve competitive performance for three real HSI datasets while leveraging tiny labeled samples, which is further improved by introducing RMG in a new mode.

2. Relevant Work

2.1. MNF

Let be the HSI data, and and are the signal and noise part of , respectively. The goal of MNF is to seek out a linear transformation matrix to maximize the SNR of the transformed data. Assume that and are uncorrelated; then can be represented as [43]

And, the covariance matrix (CM) of can be obtained bywhere and are the CMs of and , respectively. The MNF transform can be expressed aswhere is the MNF result of , are the eigenvectors associated with the largest eigenvalues of , and is the number of MNF principal components.

Then the SNR of each can be described aswhere computes the variance and is the component in . Then we can obtain by solving the following problem:

2.2. LBP

For each pixel in HSI, a P-bit binary code and a new matrix with the new value (binary to decimal value) are generated by thresholding adjacent pixel values. The LBP operator can be calculated bywhere is the number of neighbors represented on a circle with the radius , and and , respectively, represent the gray level intensity values of the center and the neighbour. The binary threshold function is described as

Take the 10th band of Indian Pines HSI as an example; the procedure of LBP is shown in Figure 1. As shown in Figure 1, for a given center pixel in a 3 × 3 window, binary labels (“0” or “1”) are assigned to adjacent pixels according to whether the gray value of the center pixel is large. Starting at the top left corner, all binary codes are joined clockwise to produce an 8-bit binary number. The resulting binary number is called LBP code. The results show that LBP algorithm has significant rotation invariance and gray invariance [46], and it can be effectively applied to HSI classification [47].

2.3. RMG

Given the HSI dataset is comprised of labeled data and unlabeled data , then a weighted graph can be obtained. The vertices in the graph consist of and . Weighted edges, which can be defined as a matrix , represent the similarities between associated nodes. For a c-class classification problem, it can be defined as a quadratic optimization problem [43]:where is the trace function and is the prediction matrix to be learned. The indicator vector is the label vector corresponding to and is the 0 vector. In addition, we can obtain by the following equation:

Then each can be classified to the class if is the largest one in the row of which can be described as . is a diagonal matrix, and its element can be calculated aswhere and are two parameters.

The popular choice foris the graph Laplacian [48], which is defined aswhere is the weight matrix of the graph, which can be formulated by Gaussian kernel aswhere is the kernel width parameter to be adjusted. And the diagonal matrix is the row sum of .

However, it is difficult to discover the neighborhood structure inherent in the graph and learn the proper compact representation automatically. To solve this problem, researchers have proposed the anchor graph algorithm (AGA) [49] and multiview anchor graph algorithm [50]. The AGA extrapolates the Laplacian eigenvector of the graph to the eigenfunction, allowing constant time hashing of new data points. Then the hierarchical threshold learning method is used to make each feature function generate more than 1 bit to improve the search accuracy. And the label prediction function in the AGA can be expressed aswhereis the data-adaptive weight, and each in is an anchor point. By this formula, the solution space of the unknown labels can be reduced from a larger space to a smaller one. The centers of the K-means cluster are selected as anchor points because these centers have a powerful representation that covers the entire dataset. In this paper, we use Local Anchor Embedding (LAE) [49] to calculate the anchor points. Figure 2 shows the flowchart of RMG.

3. Proposed Method

The SS-MNF-ARMG framework is shown in Figure 3. Because of their adaptive properties, the optimal parameters can be determined without artificial auxiliaries. The framework comprises three main modules:(1)Preprocessing the HSI image by applying MNF, the noise in the HSI can be removed effectively during dimension reduction. This result avoids the problem of the dimension.(2)Through LBP, spatial vectors corresponding to different neighborhood regions are obtained. These spatial vectors are stacked with the spectral vector, respectively, and a series of spectral-spatial stacked feature information can be obtained.(3)For classification and decision fusion, a set of necessary parameters for the RMG is established and the spectral-spatial feature information is integrated into the RMG. Next, a set of classification results with different accuracies are obtained, based on which the optimal classification results are obtained through decision fusion. In addition, by injecting randomness into the graph in the RMG, overfitting due to the limited training sample can be avoided.

The proposed SS-MNF-ARMG algorithm is summarized in Algorithm 1.

Input: the original HSI set is ; the training set is ; the test set is ; the number of spectra after dimension reduction is ; the patch size for FE is , whereis odd and ;
Output: the best classification result of all test samples;
(1)for each do
(2)Obtain the dimension-reduced HSI by using MNF;
(3)Extract the spectral vector of ;
(4)for each do
(5)Calculate the spatial vector by using LBP;
(6)Obtain the spectral-spatial vector by stacking the and the ;
(7)Obtain the best (overall accuracy) by voting, and the corresponding confusion matrix ;
(8)end for
(9)Obtain the best from by decision fusion, and the corresponding confusion matrix ;
(10)end for

4. Experimental Results and Analysis

4.1. Experimental Datasets

Three hyperspectral datasets were employed to evaluate the performance of the SS-MNF-ARMG.

Indian Pines: this scene was gathered by the AVIRIS sensor over the Indian Pines test site in northwestern Indiana and consists of 145 × 145 pixels and 224 spectral reflectance bands in the wavelength range 0.4 to 2.5 μm. This scene, which includes 16 different ground truths, contains two-thirds of agriculture and one-third of forest or other natural perennial vegetation. The number of bands was reduced to 200 by removing the 24 water absorption bands.

Pavia University: this scene was acquired by the ROSIS sensor during a flight campaign over Pavia, northern Italy. The number of spectral bands was 103 at Pavia University. It is a 610 × 340 pixel image containing nine different ground objects with a geometric resolution of 1.3 meters.

Salinas: this scene was collected by the 224-band AVIRIS sensor over the agricultural area of Salinas Valley, California, and is characterized by high spatial resolution (3.7 m pixels). After discarding 20 water absorption bands, the size of this data image was 512 × 217, with 204 bands. Salinas ground truth contains 16 classes, including vegetables, bare soils, and vineyard fields.

4.2. Analysis of Experimental Parameters

Parameters can affect the classification accuracy, such as the number of spectral bands , the patch size , the number of sampling points , the number of graphs , and the number of features . The adaptive properties of the proposed SS-MNF-ARMG are such that most optimal parameters for , , , and can be determined without artificial auxiliaries.

Based on the existing result [44], we let , , , and , where represents the dimensionality of the spectral-spatial vector .

In this study, we set the number of bands from 3 to 35 to evaluate the impact on the three HSI datasets. We conducted the experiment five times, and the optimal experimental results are shown in Figure 4.

It can be observed that, on the Indian Pines dataset and the Salinas scene dataset, the value of overall classification accuracy (OA) shows a trend of steady growth. In other words, the increase in the number of spectral bands number of training has a positive promotion on the classification performance. In particular, on the Pavia University dataset, there is a sharp growth when the number of bands is above 15. In general, the OA of the proposed method is above 78% for all three HSI datasets. Especially in the Salinas scene dataset, the OA of the method is not less than 92% even with a small number of spectral bands, demonstrating the robustness of our method.

4.3. Comparison and Analysis

The proposed SS-MNF-ARMG method was compared with several state-of-the-art spectral-spatial fusion methods, such as Pixon-based classifier [19], PCA-SPCA-2D-SSA [14], R-VCANet [20], RN-FSC [31], iCapsNet [30], RPNet [29], and MBFSDA [21]. A comparison of the above algorithms can be seen in Table 1.

To quantitatively compare the classification performance of the methods shown in Table 1, we used the average classification accuracy (AA), overall classification accuracy (OA), and the kappa coefficient (kappa) to assess the classification performance. To demonstrate the superior performance of the SS-MNF-ARMG with a limited number of training samples, we randomly selected 10 samples from each class as training samples. Tables 24 show the ground truth classes and their respective training and testing numbers for the three HSI datasets. Tables 57 summarize the experimental results for the three HSI datasets, from which the following conclusions can be drawn:(1)The results on the Indian Pines dataset show that almost all algorithms are effective. We can observe from Table 5 that the proposed SS-MNF-ARMG achieves a 3.78–28.6% advantage over the other methods in OA. In addition, for the classes that other methods do not recognize accurately, SS-MNF-ARMG can obtain better results, such as objects 1#, 2#, 3#, 5#, 6#, and 7# in Table 5.(2)The results on the Pavia University dataset demonstrate that our method has advantages over some state-of-the-art methods. As a decision fusion-based algorithm, the proposed SS-MNF-ARMG surpasses MBFSDA by 3.87% in OA. In addition, for the classes that other methods do not recognize accurately, SS-MNF-ARMG can obtain better results, such as objects 2#, 3#, 6#, and 8# in Table 6.(3)The results on the Salinas dataset show that all methods show close results in AA, but SS-MNF-ARMG has a competitive advantage in OA and kappa. From Table 7, we can observe that, compared with R-VCANet, SS-MNF-ARMG obtains a 0.1% improvement in OA, a 3.13% improvement in AA, and a 7.31% improvement in kappa. Furthermore, the proposed SS-MNF-ARMG surpasses iCapsNet by 11.01% in OA. In addition, for the classes that other methods do not recognize accurately, SS-MNF-ARMG can obtain better results, such as objects 3#, 7#, 8#, 9#, 15#, and 16# in Table 7.(4)In general, the decision fusion-based methods outperform the segmentation-based methods and feature fusion-based methods. First and foremost, the LBP features improve the performance of the SS-MNF-ARMG, and the application of MNF reduces the HSI data dimension, controlling the noise in the HSI data. In addition, the randomness in the SS-MNF-ARMG can be regarded as a regularization technique, which alleviates the phenomenon of overfitting. Finally, benefitting from the semisupervised deep learning framework, the classification accuracy is relatively satisfactory even with only a few labeled samples.

Our experiments were performed using MATLAB 2018b on a computer with an IntelCore(TM) i5-4300M 2.60 GHz CPU, 16 GB memory, and a 64-bit Windows 7 system. For the three real HSI datasets, the duration to execute our algorithm was several minutes to several hours; notably, the duration to execute the other methods that we reviewed in this study was shorter than that of our method.

5. Conclusions and Further Research

In this study, we developed a novel decision fusion method for HSI data classification. In the proposed SS-MNF-ARMG, MNF and multiscale LBP were integrated to extract local SSFs. On the one hand, MNF helped reduce the dimension of the HSI, remove the noise in the HSI, and extract the spectral features from the HSI data. On the other hand, multiscale LBP was applied to the MNF dimension-reduced images to derive spatial features at different scales. These spatial features were further fused with the MNF spectral feature to form the SSFs. Compared with some state-of-the-art spectral-spatial classification methods, our experimental results have demonstrated that SS-MNF-ARMG can achieve higher classification accuracy with limited training samples. This method is effective for distinguishing different land cover types. In addition, a set of optimal parameters for different hyperspectral data can be obtained automatically.

Although the SS-MNF-ARMG algorithm has provided promising results, the classification accuracies of various datasets remain different. Further research could attempt to further improve the generalization ability of our method. Due to human ecological destruction or natural disasters (e.g., earthquakes), the vegetation in some areas has significantly changed. Monitoring vegetation restoration in these areas is of substantial importance. Therefore, we plan to research the application of the HSI classification in vegetation restoration monitoring.

Data Availability

The data used to support the findings of this study are available at GIC (http://www.ehu.eus/ccwintco/index.php?title=Hyperspectral_Remote_Sensing_Scenes).

Conflicts of Interest

The authors declare no conflicts of interest.

Acknowledgments

This work was supported by the National Natural Science Foundation of China under Grant nos. 61703060, 61802036, 61701048, and 61873305, the Sichuan Science and Technology Program under Grant no. 21YYJC0469, the Project funded by China Postdoctoral Science Foundation under Grant no. 2020M683274, the Fundamental Research Funds of the Central Universities, Southwest Minzu University (2019NQN07), the Opening Fund of Geomathematics Key Laboratory of Sichuan Province (scsxdz2020zd01 and csxdz201710), and the Key Laboratory of Pattern Recognition and Intelligent Information Processing of Sichuan (MSSB-2018-0 and MSSB-2020-9).