DLPNet: A deep manifold network for feature extraction of hyperspectral imagery

Abstract

Deep learning has received increasing attention in recent years and it has been successfully applied for feature extraction (FE) of hyperspectral images. However, most deep learning methods fail to explore the manifold structure in hyperspectral image (HSI). To tackle this issue, a novel graph-based deep learning model, termed deep locality preserving neural network (DLPNet), was proposed in this paper. Traditional deep learning methods use random initialization to initialize network parameters. Different from that, DLPNet initializes each layer of the network by exploring the manifold structure in hyperspectral data. In the stage of network optimization, it designed a deep-manifold learning joint loss function to exploit graph embedding process while measuring the difference between the predictive value and the actual value, then the proposed model can take into account the extraction of deep features and explore the manifold structure of data simultaneously. Experimental results on real-world HSI datasets indicate that the proposed DLPNet performs significantly better than some state-of-the-art methods.

Introduction

As an effective imaging and spectroscopy technology, hyperspectral imaging technology has attracted much attention in the past decades (Du et al., 2018, Huang et al., 2019, Liang et al., 2018, Song et al., 2018, Xie et al., 2019, Zhang, Zhang, et al., 2019). Pixel-level hyperspectral image (HSI) classification utilizes rich spectral information contained in HSI, which is widely used in urban planning, precision agriculture, mineralogy, and geological research (Hong, Yokoya, Chanussot, and Zhu, 2019, Maxwell et al., 2018, Peng et al., 2019, Su et al., 2019, Xie et al., 2018). Hundreds of continuous spectral bands provide useful information for the fine classification of land covers. However, high-dimensional characteristics of spectral bands often lead to the curse-of-dimensionality, and the traditional classification methods will suffer from the Hughes phenomena due to the limited training samples (Ghamisi et al., 2017, Huang et al., 2019, Jiao et al., 2018, Kang et al., 2018, Luo et al., 2018, Qian et al., 2017).

A straightforward strategy to tackle the above problem is to reduce the dimensionality of hyperspectral data and map high-dimensional bands to a low-dimensional embedding space through feature extraction (FE), which can obtain the intrinsic features of high-dimensional data (Chen et al., 2019, Hong, Yokoya, Chanussot, Xu, and Zhu, 2019, Li, Li, and Zhang, 2019, Luo et al., 2020, Zhang, Gong, et al., 2019). FE methods can be classified into supervised, unsupervised and semi-supervised methods (Hong, Yokoya, Ge, Chanussot, & Zhu, 2019). Unsupervised FE methods obtain low-dimensional features of HSI without priori information, such methods include principal component analysis (PCA) (Tyo, Konsolakis, Diersen, & Olsen, 2003), independent component analysis (ICA) (Bayliss, Gualtieri, & Cromp, 1998), neighborhood preserving embedding (NPE) (Lu, Jin, & Zou, 2012), and locality preserving projections (LPP) (Li, Pan, He, & Liu, 2015). However, unsupervised methods fail to exploit label information to get discriminant features for classification. Supervised FE methods attempt to explore priori knowledge of training samples to compact the intraclass samples and separate the interclass samples, which brings benefits for classification. Linear discriminant analysis (LDA) (Du & Chang, 2001) and maximum margin criterion (MMC) (Datta, Ghosh, & Ghosh, 2014) are two classical supervised learning methods. To discover the intrinsic manifold structure in HSI, many supervised manifold learning algorithms are designed to explore the discriminative properties of data by using labeled samples. The marginal Fisher analysis (MFA) (Yan et al., 2007) method constructs an intrinsic graph and a penalty graph to reveal the discriminative manifold structure in hyperspectral data. The local geometric structure Fisher analysis (LGSFA) (Luo, Huang, Duan, Liu, & Liao, 2017) method was designed to explore the geometric properties in HSI, which compacts the intraclass neighbors with its reconstruction points while separates the interclass samples from its reconstruction points. Zhang, He, and Gao (2019) designed a manifold learning-based framework to utilize an explicit polynomial mapping to learn a low-dimensional embedding space. However, the above methods depend on shallow-based feature descriptors, which are sensitive to noises and difficult to extract discriminant features under a small size of training samples.

Deep learning (DL) has been regarded as one of the state-of-the-art machine learning tools, and it has shown the advantages in many fields, such as change detection, scene understanding, and image classification (Ghamisi et al., 2018, Li, Song, et al., 2019). In recent years, DL has been applied to hyperspectral imagery to extract deep abstract features through hierarchical networks, which can deal with the complex nonlinear relationships contained in HSI (Li et al., 2020, Liu et al., 2019, Mostafa, 2017, Zhou et al., 2019). Ratle, Camps-Valls, and Weston (2010) utilized artificial neural networks (ANN) to learn the deep features in hyperspectral data, and a regularizer was designed to improve the ability of feature extraction. Chen, Zhao, and Jia (2015) investigated a deep learning model based on deep belief network (DBN) to explore the intrinsic information in HSI data, and it achieved good performance through combining deep features with spectral-spatial features. Chen, Lin, Zhao, Wang, and Gu (2014) introduced the stacked autoencoder (SAE) to classify HSI data by utilizing the rich spectral bands and obtained high classification accuracy. Although deep learning methods can extract deep features of samples to deal with the nonlinear relationship in complex scenes, they ignore to reveal manifold structure in HSI.

Based on the above discussion, a novel graph-based deep learning model termed deep locality preserving neural network (DLPNet) was proposed in this paper. The main purpose of this method is to explore the manifold structure in HSI data in the process of extracting deep features, and then deep manifold features can be extracted for classification. DLPNet introduces graph embedding into neural network and it constructs the deep-manifold joint loss function to optimize the parameters of neural network. Based on the proposed joint loss function, the difference between the predictive value and the actual value of data is calculated, and an intrinsic graph and a penalty graph are constructed to explore the manifold structure in HSI data. DLPNet designs an iteration strategy to minimize the loss value, which characterizes the intraclass compactness by calculating the sum of the distance between each sample and its neighbors from the same class, and measures the interclass separability through the sum of the distance between each sample and its neighbors belonging to different classes. In the iteration process, it enhances the separability of different classes of samples and then extracts the discriminant features.

The main characteristics of the proposed method can be concluded as the following: (1) DLPNet combines graph embedding with deep learning to explore the manifold structure of HSI data; (2) An iteration strategy is designed to optimize the network adaptively by using manifold structure embedded in hyperspectral data; (3) By exploring the structure information of HSI, the dependence of deep learning on the labeled samples is reduced, and high classification accuracy can be achieved under a small number of labeled training samples.

The remainder of this paper is organized as follows. Section 2 briefly describes the related work. The details of the proposed method are introduced in Section 3. Section 4 presents experimental results and detailed analysis, and these demonstrate the effectiveness of the proposed DLPNet. Finally, Section 5 presents our concluding remarks and gives recommendations for future works.