Robust distributed estimation based on a generalized correntropy logarithmic difference algorithm over wireless sensor networks

doi:10.1016/j.sigpro.2020.107731

Signal Processing

Volume 177, December 2020, 107731

https://doi.org/10.1016/j.sigpro.2020.107731 Get rights and content

Highlights

•
A diffusion generalized correntropy-based logarithmic difference (d-GCLD) algorithm is proposed for distributed estimation.
•
The proposed algorithm can achieve desirable performance both in Gaussian and Impulsive noise environment over WSNs.
•
Mean and mean-square performance analyses of the d-GCLD algorithm are provided.
•
Experimental results are provided to show the excellent performance of the developed new algorithms.

Abstract

Distributed adaptive learning algorithms have played a critical role in signal processing and parameter estimation over networks. Most existing algorithms are based on the mean-square error (MSE) criterion, and they can achieve desirable performance when the noise is modeled as Gaussian. However, the performance of MSE-based algorithms may degrade dramatically with the impulsive noise. Therefore, the aim of this paper is to present a diffusion algorithm, named generalized correntropy-based logarithmic difference (d-GCLD) algorithm, for distributed estimation that incorporates robustness to wireless sensor networks (WSNs). By combining the logarithm operation and the correntropy criterion as the loss function, the proposed algorithm is robust to impulsive noise and achieves satisfactory performance in various situations. In addition, the stability problem is studied theoretically. Experimental results are given to demonstrate the validity of the new algorithm in different scenarios.

Introduction

With the development of the Internet of Things (IoT), there is a growing concern about distributed signal processing over Wireless Sensor Networks (WSNs). The WSNs usually consist of a large number of distributed, small nodes that are constrained by limited power and wireless bandwidth. Moreover, these networks have a broad range of practical applications: area-monitoring applications, industrial automation, measuring phenomena, etc [1], [2], [3]. Distributed estimation is of great importance for distributed signal processing [4], [5], [6]. The aim of distributed estimation is to assess some parameters of interest against noise measurements with cooperation between nodes over networks. Distributed estimation algorithms are mainly divided into three categories: incremental strategies [7], consensus strategies [8] and diffusion strategies [9], [10], [11]. Diffusion strategies for distributed estimation over networks are especially of great significance since they are robust, flexible and accurate [12], [13]. For the above reasons, we pay more attention to the diffusion strategies for distributed estimation here.

Most prior algorithms for distributed estimation on diffusion strategies have been derived by the well-known mean square error (MSE), such as diffusion Least Mean Square (d-LMS) [14], [15], the diffusion normalized least-mean-square algorithm (d-NLMS) [16], and the diffusion Recursive Least Squares (d-RLS) [17]. These algorithms can achieve ideal performance if the assumption about the noise signal with a Gaussian distribution is met. However, in real-world applications, this assumption is not often available. Compared to Gaussian noise, these algorithms based on MSE are extremely sensitive to non-Gaussian noise, especially outliers (impulsive noise) [18], [19], [20]. Therefore, algorithms for distributed estimation over networks may deteriorate dramatically with outliers, such as impulsive noise.

To address these issues, diffusion strategies for distributed estimation have been proposed, such as diffusion least-mean power (d-LMP) and diffusion sign-error Least Mean Square (dSE-LMS) [21], [22]. In recent years, Chen and his co-workers conduct in-depth research and propose several novel algorithms based on kernel method in order to deal with non-Gaussian noise, and these methods provide a nonlinear measure of similarity between two random variables [23], [24]. The diffusion maximum correntropy criterion (d-MCC) algorithm is an efficient kernel method based on information theoretic learning (ITL), and can handle non-Gaussian noise effectively [25]. Compared to correntropy, generalized correntropy and the minimum kernel risk-sensitive loss (MKRSL) provide more flexible frameworks and show better performance in many problems such as parameter estimation [26], [27], [28], [29], [30]. There are also some other effective methods proposed to resist impulsive noise, such as maximum versoria criterion-based robust adaptive filtering algorithm [31], a family of robust M-shaped error weighted least mean square algorithms [32], and a family of robust adaptive filtering algorithms based on sigmoid cost [33].

For the anti-impulsive noise algorithms discussed above, the ATC-DSE-LMS algorithm is flexible and effective without introducing any signal delay. But a lower stable-state error performance is the main defect of the robust algorithm. While the D-LMP is one of the most classical robust diffusion algorithms, the robustness of the D-LMP algorithm and its varieties are highly affected by the parameter p. Based on the instantaneous gradient-descent method to minimize the cost function, the diffusion Huber-based NLMS (d-NHuber LMS) algorithm was proposed [34]. However, some important parameters controlling the algorithm robustness should be tuned in accordance with the practitioner knowledge of the noise distribution, which may not always be available.

A robust algorithm based on a diffusion strategy is proposed in this paper for the purpose of improving the estimate performance over networks in various environments. The algorithm combines the logarithm operation and generalized correntropy criterion to construct a new cost function, which is proven to be an effective measure for rejecting outliers (or impulsive noise). Furthermore, inspired by the basic implementation of the diffusion strategy, namely the adapt-then-combine (ATC) d-LMS algorithm [14], the algorithm is designed for distributed estimation over networks. Simulations confirm the advantages of the proposed algorithm under Gaussian noise and its robustness even to impulsive noise. Additionally, the algorithm has also been verified for a nonlinear model and nonstationary environment, where it still maintains its desirable performance. The main contributions of this paper are summarized as follows: (i) A new cost function is designed, which can provide the anti-noise capacity. (ii) A robust diffusion algorithm based on the new designed cost function is derived and shows superior performance for distributed estimation. (iii) The stability of the proposed d-GCLD algorithm is analyzed. (iv) Simulation results are demonstrate the proposed algorithm has potential applications in WSNs, where the sensors may suffer from various situations, including Gaussian and impulsive noise.

This paper is organized as follows. In Section 2, we briefly introduce some preliminary knowledge about the data model and diffusion LMS. The diffusion generalized correntropy logarithmic difference (d-GCLD) algorithm is formulated in Section 3. In Section 4, the theoretical stability analysis of the proposed algorithm is proven. In Section 5, simulation results are presented to show the advantages of the proposed algorithm. Finally, we state the conclusions in Section 6.

Notation: In this paper, (.)^T denotes transposition, and E[.] stands for expectation. sign(.) and ⊗ is sign and Kronecker product operators, respectively. I_m denotes an m × m identity matrix. 1 is an N × 1 all-unity vector. We use Tr{.} to calculate the trace of a matrix and |.| is the absolute value of a scalar.

Section snippets

Data model

Let us consider a connected network with N nodes. At time i, each node k communicates only with its neighborhood N_k. The node collects a scalar data d_k,i and random measurement vectors $u_{k, i}^{T}$ . Our goal is to estimate a vector w^o of length M from the background noise v_k. The data model of node k can be depicted as: $d_{k} (i) = u_{k, i}^{T} w^{o} + v_{k} (i)$ where v_k(i) is the background noise with variance $σ_{v, k}^{2}$ and each node has a different value of v_k(i). According to the general settings in signal processing, the

Proposed algorithm

The basic concept of generalized correntropy is first introduced in this section. Moreover, the diffusion generalized correntropy logarithmic difference (d-GCLD) algorithm for distributed estimation is proposed.

The correntropy criterion was proposed by Liu et.al. [35], [36], as an important part of ITL. Correntropy is a nonlinear measure of the similarity of random processes. Given the random variables X, and Y, the correntropy criterion is defined by $V (X, Y) = E [〈 Φ (X), Φ (Y) 〉] = E [κ_{σ} (X, Y)]$ where κ_σ(.)

Performance analysis

In this section, the stability of the diffusion generalized correntropy logarithmic difference algorithm is theoretically analyzed. To perform tractable analysis, we use the following assumptions to facilitate the performance analysis:

Assumption 1

The input vector u_k,i is independently and identically distributed (i.i.d.). Moreover, $E [u_{k, i} u_{_{k, i}}^{T}] = R_{u, k}$ .

Assumption 2

The noise v_{k, i} is independent of u_k,i for all k and i, and independent of v_{l, j} for l ≠ k or i ≠ j. Additionally, the noise v_{k, i} is a mixture signal of zero

Simulation

In this section, we first evaluate the simulation results of the d-GCLD algorithm over networks under Gaussian as well as Impulsive noise signals. Moreover, the proposed algorithm is also verified over a non-stationary environment and a non-linear system. During the simulations, 1000 independent Monte Carlo trials are performed to obtain averaged results.

The noise v_k(i) is modeled as a summation of two independently zero-mean Gaussian noise [38], [44], [45] as $v_{k} (i) = ν_{1, i} + b_{i} ν_{2, i}$ where ν_1,i is the

Conclusion

In this article, a robust diffusion algorithm for distributed estimation over networks is proposed, called d-GCLD algorithm. This algorithm combines the “logarithmic function” and “generalized correntropy” as its cost function. We also investigate the stability of the proposed algorithm. Through theoretical analysis, the stability condition of the d-GCLD algorithm is derived. Numerical simulations show that the d-GCLD algorithm can achieve superior performance than related algorithms in both

CRediT authorship contribution statement

Xinyu Li: Software, Data curation, Writing - original draft. Mingyu Feng: Writing - review & editing. Feng Chen: Conceptualization, Methodology. Qing Shi: Visualization, Software, Validation. Jurgen Kurths: Investigation, Supervision.

Declaration of Competing Interest

We declare that we have no financial and personal relationships with other people or organizations that can inappropriately influence our work, there is no professional or other personal interest of any nature or kind in any product, service and/or company that could be construed as influencing the position presented in, or the review of, the manuscript entitled” “Robust distributed estimation based on a generalized correntropy logarithmic difference algorithm over wireless sensor networks”.

References (49)

W. Huang et al.
Diffusion fused sparse LMS algorithm over networks
Signal Process.
(2020)
F. Chen et al.
Broken-motifs diffusion LMS algorithm for reducing communication load
Signal Processing
(2017)
F. Chen et al.
Diffusion least logarithmic absolute difference algorithm for distributed estimation
Signal Process.
(2018)
S. Ashkezari-Toussi et al.
Robust diffusion LMS over adaptive networks
Signal Process.
(2019)
B. Chen et al.
Mixture correntropy for robust learning
Pattern Recognit.
(2018)
W. Ma et al.
Diffusion maximum correntropy criterion algorithms for robust distributed estimation
Digit. Signal Process.
(2016)
F. Chen et al.
Diffusion generalized maximum correntropy criterion algorithm for distributed estimation over multitask network
Digit Signal Process.
(2018)
F. Huang et al.
A family of robust adaptive filtering algorithms based on sigmoid cost
Signal Process
(2018)
L. Xiao et al.
Fast linear iterations for distributed averaging
Syst. Control Lett.
(2004)
W. Huang et al.
Distributed maximum a posteriori estimation under non-stationary condition
Inf. Sci. (Ny)
(2019)

I. Akyildiz et al.

A surevy on sensor networks

IEEE Commun. Mag.

(2002)

K. Romer et al.

The design space of wireless sensor networks

IEEE Wirel. Commun.

(2004)

J. Chen et al.

Distributed collaborative control for industrial automation with wireless sensor and actuator networks

IEEE Trans. Ind. Electron.

(2010)

L. Zhang et al.

Joint optimization of dimension assignment and compression in distributed estimation fusion

IEEE Trans. Signal Process.

(2019)

Z. Wang et al.

Optimum distributed estimation of a spatially correlated random field

IEEE Trans. Signal Inf. Process. Netw.

(2019)

V. Vahidpour et al.

Partial diffusion Kalman filtering for distributed state estimation in multiagent networks

IEEE Trans. Neural Netw. Learn. Syst.

(2019)

C.G. Lopes et al.

Incremental adaptive strategies over distributed networks

IEEE Trans. Signal Process.

(2007)

G. Soatti et al.

Consensus-based algorithms for distributed network-state estimation and localization

IEEE Trans. Signal Inf. Process. Netw.

(2017)

Y. Yu et al.

Robust distributed diffusion recursive least squares algorithms with side information for adaptive networks

IEEE Trans. Signal Process.

(2019)

S. Modalavalsa et al.

Diffusion minimum generalized rank norm over distributed adaptive networks: Formulation and performance analysis

IEEE Trans. Signal Inf. Process. Netw.

(2019)

J. Chen et al.

Diffusion adaptation strategies for distributed optimization and learning over networks

IEEE Trans. Signal Process.

(2012)

S.Y. Tu et al.

Diffusion strategies outperform consensus strategies for distributed estimation over adaptive networks

IEEE Trans. Signal Process.

(2012)

F.S. Cattivelli et al.

Diffusion LMS strategies for distributed estimation

IEEE Trans. Signal Process.

(2010)

X. Zhao et al.

Performance limits for distributed estimation over LMS adaptive networks

IEEE Trans. Signal Process.

(2012)

Cited by (14)

Diffusion robust algorithm based on inverse hyperbolic sine and generalized entropy
2024, Digital Signal Processing: A Review Journal
This paper proposes a new robust cost function, which combines the inverse hyperbolic sine cost function and the generalized maximum correlation entropy. Based on this cost function, a robust diffusion generalized correntropy-based inverse hyperbolic sine (DGCIHS) algorithm is proposed. The mean-square performance and convergence analysis are performed to theoretically prove the reliability of the algorithm. Simulation experimental results confirm the theoretical analysis, demonstrating the algorithm's robustness. Compared with similar algorithms, the robust diffusion generalized maximum correlation entropy inverse hyperbolic sinusoidal algorithm demonstrates superior performance.
A diffusion strategy for robust distributed estimation based on streaming graph signals
2023, ISA Transactions
The problem of robust distributed estimation over dynamic and streaming graph signals is investigated in this paper. Existing works related to distributed estimation over dynamic and streaming graph signals are mainly derived from the Mean-Square-Error criterion, and they are vulnerable to non-Gaussian noise. Therefore, a new kind of diffusion Mixture correntropy (d-MC) algorithm is developed to deal with the disadvantage in this paper. Incorporating the diffusion strategy and a novel cost function, the proposed algorithm could accurately estimate the graph filter parameter with the dynamic and streaming graph signals, and achieve desirable performance under both Gaussian and impulsive noise environment. Besides, the theoretical analysis results of mean and mean-square stability are derived. Simulations on various case studies indicate the desirable performance of proposed d-MC algorithm by comparing it to other benchmarks.
Management of the optimizer's curse concept in single-task diffusion networks
2023, Information Sciences
In most optimization problems, loss functions assign the minimum cost to the excellent/ideal sample with a minimum error magnitude. However, this sample may have been mistakenly deemed excellent due to noise. This can lead to an anomaly known as the “optimizer's curse”. Our goal is to manage this anomaly in single-task diffusion networks by giving almost the same importance to a set of good samples instead of just the excellent/ideal one. For this purpose, we introduce a novel loss function and propose a “robust diffusion smooth σ-insensitive correntropy” algorithm for distributed estimation. The proposed convex and smooth loss function is somehow related to the ϵ-insensitive loss family and also inherits some beneficial properties of the correntropy loss function to handle very small and very large errors at once. Accordingly, it provides high resistance to various types of Gaussian, Uniform, and impulsive non-Gaussian noises. Moreover, there are theoretical discussions of the convergence and stability of the proposed method, its steady-state and transient behavior, its computational complexity, and an explanation of the parameter values. Simulation results show the superior performance of the presented algorithm in both stationary and non-stationary environments compared to some recent diffusion-based methods in the literature.
Adaptive multitask clustering algorithm based on distributed diffusion least-mean-square estimation
2022, Information Sciences
Citation Excerpt :
With the performance improvement of sensors, wireless sensor networks (WSNs) are becoming more and more popular in many types of researches [18,20,24].
Distributed estimation with multitask clustering in wireless sensor networks has always been a challenging and attractive topic. When the multitask estimation problem becomes complex and general, such as more complex parameters and more general networks, many multitask clustering algorithms may experience significant performance degradation. To overcome these situations, a highly robust multitask clustering algorithm is proposed in this paper, which can be applied to more general and complex multitask estimation problems. The proposed algorithm is considered as a hybrid system. One subsystem is used to provide a reference estimation, which is further used to perform multitask clustering. Based on the clustering network, the other subsystem is performed to get a distributed adaptive estimation. In addition, each node is considered as a relaying node, so it can obtain the relaying information across the network for enhancing the performance. The convergence performance of the proposed algorithms is analyzed in the mean and mean-square senses. Simulation results demonstrate the robustness and accuracy of the proposed algorithms as compared to other multitask clustering algorithms.
Simplified augmented cubature information filtering and multi-sensor fusion for additive noise systems
2022, Aerospace Science and Technology
Citation Excerpt :
However, as the number of sensors increases, the centralized model imposes a heavy computational burden on the processing center [5,19,20]. To handle this problem, a decentralized information fusion model is presented [21,22], in which the observation information from every sensor is processed locally, and all the processed information is transformed to the processing center for updating the global state [23]. Ge et al. [24] combined the ensemble scheme with centralized and distributed frameworks and the proposed ensemble multi-sensor fusion method obtained better filtering accuracy compared to conventional nonlinear filters, and further Ge et al. indicated that the centralized and distributed frameworks can achieve a comparable filtering efficiency.
For a highly nonlinear system with additive process and observation noises, the non-augmented sigma-point nonlinear filter induces a loss of odd-order moment information (skewness), thereby resulting in a degradation of state estimation accuracy. To address this problem, we present a novel filtering algorithm, namely augmented cubature information filter (ACIF). The adopted augmentation strategy can facilitate the estimator to capture and propagate higher odd-order moment information of random variables. In addition, the covariance resulting from linearization errors is compensated based on the proposed filtering framework. Further, the ACIF is extended to the decentralized multi-sensor system. The employed modified state augmentation strategy can eliminate the inconsistency of covariance estimation induced by the state-mean samples. And the propagation of cubature points is simplified to reduce the computational cost associated with state augmentation. To validate the proposed algorithm, a comparative study is performed via Monte Carlo simulation in the scenario of tracking a maneuvering target. The results show that the simplified ACIF can improve the filtering efficiency without dramatically increasing the computational burden.
Diversity-based diffusion robust RLS using adaptive forgetting factor
2021, Signal Processing
In this study, we propose a diffusion robust recursive least squares (D-R $^{2}$ LS) algorithm over adaptive networks. Instead of conventional mean square error cost function, the suggested method is derived from the maximum correntropy criterion (MCC) cost function, being more suitable for non-Gaussian noise. Furthermore, to improve tracking ability when encountering sudden changes in unknown systems in non-stationary environments, a diversity-based extension of D-R $^{2}$ LS is developed by adaptive forgetting factor for each node. Also, to conduct performance analysis, we employ a half-quadratic optimization to approximate our model iteratively by a quadratic problem. The mean, mean-square convergence and stability of the D-R $^{2}$ LS are discussed theoretically. The simulation results show that the proposed methods outperform the other robust algorithms and enhance tracking quality in the presence of non-Gaussian noise in the stationary and non-stationary environments.

View all citing articles on Scopus

^☆: This work was supported in part by the Fundamental Research Funds for the Central Universities (XDJK2020B034, SWU019029)

View full text

Robust distributed estimation based on a generalized correntropy logarithmic difference algorithm over wireless sensor networks☆

Highlights

Abstract

Introduction

Section snippets

Data model

Proposed algorithm

Performance analysis

Simulation

Conclusion

CRediT authorship contribution statement

Declaration of Competing Interest

Signal Process.

Signal Processing

Signal Process.

Signal Process.

Pattern Recognit.

Digit. Signal Process.

Digit Signal Process.

Signal Process

Syst. Control Lett.

Inf. Sci. (Ny)

A surevy on sensor networks

IEEE Commun. Mag.

The design space of wireless sensor networks

IEEE Wirel. Commun.

Distributed collaborative control for industrial automation with wireless sensor and actuator networks

IEEE Trans. Ind. Electron.

Joint optimization of dimension assignment and compression in distributed estimation fusion

IEEE Trans. Signal Process.

Optimum distributed estimation of a spatially correlated random field

IEEE Trans. Signal Inf. Process. Netw.

Partial diffusion Kalman filtering for distributed state estimation in multiagent networks

IEEE Trans. Neural Netw. Learn. Syst.

Incremental adaptive strategies over distributed networks

IEEE Trans. Signal Process.

Consensus-based algorithms for distributed network-state estimation and localization

IEEE Trans. Signal Inf. Process. Netw.

Robust distributed diffusion recursive least squares algorithms with side information for adaptive networks

IEEE Trans. Signal Process.

Diffusion minimum generalized rank norm over distributed adaptive networks: Formulation and performance analysis

IEEE Trans. Signal Inf. Process. Netw.

Diffusion adaptation strategies for distributed optimization and learning over networks

IEEE Trans. Signal Process.

Diffusion strategies outperform consensus strategies for distributed estimation over adaptive networks

IEEE Trans. Signal Process.

Diffusion LMS strategies for distributed estimation

IEEE Trans. Signal Process.

Performance limits for distributed estimation over LMS adaptive networks

IEEE Trans. Signal Process.