A statistical approach to signal denoising based on data-driven multiscale representation

https://doi.org/10.1016/j.dsp.2020.102896Get rights and content

Highlights

  • A procedure is introduced to estimate the distribution of noise from within the noisy signal based on VMD.

  • Use of the EDF statistics based statistical (CVM) distance is introduced for detection of the predominantly noisy BLIMFs.

  • A fully data-driven statistical signal estimation method is proposed by exploiting the VMD based multiscale representation.

  • Extensive experiments demonstrate the efficacy and utility of the proposed work in practical applications.

Abstract

We develop a data-driven approach for signal denoising that utilizes variational mode decomposition (VMD) algorithm and Cramer Von Misses (CVM) statistic. In comparison with the classical empirical mode decomposition (EMD), VMD enjoys superior mathematical and theoretical framework that makes it robust to noise and mode mixing. These desirable properties of VMD materialize in segregation of a major part of noise into a few final modes while majority of the signal content is distributed among the earlier ones. To exploit this representation for denoising purpose, we propose to estimate the distribution of noise from the predominantly noisy modes and then use it to detect and reject noise from the remaining modes. The proposed approach first selects the predominantly noisy modes using the CVM measure of statistical distance. Next, CVM statistic is used locally on the remaining modes to test how closely the modes fit the estimated noise distribution; the modes that yield closer fit to the noise distribution are rejected (set to zero). Extensive experiments demonstrate the superiority of the proposed method as compared to the state of the art in signal denoising and underscore its utility in practical applications where noise distribution is not known a priori.

Introduction

Signals from various practical applications are subject to unwanted noise due to various physical limitations of acquisition systems, e.g., audio recording systems, lidar systems, EEG and ECG acquisition systems etc. Hence, to avoid any false decisions based on these noisy signals, it is necessary to remove the unwanted noise beforehand. In this regard, it is customary to assume the noise in time series data follows the additive white Gaussian noise (wGn) model. That problem of removal of additive wGn has been optimally solved for wide sense stationary signals, i.e., signals with perfectly known invariable statistics, using the Weiner filter. However, that approach may not be adequate in practical settings due to the following reasons. Firstly, majority of real life signals are nonstationary in that their attributes (statistics) change with time. Secondly, the assumed wGn model may not always be used to characterize noise in time series data, e.g., EEG/ECG signals. Consequently, more evolved techniques capable of accounting for the non-stationarity of signal and non-Gaussianity of noise are required.

Discrete wavelet transform (DWT) is a multiscale method to process the non-stationary signals that exhibits property of sparse distribution of signal singularities within its coefficients. The noise coefficients, on the other hand, have lower amplitudes and uniform spread [1]. That allows to differentiate between signal and noise coefficients using a suitable threshold (e.g., universal threshold [2]) or statistical shrinkage functions, e.g., [3], [4]. The foundations of these methods are built on prior assumptions about the (distribution) models of (true) signal and noise. In this regard, random noise due to data-acquisition and communication systems is conventionally modeled using the additive wGn model but that do not fully account for the factors contributing to the noise during the acquisition. However, specifying a signal model is challenging owing to the arbitrary nature of information generally found within the times series data. Moreover, specification of prior models restricts the efficacy of these methods in real world signals.

This issue has been partially addressed within the framework presented in [5] that combines DWT with the goodness of fit (GoF) test. Hereafter, this approach is called as DWT-GoF method. It is worth mentioning that the DWT-GoF method requires only a prior noise model. Here, noise is expediently modeled as a zero-mean additive wGn that is, for example. The detection of wGn at multiple wavelet scales is facilitated by the fact that the DWT preserves the Gaussianity of noise. This essentially requires detection and rejection of wavelet coefficients fitting the Gaussian distribution for denoising. Henceforth, the DWT-GoF method [5] rejects noise from DWT scales by estimating the GoF of Gaussian distribution on the multiscale coefficients. An improved version of the DWT-GoF method has been proposed in [6], [7] that employ GoF test along with the dual-tree complex wavelet transform (DTCWT), which is called the DT-GOF-NeighFilt method in the sequel. The key feature of the DT-GOF-NeighFilt method is to incorporate a novel neighborhood filtering technique to minimize the loss of signal details while rejecting the noise. Apart from the GoF test, other hypothesis testing tools such as False discovery rate (FDR), Bayesian local false discovery rate (BLFDR) are also used in combination with wavelet transforms for signal denoising [8].

Another avenue for multiscale denoising involves data-driven decomposition techniques. For instance, empirical mode decomposition (EMD) [9] that employs a data-driven approach to extract the principal oscillatory modes from a signal. Owing to EMD's ability to expand a signal into its inherent intrinsic mode functions (IMFs), EMD is considered well suited for processing the non-stationary signals generally encountered in practice. When employed for denoising, EMD aims at detecting the IMFs representing the (oscillatory) signal parts and rejecting the IMFs corresponding to the non-oscillatory noise. A wavelet-inspired interval-thresholding function is used for detecting the oscillatory signal parts from the noisy IMFs [10]. Specifically, the EMD-based interval thresholding (EMD-IT) [10] aims to detect the oscillations separated by two consecutive zero crossings. This is achieved by comparing the extrema of an interval against a threshold value leading to either retention or rejection of the whole interval.

Instead of performing thresholding, the work in [11] employed statistical tools to detect the relevant (signal) modes for a partial reconstruction of the denoised signal. However, these denoising approaches may result in suboptimal performance due to the mode mixing (i.e., manifestation of multiple IMFs within a single IMF) property of EMD and its sensitivity to noise and sampling. Essentially, the aforementioned shortcomings within EMD framework result in leakage of noise into a few signal modes which leads to their rejection resulting in suboptimal denoising. The lack of mathematical foundation of the EMD limits the chances of rectification of these issues within its framework.

The recently proposed variational mode decomposition (VMD) is based on optimization of a variational problem to obtain an ensemble of a fixed number of band limited IMFs (BLIMFs) [12]. Owing to its sound mathematical foundation, VMD successfully avoids mode mixing and is robust to noise and sampling unlike EMD [12], [13]. From the view point of denoising, a very important feature of VMD is its ability to segregate the desired signal into a few initial BLIMFs while noise is mostly stashed into a few final BLIMFs. Hence, by rejecting the modes with noise, a good estimate of the true signal may be obtained by partial reconstruction.

A literature review shows that the existing VMD-based denoising approaches select relevant signal modes by comparing the probability distribution function (PDF) of an individual BLIMF against the PDF of the noisy signal. This is well founded because a distribution function is generally reflective of the signal present within the noisy data. An estimate of the signal present in a BLIMF may be obtained by measuring the closeness of its PDF with that of the noisy signal, for example, by employing Euclidean distance [14], Bhatacharya distance [15], etc. Therein, the modes statistically close to the noisy signal are selected as relevant (signal) modes while largely dissimilar modes are rejected as noise. For a detailed study on the efficacy of various statistical distances for estimating relevant modes, the interested reader is referred to [16]. Moreover, the method in [17] selects relevant modes using the detrended fluctuation analysis (DFA) (originally used with EMD within the EMD-DFA method [11]) that estimates the randomness of data by observing the lack of trend. This method, hereafter referred as VMD-DFA [17], rejects the noisy BLIMFs and reconstructs the denoised signal based on the remaining modes.

The problem with these VMD-based methods is (a) the presence of noise within the selected (low frequency) signal modes that results in significant artifacts in the denoised signal and (b) the loss of high frequency signal details as part of the rejected modes leading to substandard denoising. This issue has not received much attention from researchers which may be because the interval thresholding techniques [10], [18], [19], widely used with EMD/MEMD, are not directly applicable to the VMD/MVMD modes owing to the arbitrary spread of noise power across the BLIMFs. On the other hand, the design of a wavelet like statistical thresholding technique for the BLIMFs of VMD requires prior knowledge of the distribution of noise (and/or signal). That is challenging because VMD, owing to its non-linearity, transforms a known noise distribution (e.g., Gaussian distribution) to an unknown one. Thus, discouraging the possibility of development of a statistical method for signal estimation.

To address this issue, we propose a novel statistical approach to signal denoising that exploits the nature of multiscale representation obtained using VMD to empirically estimate the distribution of noise residing in the BLIMFs of the noisy signal. This estimated noise distribution is then used to develop a nonlinear statistical signal estimation method using Cramer Von Mises (CVM) statistic from the BLIMFs of the VMD. Moreover, CVM statistic is also used to select the relevant signal modes which, in our case, involve the BLIMFs containing signal (with or without noise). That is a departure from the existing definition of relevant modes where only low frequency signal-only modes are selected to reconstruct an approximation of the original signal. Since our method selects all the modes containing signal, it yields a much closer estimate of the original signal because of its ability to reject noise from the selected (signal plus noise) modes using the CVM based thresholding function. We next list the main contributions of this work as follows:

  • Development of an empirical procedure to estimate noise distribution model from within the noisy signal that is facilitated by the robustness of the VMD to noise and mode mixing that culminates in effective segregation of noise from the true-signal into a few final modes.

  • Development of a signal estimation method based on the noise distribution that checks how closely the estimated noise distribution fits the local segments of the selected relevant signal BLIMFs.

  • Use of the robust empirical distribution function (EDF)-based CVM distance for selecting the relevant signal modes ahead of the distances based on crude approximations of the probability density function (PDF).

To validate the performance of the proposed method, extensive computer simulations have been carried out for denoising a variety of benchmark signals corrupted by artificially generated Gaussian noise. Furthermore, the efficacy of the proposed method is demonstrated by denoising a (real) ECG signal corrupted by actual (non-Gaussian) sensor noise.

The rest of paper is organized as follows: Section 2 provides the background of the CVM statistic required to explain the proposed methodology that is subsequently presented in Section 3. Section 4 reports experiments analyzing the performance of our proposed method while Section 5 presents a few practical denoising examples. Finally, conclusion along with the future prospects of this work are discussed in Section 6.

Section snippets

Cramer Von Mises (CVM) statistic

CVM statistic [20] belongs to a class of statistical distances [21] that estimate how closely a dataset or observations follow a given distribution function. In this regard, the CVM statistic requires an estimate of the distribution of given observations, that is obtained using the EDF. It is worth mentioning that EDF happens to be a robust model of distribution even for small-sized data and is easy to compute [22]. More importantly, EDF is a discrete approximation of the cumulative

Description of proposed approach

Consider the signal modely(t)=x(t)+ψ(t), where y(t), x(t) and ψ(t) denote the noisy, true signal, and additive noise component, each of length N. It is customary to assume ψ(t) being modeled as N(0,σ), i.e., zero-mean wGn process with variance σ2. However, the additive noise in the real-life signals may be non-Gaussian. Therefore, the existing denoising approaches developed on the assumption of wGn may have a limited scope for the practical signals. In this work, we propose to address the noise

Simulation results and discussion

This section demonstrates the effectiveness of the performance of the proposed VMD-CVM method against the state of the art in signal denoising. In this regard, the used comparative state of the art methods includes a good mix of wavelet and data driven decomposition (i.e., EMD and VMD) based methods, given as follows:

  • DTCWT-Thr [30]: Employs neighborhood based thresholding along with the quasi-translation invariant DTCWT.

  • BLFDR [8]: Performs noise shrinkage by employing the false discovery rate

Denoising ECG signal corrupted by sensor noise

In this section, we present denoising results of the proposed method when applied to an ECG signal corrupted by the sensor noise. The raw ECG signal in this regard is taken from [32] that is corrupted by actual sensor noise (during acquisition) with unknown distribution model. To further exaggerate the noise we added an additive wGn of at input SNR =20 dB to account for the thermal noise component due to electronic devises. As a result, the noise mostly obscures the useful information within

Conclusions and discussion

In this paper, we have addressed the problem of noise removal from the practical signals whereby the noise is considered to be governed by unknown probability distribution. We propose to exploit the desirable properties of VMD to estimate the EDF of noise from within the noisy signal. As stated earlier, VMD possesses ability to segregate the signal and noise in separate group of BLIMFs owing to its robustness to noise and mode mixing. First, we detect the group of BLIMFs predominantly composed

CRediT authorship contribution statement

Khuram Naveed: Conceptualization, Methodology, Experiments, Writing - original draft, Formal analysis, Programming, Software, Investigation, Visualization. Muhammad Tahir Akhtar: Conceptualization, Writing - reviewing and editing, Supervision, Visualization. Muhammad Faisal Siddiqui: Methodology, Writing - reviewing and editing, Formal analysis, Investigation, Visualization. Naveed ur Rehman: Conceptualization, Methodology, Experiments, Supervision, Writing - reviewing and editing, Formal

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgements

Authors would like to acknowledge anonymous reviewers for providing many critical and insightful comments on the manuscript. This has greatly helped in improving the contents as well as quality of presentation. The work of Dr. Akhtar has been partially funded from the Faculty Development Competitive Research Grants Program of Nazarbayev University under the Grant Number 110119FD4525.

Khuram Naveed received his PhD degree in 2020 from COMSATS University Islamabad (CUI), Islamabad, Pakistan. He is currently working as a lecturer in the Department of Electrical and Computer Engineering at CUI Islamabad where he has also worked as research associate from May 2011 to Jan 2014. He was a visiting researcher at Tandon School of Engineering, New York University (NYU), New York USA from Jan 2018 – July 2018. His research interests include signal and image processing using multiscale

References (32)

  • K. Naveed et al.

    Signal denoising based on dual tree complex wavelet transform and goodness of fit test

  • I. Lavrik et al.

    Bayesian false discovery rate wavelet shrinkage: theory and applications

    Commun. Stat., Simul. Comput.

    (2008)
  • N.E. Huang et al.

    The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis

    Proc. R. Soc. Lond. A, Math. Phys. Eng. Sci.

    (1998)
  • Y. Kopsinis et al.

    Development of emd-based denoising methods inspired by wavelet thresholding

    IEEE Trans. Signal Process.

    (2009)
  • K. Dragomiretskiy et al.

    Variational mode decomposition

    IEEE Trans. Signal Process.

    (2014)
  • M. Ali et al.

    Hybrid multiscale wind speed forecasting based on variational mode decomposition

    Int. Trans. Electr. Energy Syst.

    (2018)
  • Cited by (25)

    • Asymptomatic COVID-19 CT image denoising method based on wavelet transform combined with improved PSO

      2022, Biomedical Signal Processing and Control
      Citation Excerpt :

      The sub-band weighting aims to better improve the image representation ability and adaptively remove noise in the image, and it has a good noise coefficient representation ability. Reference [13] proposed the framework of dual tree complex wavelet transform (DTCWT). DTCWT overcomes the defects of wavelet transform.

    • EDLA-EFDS: A Novel Ensemble Deep Learning Approach For Electrical Fault Detection Systems

      2022, Electric Power Systems Research
      Citation Excerpt :

      This can be explained due to the fact that all the descriptive statistics of the collected data such as mean and standard deviation are sensitive to noise which can cause tests to either miss significant findings or distort real results. Therefore, the most effective solution in this case is filtering data from noise [48]. The better understanding of the concept of noise in data, the better data cleaning we can have.

    • Enhanced symplectic geometry mode decomposition and its application to rotating machinery fault diagnosis under variable speed conditions

      2022, Mechanical Systems and Signal Processing
      Citation Excerpt :

      Finally, pseudo components identification strategies remain to be developed to make the signal decomposition method more robust to interferences. Motivated by the superiority of Cramer-von Mises (CVM) statistic [38,39] in distribution similarity evaluation for components reconstruction and surrogate data test [40] in pseudo components identification against noise, an enhanced symplectic geometry mode decomposition (ESGMD) method is proposed in this paper to address the above mentioned obstacles for rotating machinery fault diagnosis under variable speed conditions. Firstly, the vibration signal is low-pass filtered and down-sampled to narrow the frequency range and reduce signal points for computation speed acceleration.

    • NSTBNet: Toward a nonsubsampled shearlet transform for broad convolutional neural network image denoising

      2022, Digital Signal Processing: A Review Journal
      Citation Excerpt :

      Recently, a variety of denoising methods were proposed and showed admirable denoising performance [1–4].

    View all citing articles on Scopus

    Khuram Naveed received his PhD degree in 2020 from COMSATS University Islamabad (CUI), Islamabad, Pakistan. He is currently working as a lecturer in the Department of Electrical and Computer Engineering at CUI Islamabad where he has also worked as research associate from May 2011 to Jan 2014. He was a visiting researcher at Tandon School of Engineering, New York University (NYU), New York USA from Jan 2018 – July 2018. His research interests include signal and image processing using multiscale and multi-component signal decomposition methods. His recent research focuses on the design of multivariate signal processing techniques and machine learning applications.

    Muhammad Tahir Akhtar received his PhD degree in Electronic Engineering from the Tohoku University, Sendai, Japan, in 2004, MSc in Systems Engineering from the Quaid-i-Azam University, Islamabad, Pakistan in 1999, and BSc Electrical (Electronics and Communication) Engineering from the University of Engineering & Technology, Taxila, Pakistan in 1997.

    He is currently working as an Associate Professor at the School of Engineering and Digital Sciences, Nazarbayev University, Nur-Sultan City, Kazakhstan. From 2014 to 2017, he was an Associate Professor at the COMSATS University Islamabad, Pakistan. From 2008 to 2014, he was an Assistant Professor at the University of Electro-Communications, Tokyo, Japan, and a Special Visiting Researcher at the Tokyo Institute of Technology, Tokyo, Japan. He was a visiting researcher at Institute of Sound and Vibration Research (ISVR), University of Southampton, UK (Dec. 2008-Feb. 2009), and at Institute for Neural Computations (INC), University of California San Diego (Nov. 2010-Mar. 2011). Prior to that, he was a COE postdoctoral fellow at Tohoku University, Sendai Japan (2004-2005), and has worked as an Assistant Professor at the United Arab Emirates University, UAE (2006-2008).

    His research interests include adaptive signal processing, active noise control, blind source separation, and biomedical signal processing. Dr. Akhtar has published about 95 papers in the peer-reviewed international journals and conference proceedings. He won Best Student Paper at the IEEE 2004 Midwest Symposium on Circuits and Systems, Hiroshima, Japan, and student paper award (with Marko Kanadi) at 2010 RISP International Workshop on Nonlinear Circuits, Communications and Signal Processing. He is on the editorial board of Advances in Mechanical Engineering, and has served (2011-2013) as a co-editor for the newsletter of Asia-Pacific Signal & Information Processing Association (APSIPA). Currently he is member IEEE Signal Processing Society, IEEE Industrial Electronic Society, and he is a Senior Member IEEE.

    Muhammad Faisal Siddiqui has been working as an Assistant Professor at Department of Electrical & Computer Engineering, COMSATS University Islamabad, Pakistan since July 2016. He is a Computer Engineer with expertise in Digital System Design and Medical Imaging. He received his PhD from Electrical Engineering Department, Faculty of Engineering, University of Malaya, Kuala Lumpur, Malaysia. He received MS (EE) and BS (CE) degrees from COMSATS, Institute of Information Technology, Islamabad, Pakistan. His research interests are FPGA based digital system designing, biomedical image processing, embedded systems, application specific architectural design and real-time systems.

    Naveed ur Rehman obtained his PhD from Imperial College London in 2011 and is currently working as Assistant Professor in the Department of Engineering at Aarhus University, Denmark. His research interests mainly include data-driven multi-scale signal decomposition methods, time-frequency analysis, optimization based methods in signal processing and applications of machine learning.

    View full text