Detection of data anomalies at the edge of pervasive IoT systems

Abstract

Validation of input data is essential in any computer system, but perhaps particularly important in pervasive IoT systems such as smart homes, smart cars, wearable health monitors, etc. In such systems, actions taken based on invalid inputs could have severe consequences. In this paper, we present statistical techniques for identifying data anomalies at the gateway that connects an edge network to its associated cloud services. We address two kinds of anomalies in environmental sensor data: data bias anomalies and sensor cut-off anomalies. In simulation experiments, we evaluate the effectiveness of applying control charts, a statistical process monitoring technique, to both kinds of anomalies. Our results show that using control charts as statistical methods for anomaly detection in IoT systems not only provides high performance in terms of accuracy and power (probability of detecting the anomaly), but also offers a graphical tool to monitor the IoT sensor data.

Appendix: Mathematical Notations

Symbol	Definition
p	Number of sensors
n	Number of observations
$Y_i$	Observation i of p sensor data streams
${\overline{Y}}$	Sample vector mean
$S^{-1}$	The inverse of the sample variance-covariance matrix
$\hbox {Beta}_{(p/2,n-p-1)}$	beta probability function with shape parameters
$\hbox {F}_{(p,n-p)}$	F probability function with degrees of freedom n and $n-p$