Change-point detection in hierarchical circadian models
Introduction
Change-point detection (CPD) consists of locating abrupt transitions in the generative model of a sequence of observations. Detecting these abrupt transitions or change points (CPs) has been long studied and appears in a vast amount of real-world scenarios, for instance, investment strategies, the analysis of social networks or cognitive radio in signal processing.
The problem of CPD can be faced from a statistical point of view, which typically requires a model defined a priori, or alternatively, using model-free methods. The statistical approaches can be further divided between frequentist and Bayesian techniques. The main idea behind frequentist methods is to derive a measure, usually based on likelihood ratio tests [1], [2], between the pre-change and post-change distributions and compare it with a threshold. On the other hand, Bayesian approaches are based on assigning a prior distribution over the change points, or a proper surrogate, and derive its posterior [3]. Typically, these last models focus on batch settings with unknown number of changes [4]. In contrast, online methods have been proposed based on particle filters [5] or the Bayesian online change-point detection (BOCPD) model of [6].
In this work we focus on a novel application of CPD, namely, the problem of modeling and detecting changes in human behavior. CPD can be used to detect behavioral changes, which in patients with prevalent chronic disorders (e. g., schizophrenia or chronic depression) may be signals of future relapses [7]. Other works, like [8], study the rapid detection of suicidal behavior based on pose representations of patients. Here, among the alternatives to monitor psychiatric patients’ symptoms, we consider smartphone-based monitoring [9]. The ubiquitous presence of smartphones is the key point and the large amount of data gathered by these devices opens new opportunities in this area. For instance, smartphone observations range from inertial sensors measurements or GPS information to duration of calls or app usage log. However, all these monitoring traces pose some challenges that have not been considered in detail within the CPD framework. First, the observations provided by smartphones are high-dimensional and heterogeneous. Second, since humans are strictly conditioned by 24-hour periods, also known as circadian rhythm, samples usually present some underlying periodicities [10]. Finally, since some of these sensors may fail or be turned off, it is common to have missing values in the data.
A large portion of CPD algorithms proposed in the literature do not consider the aforementioned features of data. Concretely, the general assumption of CPD models is to consider low-dimensional and homogenous datasets [5], [6]. However, these models will not work as the detection of behavioral changes using smartphones usually involve observations that are high-dimensional and heterogeneous, i.e., they are composed by a mixture of continuous, categorical, binary or discrete variables. Typically, heterogenous observations are difficult to deal with [11], since an improper learning process would yield a system that mainly uses only one of the different data types. Moreover, the CPD problem becomes even more difficult when observations possess temporal structure, that is, they deviate from the independent and identically distributed (i. i. d.) assumption.
This case is also studied in this work, where there exists complex temporal structure, which in our setting is induced by the circadian rhythm. The works [12], [13] also consider CPD on non-stationary sequential data, but our model captures periodic patterns, similarly to the approach in Tsai and Lai [14].
This paper addresses the problem using a Bayesian CPD approach. This sort of methods provides a general measure of uncertainty, which is critical for us, due to the need of reliable decision-making algorithms in the mental-health context. There has been a considerable effort in Bayesian estimation of change points on different domains. For example, Höhle [15] introduced an online cumulative sum detector that finds changes on categorical time series. Alternative approaches are based on mixture models [16], [17], hidden Markov models (HMM) [18], nearest neighbor methods [19] or support vector machines (SVM) [20].
We generalize the BOCPD algorithm [6] to address the challenges raised by the application of CPD in psychiatry. Several extensions of the BOCPD algorithm exist as well, including adaptive sequential methods [21] or the Gaussian Process BOCPD model [22]. The proposed generalization of the BOCPD algorithm considers the CPD problem on heterogeneous and high-dimensional sequences through hierarchical models within latent variables. In Section 2, the hierarchical extension of the BOCPD method is described. First, we propose a general latent variable model and, later on, due to the complexity induced by arbitrary latent variables, we particularize it to categorical ones. This is also well justified by the proposed application since these categorical latent variables represent what we could call type of day, i.e., one day could correspond to a typical workday, another one to a weekend, etc. Thus, the proposed method is actually detecting changes in the distributions of such types of days. For instance, a change point should be detected if the category that represents typical workdays disappears. Moreover, for the hierarchical model we also present the inference procedure and a simplified model, as well as an approach to consider missing data. The association between observations and types of days is presented in Section 3, where a clustering technique based on mixtures of Gaussians is described. Interestingly, the covariance matrix of the Gaussian random vectors will be designed to capture the circadian features induced by human behavior. We develop the inference based on the expectation-maximization (EM) algorithm [23], which also allows us to handle missing data. Finally, the performance of the proposed method is evaluated in Section 4 over synthetic experiments and real data acquired by smartphones.
Section snippets
Change-point detection in hierarchical models
This section generalizes the method proposed in Adams and MacKay [6] to consider hierarchical models, which allow us to deal with high-dimensional observations. The work in Adams and MacKay [6] considers that the sequence of observations, may be divided into non-overlapping partitions separated by change points. With a few exceptions, such as [22], the work in Adams and MacKay [6] and its generalizations assume that the data within each partition is i. i. d. according to some generative
Heterogeneous circadian models
We now study how to embed heterogeneous models into the hierarchical BOCPD technique presented before, as well as to handle periodic temporal structures. Motivated by the application to human behavior characterization, we consider data that possess a periodic temporal structure, with 24-hours period, which is induced by the circadian rhythm. However, the ideas presented here are valid for any dataset with known periodicity.
To account for the periodic dependencies, we propose to arrange the data
Experiments
In this section, we evaluate the performance of the proposed method with synthetic and real data, and also compare it with that of state-of-the-art detectors.
Conclusions
In this paper we have introduced a novel generalization for Bayesian change-point detection methods to handle heterogeneous high-dimensional observations with unknown periodic structure. The proposed technique makes two contributions. The first one, denoted as the hierarchical detector, is a probabilistic extension of the widely known Bayesian online change-point detection (BOCPD) algorithm presented in Adams and MacKay [6]. We extended the method to accept any type of latent variable model as
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
This work was supported by the Ministerio de Ciencia, Innovación y Universidades under grant TEC2017-92552-EXP (aMBITION), by the Ministerio de Ciencia, Innovación y Universidades, jointly with the European Commission (ERDF), under grants TEC2017-86921-C2-2-R (CAIMAN) and RTI2018-099655-BI00 (CLARA), by the Comunidad de Madrid under grant Y2018/TCS-4705 (PRACTICO-CM), and by Fundación BBVA under project Deep-DARWIN. The work of P. Moreno-Muñoz has been supported by FPI grant BES-2016-077626.
Pablo Moreno-Muñoz obtained his B.Sc. and M.Sc in Telecommunication Engineering from Universidad Carlos III de Madrid, Spain in 2014 and 2016, respectively. In 2015, he held a 6-month traineeship at European Space Agency for the investigation of probabilistic methods applied to distance calculations in astronomy. He is currently Ph.D. student at the Dept. of Signal Theory and Communications also at the Universidad Carlos III de Madrid. During the last years, he has been research visitor at the
References (31)
- et al.
Intelligent video surveillance for real-time detection of suicide attempts
Pattern Recognit. Lett.
(2018) - et al.
A least-squares approach to anomaly detection in static and sequential data
Pattern Recognit. Lett.
(2014) - et al.
Principles of time–frequency feature extraction for change detection in non-stationary signals: applications to newborn EEG abnormality detection
Pattern Recognit.
(2015) - et al.
Defect detection in periodically patterned surfaces using independent component analysis
Pattern Recognit.
(2008) Bayesian change detection based on spatial sampling and Gaussian mixture model
Pattern Recognit. Lett.
(2011)- et al.
Proportional data modeling with hidden Markov models based on generalized Dirichlet and Beta-Liouville mixtures applied to anomaly detection in public areas
Pattern Recognit.
(2016) - et al.
Parameter k search strategy in outlier detection
Pattern Recognit. Lett.
(2018) - et al.
On-line anomaly detection and resilience in classifier ensembles
Pattern Recognit. Lett.
(2013) Procedures for reacting to a change in distribution
Ann. Math. Stat.
(1971)- et al.
Optimal detection of changepoints with a linear computational cost
J. Am. Stat. Assoc.
(2012)
Detection of Abrupt Changes: Theory and Application
Exact and efficient Bayesian inference for multiple changepoint problems
Stat. Comput.
On-line inference for multiple changepoint problems
J. R. Stat. Soc.
Combining continuous smartphone native sensors data capture and unsupervised data mining techniques for behavioral changes detection: a case series of the evidence-based behavior (eB2) study
JMIR mHealth uHealth
Cited by (0)
Pablo Moreno-Muñoz obtained his B.Sc. and M.Sc in Telecommunication Engineering from Universidad Carlos III de Madrid, Spain in 2014 and 2016, respectively. In 2015, he held a 6-month traineeship at European Space Agency for the investigation of probabilistic methods applied to distance calculations in astronomy. He is currently Ph.D. student at the Dept. of Signal Theory and Communications also at the Universidad Carlos III de Madrid. During the last years, he has been research visitor at the University of Sheffield, UK and the Max Planck Institute for Intelligent Systems in Tübingen, Germany. His research interests include probabilistic machine learning methods for heterogeneous data, Gaussian processes, change-point detection, continual Bayesian inference, and its application to human behaviour modelling in medicine.
David Ramírez (S’07-M’12-SM’16) received the Telecommunication Engineer degree and the Ph.D. degree in electrical engineering from the University of Cantabria, Santander, Spain, in 2006 and 2011, respectively. From 2006 to 2011, he was with the Communications Engineering Department, University of Cantabria. In 2011, he joined as a Research Associate with the University of Paderborn, Germany, and later on, he became an Assistant Professor (Akademischer Rat). He is currently an Associate Professor with the University Carlos III of Madrid. He was a Visiting Researcher with the University of Newcastle, Australia and with the University College London. He was the recipient of the 2012 IEEE Signal Processing Society Young Author Best Paper Award and the 2013 extraordinary Ph.D. award of the University of Cantabria. Moreover, he is currently an Associate Editor for the IEEE Transactions on Signal Processing. Finally, he is a member of the IEEE Technical Committee on Signal Processing Theory and Methods and was Publications Chair of the 2018 IEEE Workshop on Statistical Signal Processing.
Antonio Artés-Rodríguez(S’89-M’93-SM’01) was born in Alhama de Alméra, Spain, in 1963. He received the Ingeniero de Telecomunicación and Doctor Ingeniero de Telecomunicación degrees, both from the Universidad Politecnica de Madrid, Madrid, Spain, in 1988 and 1992, respectively. He is a Professor at the Department of Signal Theory and Communications, Universidad Carlos III de Madrid, Madrid. Prior to this, he held different teaching positions at Universidad de Vigo, Universidad Politecnica de Madrid, and Universidad de Alcalá, all of them in Spain. He has participated in more than 70 projects and contracts and has coauthored more that 50 journal papers and more than 100 international conference papers. His research interests include signal processing, machine learning, and information theory methods, and its application to health and sensor networks.