Change-point detection in hierarchical circadian models

https://doi.org/10.1016/j.patcog.2021.107820Get rights and content

Highlights

  • Bayesian change-point detection on sequences of high-dimensional and heterogeneous observations with temporal structure.

  • Heterogeneous-cirdadian mixture models with non-stationary and periodic covariance functions.

  • Maximum-a-posteriori (MAP) detection from low-dimensional time-series of discrete latent variables.

  • Applied experiments to human behavior modelling and detection of changes in mental health patients.

Abstract

This paper addresses the problem of change-point detection in sequences of high-dimensional and heterogeneous observations, which also possess a periodic temporal structure. Due to the dimensionality problem, when the time between change points is of the order of the dimension of the model parameters, drifts in the underlying distribution can be misidentified as changes. To overcome this limitation, we assume that the observations lie in a lower-dimensional manifold that admits a latent variable representation. In particular, we propose a hierarchical model that is computationally feasible, widely applicable to heterogeneous data and robust to missing instances. Additionally, the observations’ periodic dependencies are captured by non-stationary periodic covariance functions. The proposed technique is particularly well suited to (and motivated by) the problem of detecting changes in human behavior using smartphones and its application to relapse detection in psychiatric patients. Finally, we validate the technique on synthetic examples and we demonstrate its utility in the detection of behavioral changes using real data acquired by smartphones.

Introduction

Change-point detection (CPD) consists of locating abrupt transitions in the generative model of a sequence of observations. Detecting these abrupt transitions or change points (CPs) has been long studied and appears in a vast amount of real-world scenarios, for instance, investment strategies, the analysis of social networks or cognitive radio in signal processing.

The problem of CPD can be faced from a statistical point of view, which typically requires a model defined a priori, or alternatively, using model-free methods. The statistical approaches can be further divided between frequentist and Bayesian techniques. The main idea behind frequentist methods is to derive a measure, usually based on likelihood ratio tests [1], [2], between the pre-change and post-change distributions and compare it with a threshold. On the other hand, Bayesian approaches are based on assigning a prior distribution over the change points, or a proper surrogate, and derive its posterior [3]. Typically, these last models focus on batch settings with unknown number of changes [4]. In contrast, online methods have been proposed based on particle filters [5] or the Bayesian online change-point detection (BOCPD) model of [6].

In this work we focus on a novel application of CPD, namely, the problem of modeling and detecting changes in human behavior. CPD can be used to detect behavioral changes, which in patients with prevalent chronic disorders (e. g., schizophrenia or chronic depression) may be signals of future relapses [7]. Other works, like [8], study the rapid detection of suicidal behavior based on pose representations of patients. Here, among the alternatives to monitor psychiatric patients’ symptoms, we consider smartphone-based monitoring [9]. The ubiquitous presence of smartphones is the key point and the large amount of data gathered by these devices opens new opportunities in this area. For instance, smartphone observations range from inertial sensors measurements or GPS information to duration of calls or app usage log. However, all these monitoring traces pose some challenges that have not been considered in detail within the CPD framework. First, the observations provided by smartphones are high-dimensional and heterogeneous. Second, since humans are strictly conditioned by 24-hour periods, also known as circadian rhythm, samples usually present some underlying periodicities [10]. Finally, since some of these sensors may fail or be turned off, it is common to have missing values in the data.

A large portion of CPD algorithms proposed in the literature do not consider the aforementioned features of data. Concretely, the general assumption of CPD models is to consider low-dimensional and homogenous datasets [5], [6]. However, these models will not work as the detection of behavioral changes using smartphones usually involve observations that are high-dimensional and heterogeneous, i.e., they are composed by a mixture of continuous, categorical, binary or discrete variables. Typically, heterogenous observations are difficult to deal with [11], since an improper learning process would yield a system that mainly uses only one of the different data types. Moreover, the CPD problem becomes even more difficult when observations possess temporal structure, that is, they deviate from the independent and identically distributed (i. i. d.) assumption.

This case is also studied in this work, where there exists complex temporal structure, which in our setting is induced by the circadian rhythm. The works [12], [13] also consider CPD on non-stationary sequential data, but our model captures periodic patterns, similarly to the approach in Tsai and Lai [14].

This paper addresses the problem using a Bayesian CPD approach. This sort of methods provides a general measure of uncertainty, which is critical for us, due to the need of reliable decision-making algorithms in the mental-health context. There has been a considerable effort in Bayesian estimation of change points on different domains. For example, Höhle [15] introduced an online cumulative sum detector that finds changes on categorical time series. Alternative approaches are based on mixture models [16], [17], hidden Markov models (HMM) [18], nearest neighbor methods [19] or support vector machines (SVM) [20].

We generalize the BOCPD algorithm [6] to address the challenges raised by the application of CPD in psychiatry. Several extensions of the BOCPD algorithm exist as well, including adaptive sequential methods [21] or the Gaussian Process BOCPD model [22]. The proposed generalization of the BOCPD algorithm considers the CPD problem on heterogeneous and high-dimensional sequences through hierarchical models within latent variables. In Section 2, the hierarchical extension of the BOCPD method is described. First, we propose a general latent variable model and, later on, due to the complexity induced by arbitrary latent variables, we particularize it to categorical ones. This is also well justified by the proposed application since these categorical latent variables represent what we could call type of day, i.e., one day could correspond to a typical workday, another one to a weekend, etc. Thus, the proposed method is actually detecting changes in the distributions of such types of days. For instance, a change point should be detected if the category that represents typical workdays disappears. Moreover, for the hierarchical model we also present the inference procedure and a simplified model, as well as an approach to consider missing data. The association between observations and types of days is presented in Section 3, where a clustering technique based on mixtures of Gaussians is described. Interestingly, the covariance matrix of the Gaussian random vectors will be designed to capture the circadian features induced by human behavior. We develop the inference based on the expectation-maximization (EM) algorithm [23], which also allows us to handle missing data. Finally, the performance of the proposed method is evaluated in Section 4 over synthetic experiments and real data acquired by smartphones.

Section snippets

Change-point detection in hierarchical models

This section generalizes the method proposed in Adams and MacKay [6] to consider hierarchical models, which allow us to deal with high-dimensional observations. The work in Adams and MacKay [6] considers that the sequence of observations, X1:t, may be divided into non-overlapping partitions separated by change points. With a few exceptions, such as [22], the work in Adams and MacKay [6] and its generalizations assume that the data within each partition is i. i. d. according to some generative

Heterogeneous circadian models

We now study how to embed heterogeneous models into the hierarchical BOCPD technique presented before, as well as to handle periodic temporal structures. Motivated by the application to human behavior characterization, we consider data that possess a periodic temporal structure, with 24-hours period, which is induced by the circadian rhythm. However, the ideas presented here are valid for any dataset with known periodicity.

To account for the periodic dependencies, we propose to arrange the data

Experiments

In this section, we evaluate the performance of the proposed method with synthetic and real data, and also compare it with that of state-of-the-art detectors.

Conclusions

In this paper we have introduced a novel generalization for Bayesian change-point detection methods to handle heterogeneous high-dimensional observations with unknown periodic structure. The proposed technique makes two contributions. The first one, denoted as the hierarchical detector, is a probabilistic extension of the widely known Bayesian online change-point detection (BOCPD) algorithm presented in Adams and MacKay [6]. We extended the method to accept any type of latent variable model as

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

This work was supported by the Ministerio de Ciencia, Innovación y Universidades under grant TEC2017-92552-EXP (aMBITION), by the Ministerio de Ciencia, Innovación y Universidades, jointly with the European Commission (ERDF), under grants TEC2017-86921-C2-2-R (CAIMAN) and RTI2018-099655-BI00 (CLARA), by the Comunidad de Madrid under grant Y2018/TCS-4705 (PRACTICO-CM), and by Fundación BBVA under project Deep-DARWIN. The work of P. Moreno-Muñoz has been supported by FPI grant BES-2016-077626.

Pablo Moreno-Muñoz obtained his B.Sc. and M.Sc in Telecommunication Engineering from Universidad Carlos III de Madrid, Spain in 2014 and 2016, respectively. In 2015, he held a 6-month traineeship at European Space Agency for the investigation of probabilistic methods applied to distance calculations in astronomy. He is currently Ph.D. student at the Dept. of Signal Theory and Communications also at the Universidad Carlos III de Madrid. During the last years, he has been research visitor at the

References (31)

  • M. Basseville et al.

    Detection of Abrupt Changes: Theory and Application

    (1993)
  • P. Fearnhead

    Exact and efficient Bayesian inference for multiple changepoint problems

    Stat. Comput.

    (2006)
  • P. Fearnhead et al.

    On-line inference for multiple changepoint problems

    J. R. Stat. Soc.

    (2007)
  • R.P. Adams, D.J. MacKay, Bayesian online changepoint detection, 2007....
  • S. Berrouiguet et al.

    Combining continuous smartphone native sensors data capture and unsupervised data mining techniques for behavioral changes detection: a case series of the evidence-based behavior (eB2) study

    JMIR mHealth uHealth

    (2018)
  • Cited by (0)

    Pablo Moreno-Muñoz obtained his B.Sc. and M.Sc in Telecommunication Engineering from Universidad Carlos III de Madrid, Spain in 2014 and 2016, respectively. In 2015, he held a 6-month traineeship at European Space Agency for the investigation of probabilistic methods applied to distance calculations in astronomy. He is currently Ph.D. student at the Dept. of Signal Theory and Communications also at the Universidad Carlos III de Madrid. During the last years, he has been research visitor at the University of Sheffield, UK and the Max Planck Institute for Intelligent Systems in Tübingen, Germany. His research interests include probabilistic machine learning methods for heterogeneous data, Gaussian processes, change-point detection, continual Bayesian inference, and its application to human behaviour modelling in medicine.

    David Ramírez (S’07-M’12-SM’16) received the Telecommunication Engineer degree and the Ph.D. degree in electrical engineering from the University of Cantabria, Santander, Spain, in 2006 and 2011, respectively. From 2006 to 2011, he was with the Communications Engineering Department, University of Cantabria. In 2011, he joined as a Research Associate with the University of Paderborn, Germany, and later on, he became an Assistant Professor (Akademischer Rat). He is currently an Associate Professor with the University Carlos III of Madrid. He was a Visiting Researcher with the University of Newcastle, Australia and with the University College London. He was the recipient of the 2012 IEEE Signal Processing Society Young Author Best Paper Award and the 2013 extraordinary Ph.D. award of the University of Cantabria. Moreover, he is currently an Associate Editor for the IEEE Transactions on Signal Processing. Finally, he is a member of the IEEE Technical Committee on Signal Processing Theory and Methods and was Publications Chair of the 2018 IEEE Workshop on Statistical Signal Processing.

    Antonio Artés-Rodríguez(S’89-M’93-SM’01) was born in Alhama de Alméra, Spain, in 1963. He received the Ingeniero de Telecomunicación and Doctor Ingeniero de Telecomunicación degrees, both from the Universidad Politecnica de Madrid, Madrid, Spain, in 1988 and 1992, respectively. He is a Professor at the Department of Signal Theory and Communications, Universidad Carlos III de Madrid, Madrid. Prior to this, he held different teaching positions at Universidad de Vigo, Universidad Politecnica de Madrid, and Universidad de Alcalá, all of them in Spain. He has participated in more than 70 projects and contracts and has coauthored more that 50 journal papers and more than 100 international conference papers. His research interests include signal processing, machine learning, and information theory methods, and its application to health and sensor networks.

    View full text