Hidden Markov Models for multivariate functional data
Introduction
Hidden Markov Models (HMMs) represent a well-known method for the study of time series involving sequences of data, widely used in many fields like biostatistics (Martino et al., 2020), bioinformatics (Durbin et al., 1998) and finance (Paas et al., 2007). In the literature of HMMs, there are several examples where the outcome consists of univariate or multivariate data, with both discrete and continuous observations; in particular, in Cappé et al. (2006) a very general definition of such processes is provided. In this paper, we want to extend the usual HMM algorithms from the finite dimensional framework to the infinite dimensional one. Therefore, we focus on the functional setting, where each observed data is considered as a multivariate random curve, that can be also seen as the realization of a stochastic process taking values in . Thanks to their versatility, these models can be applied in many research contexts, since lately more and more data are collected as suitable curves observed on a continuum domain. Let us consider, for instance, a collection of biomedical signals, as ECGs and EEGs, measured at different time points for several patients; they can be seen as a collection of multivariate curves evolving in time. With the application of our model to this type of data, we can retrieve some information about the evolution of the shape of the curves, that can lead to determine, for example, the onset of a certain pathology or the rate of transitions among healthy and non-healthy states.
The natural context to develop the statistical models and tools to describe this kind of data is the Functional Data Analysis (FDA) (see, e.g. Ramsay, 2004, Ramsay and Silverman, 2007, Ferraty and Vieu, 2006, Horváth and Kokoszka, 2012). Working with functional data can be a difficult task because of the dimensionality of the spaces of the data; moreover, the usual HMM requires the definition of a probability density that generates the observations, which may be lacking for functional random processes. Therefore, since we want to consider the most general case without making any assumptions on the law of the process that generated the data, we construct a similarity function built on distances between curves to evaluate the emission of an observation by a certain state. We consider a hidden Markov chain evolving in time where each state emits a multivariate random curve and we solve two problems. First, we estimate the parameters of the underlying Markov process to understand the time series system that generated the data; then we solve a clustering problem by finding the best sequence of states that generated the data in order to classify the curves in clusters.
The paper is organized as follows: in Section 2 we present the model, adding some information about the theory of HMMs and functional data. In Section 3 we present a simulation study to assess the performance of the model while in Section 4 we see a case study application to a dataset regarding a climate problem. Finally, in Section 5 we give some discussion and conclusions. All the analysis have been carried out using the statistical software R (R Core Team, 2017) and a package is in development. The codes are available upon request.
Section snippets
The model
The aim of this paper is to consider and study a proper Hidden Markov Model (HMM) in the multivariate functional framework. Let us consider a multivariate random curve , with and compact interval of , as defined in Horváth and Kokoszka (2012). Starting from the definition in Cappé et al. (2006), we define a HMM in the multivariate functional context as a bivariate process on a given probability space , where
- •
is a Markov
Simulation studies
We generate three samples of length of realizations on a grid of points for three independent bivariate random curves in , with . Each sample is emitted from a different state of a 3-state HMM having the following parameters:
- •
State 1: , ;
- •
State 2: , ;
- •
State 3: , .
where is the vector of the initial probabilities of the state, is the
Case study: Weather data
In this last part, we apply the described model to a real dataset regarding the weather in Basel, Switzerland, extracted from the website www.meteoblue.com. In particular, our data consists of daily registrations of temperature and wind speed from 2008 to 2018. We consider each month as an observation of a statistical unit, in order to have 12 multivariate functional observations for every year. First, we apply our algorithm to the weather data with states and compute every time the AIC
Discussion
In this work, we faced the problem of estimating the parameters of a HMM where the output is a multivariate random curve, showing that it is able to detect the underlying structure of the time series system and provide robust results. Then, using the Viterbi algorithm, we also solved a classification problem, noticing that we obtain better results by looking at the time order of the system than by only considering the shape of the curves.
CRediT authorship contribution statement
Andrea Martino: Writing - original draft, Writing - review & editing, Conceptualization, Methodology, Software, Validation, Formal analysis, Data curation, Visualization, Investigation. Giuseppina Guatteri: Writing - review & editing, Conceptualization, Methodology, Supervision, Formal analysis, Project administration. Anna Maria Paganoni: Writing - review & editing, Conceptualization, Methodology, Supervision, Project administration.
References (19)
- et al.
A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains
Ann. Math. Stat.
(1970) A gentle tutorial of the EM algorithm and its application to parameter estimation for Gaussian mixture and hidden Markov models
Int. Comput. Sci. Inst.
(1998)- et al.
Inference in Hidden Markov Models
(2006) - et al.
Maximum likelihood from incomplete data via the EM algorithm
J. R. Stat. Soc. Ser. B Methodol.
(1977) - et al.
Biological Sequence Analysis: Probabilistic Models of Proteins and Nucleic Acids
(1998) - et al.
Nonparametric Functional Data Analysis: Theory and Practice
(2006) - et al.
Inference for Functional Data with Applications, Vol. 200
(2012) - et al.
Roahd package: robust analysis of high dimensional data
The R Journal
(2019) Maximum Likelihood Estimators for Multivariate Hidden Markov Mixture ModelsTechnical Report, Report in Wirtschaftsmathematik 146
(2013)
Cited by (5)
Application of Queuing Model in Library Service
2021, Procedia CIRPFunctional concurrent hidden Markov model
2023, Statistics and ComputingTopological Hidden Markov Models
2022, arXivFault Prediction for Network Devices Using Service Outage Prediction Model
2022, Journal of CommunicationsHidden Markov Model as the Predictive Time Series Movement of Natural Gas Price
2022, Proceeding - 2022 International Symposium on Information Technology and Digital Innovation: Technology Innovation During Pandemic, ISITDI 2022