Abstract
Gaussian processes (GPs) are used widely in the analysis of astronomical time series. GPs with rational spectral densities have state-space representations which allow evaluation of the likelihood. We calculate analytic state space representations for the damped simple harmonic oscillator and the Matérn 1/2, 3/2 and 5/2 processes.
Export citation and abstract BibTeX RIS
Original content from this work may be used under the terms of the Creative Commons Attribution 4.0 licence. Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI.
1. Introduction
Gaussian processes (GPs) are now widely used in the analysis of astronomical time series. Efficient algorithms have been devised that allow likelihood evaluation in operations, where n is the number of observations (see Foreman-Mackey et al. 2017, and references therein). Linear Gaussian state space models allow likelihood evaluation in operations via the Kalman filter (Kalman 1960). In this note we present analytic linear Gaussian state space model representations for the popular Matérn and Damped Simple Harmonic Oscillator (DSHO) GPs, offering an alternative route to efficient likelihood evaluation for these processes. This representation can offer advantages in terms of numerical stability for the Matérn case (D. Foreman-Mackey 2021, personal communication).
There is a correspondence between a stationary GP over with kernel and power spectral density that is a rational function of ω, i.e., with p and q polynomials, and the prior implied by an order p linear stochastic differential equation given by
where W is a Wiener process (Brownian motion) with variance (Hartikainen & Sarkka 2010; Saatçi 2012; Brockwell & Davis 2013). Equation (1) is equivalent to a state space model with the following state equation
where , and
and the following measurement equation
where and bj =0 for . The observed process y would in the astronomical context used to model a time series y(t) observed at times with associated measurement errors .
If is stationary the process y defined by the above state space model can be shown (e.g., Brockwell 2009) to be a zero-mean Gaussian CARMA() model with power spectral density
and variance that satisfies
We can express the process given by Equation (2) as a linear Gaussian state space model observed at a set of input points
where , and
where is the measurement error, is the last column of the matrix and is a projection matrix that selects the first component of the vector, which is the process in our state space model. With all the conditional probabilities of our process determined, we can perform inference using a filtering algorithm.
A filtering algorithm computes the conditional probabilities of the states given all previous measurements and a set of parameters , i.e., . For the linear Gaussian case this can done in operations with a set of recursive equations known as the Kalman filter, which are given by 4
The mean and covariance matrices of the distributions above are obtained with the Kalman filter prediction and update steps (note that the matrices can depend on although we have not written explicitly that dependence). The prediction step is given by
and the update step by
The recursion is initialized with a prior mean and variance . To learn the parameters of a GP that has a corresponding state space representation we are interested in the marginal likelihood which can be computed recursively as
where we define . The terms in the product are part of the output of the Kalman filter (see Equation (7)). Thus, the marginal likelihood can be calculated in operations. Armed with the likelihood, we can sample the posterior using a variety of techniques. We note that the use of the Kalman filter to evaluate the likelihood of a CARMA() model in the astronomical context has been presented before in Kelly et al. (2014). In their work the likelihood for a CARMA() process is evaluated numerically using the algorithm of Jones & Ackerson (1990) which relies on the diagonalization of the matrix using the roots of the a(z) polynomial. In this note we provide analytic expressions for the and matrices for some GP kernels of wide astronomical application.
2. State Space Models Corresponding to GP Kernels of Interest
2.1. The Matérn Family
One of the most widely used kernels is the Matérn family, which are indexed by ν and have an associated spectral density for a one-dimensional input given by Rasmussen & Williams (2006)
and is thus of the form of Equation (3) for , . More precisely, we have and . As an example, for the popular kernel the corresponding stochastic differential equation is
where and the variance of dW is given in terms of and λ by
in order that the power spectra density of the process corresponds to that of Equation (11) for .
For the Matérn family with the matrices and defined in Equation (6) can be computed analytically using the Laplace transform and a symbolic mathematics package. We provide the expressions of these matrices for a set of values of ν below.
2.2. Stochastically Driven Damped Simple Harmonic Oscillator
Foreman-Mackey et al. (2017) proposed the stochastic process given by the following stochastic differential equation (SDE) as a physically motivated process to describe stellar photometric variations
The corresponding power spectral density is given by
where is the spectral density of the white noise process dW. In terms of the power at we have that , and in terms of the total power we have 5 that .
From Equation (13) we see that the DSHO is equivalent to a CARMA(2,0) process with and . The corresponding vectorial Markovian SDE is
where . Just as the case of the Matérn family, the matrices and defined in Equation (6) can be computed analytically. Note that when this process reduces to a Matérn 3/2 process with .
3. Calculation of Matrices needed for Likelihood Calculation
We follow the procedure described in Saatçi (2012) to calculate analytic expressions for the matrices needed for the likelihood calculation for the Matérn and DSHO processes. The matrices are calculated using the identity
where denotes the inverse Laplace transform and is the identity matrix. In the case of the Matérn family of processes, the matrices can be computed analytically via the integral given in Equation (6). In the case of the DSHO, we use the matrix fraction decomposition. If we define matrices and such that , then Equation (4) is satisfied when and take the form
and . To calculate the matrices for the DSHO we use Equation (17) with and , calculate the matrix exponential using the inverse Laplace transform, and then set . Note that we set to reduce the notational clutter, matrices presented here should be multiplied by the actual value of this variance when using them. We also denote for the damped simple harmonic oscillator simply by ω.
3.1. Matérn (Exponential or Ohrstein–Uhlenbeck Process)
3.2. Matérn
3.3. Matérn
3.4. DSHO with Parameters ω and Q. Case .
Define .
3.5. DSHO with Parameters ω and Q. Case .
Define . The expressions for and correspond to those of the case using the fact that and .
Note that the and matrices for the DSHO process have well-defined limits as and , and are given by the corresponding matrices for the Matérn process. This follows from the fact that .
Footnotes
- 4
An excellent introduction to Bayesian filtering is given by Särkkä (2013), from which we take our notation.
- 5
This follows from which is obtained using