Concentration inequalities for additive functionals: A martingale approach

Abstract

This work shows how exponential concentration inequalities for additive functionals of stochastic processes over a finite time interval can be derived from concentration inequalities for martingales. The approach is entirely probabilistic and naturally includes time-inhomogeneous and non-stationary processes as well as initial laws concentrated on a single point. The class of processes studied includes martingales, Markov processes and general square integrable càdlàg processes. The general approach is complemented by a simple and direct method for martingales, diffusions and discrete-time Markov processes. The method is illustrated by deriving concentration inequalities for the Polyak–Ruppert algorithm, SDEs with time-dependent drift coefficients “contractive at infinity” with both Lipschitz and squared Lipschitz observables, some classical martingales and non-elliptic SDEs.

Introduction

In this work we consider concentration inequalities for additive functionals of the form

$${\int }_0^T{X}_{t}dt$$

where $X$ is a real-valued stochastic process. The methods we develop apply to a broad class of processes, and we will give theorems and examples that go beyond the classical setting of stationary Markov processes. We will treat in depth the cases where $X$ is a martingale or $X_t=f(t,Y_t)$ for a Markov process $Y$ and $f$ in an appropriate class of functions. The concentration inequalities will be derived for the additive functionals centered around their expectation, which allows us to naturally treat non-stationary processes such as time-inhomogeneous diffusions.

We proceed to give an overview of the main results. In Section 2 we derive some representative results using short, self-contained proofs based on direct calculations. First, we show in Proposition 2.1 that for any continuous local martingale $X$ such that $X_0=0$ we have the following concentration inequality for any $T,R>0$:

$$\mathbb{P}\left({\int }_0^T{X}_{u}du\ge R;{\int }_0^T{(T-u)}^{2}d{\left[X\right]}_u\le {\sigma }^2\right)\le exp\left(-\frac{R^2}{2{\sigma }^2}\right).$$

To the author’s knowledge, the systematic treatment of concentration inequalities for additive functionals of martingales has not appeared in the literature before.

We will then move on to solutions to SDEs of the form

$$d{X}_t=b(t,X_t)dt+\sigma d{B}_t,$$

with $X_0$ deterministic. In particular, denoting $P_{s,t}$ the Markov transition operator associated to $X$, we will show in Corollary 2.7 that if there exist constants $c,\kappa >0$ such that

$$|{\sigma }^{\top }\nabla P_{t,u}f\left(x\right)|\le c{e}^{-\kappa (u-t)},x\in {\mathbb{R}}^n,0\le t\le u$$

then we have the following Gaussian concentration inequality for all $R,T>0$:

$$\mathbb{P}\left(\frac{1}{T}{\int }_0^{T}f(u,X_u)du-\mathbb{E}\left[\frac{1}{T}{\int }_0^{T}f(u,X_u)du\right]\ge R\right)\le exp\left(-\frac{{\kappa }^2{R}^{2}T}{2c^2}\right).$$

The main novelty in the corollary is the treatment of time-inhomogeneous SDEs and the method of proof. The Proposition from which the corollary is derived also provides a novel, refined statement in terms of a bound of $|{\sigma }^{\top }\nabla P_{t,u}f|$ along trajectories of $X$.

Section 2 concludes with the case of discrete-time processes in Proposition 2.12, again giving careful consideration to the time-inhomogeneous case and controlling the relevant quantities along trajectories of the process. Concretely, we show that for a discrete-time stochastic process $X_t$ and function $f$ such that

$$|X_t-X_{t-1}|\le C_t,t\ge 1,|P_{s,t}f\left(x\right)-P_{s,t}f\left(y\right)|\le {\sigma }_s{(1-{\kappa }_s)}^{t-s}|x-y|,x,y\in {\mathbb{R}}^n,0\le s\le t.$$

we have

$$\mathbb{P}\left(\sum _{u=1}^{t}f(u,X_u)-\mathbb{E}\left[\sum _{u=1}^{t}f(u,X_u)\right]\ge R;\sum _{t=1}^T\frac{{\sigma }_t^2{C}_t^2}{{\kappa }_t^2}\le a^2\right)\le exp\left(-\frac{R^2}{8a^2}\right).$$

The careful treatment of the time-inhomogeneous case and control on the level of trajectories enables in particular for the first time the derivation of concentration inequalities for the Polyak–Ruppert algorithm of the correct order in Section 4.1 (concentration inequalities for the linear case were published concurrently with this work in [30]).

Section 3 is dedicated to a wide-ranging generalization of the results from the previous section. In Section 3.1 we introduce a family of auxiliary martingales $Z_t^u=E^{{\mathcal{F}}_t}{X}_u$ and show that for general square integrable processes, the concentration properties of ${\int }_0^T{X}_{u}du$ are intimately linked to the predictable quadratic covariation $〈Z^u,Z^v〉$ and jumps $\Delta Z^u$ of the auxiliary martingales. In Section 3.2 we recover and extend the martingale results from Section 2 to the discontinuous setting using the general method. In Section 3.3 we apply the general method to general Markov processes and recover expressions for $〈Z^u,Z^v〉$ in terms of the squared field operator $\Gamma$, generalizing the results on Markov processes from Section 2 to general Markov processes on Polish spaces. All of the results from the preceding subsections are novel in their generality. We conclude Section 3 with Section 3.4 where we show how to incorporate arbitrary distributions of $X_0$ and recall a number of martingale inequalities. The subsection also includes in Corollary 3.16 a novel approach to obtain Bernstein-type inequalities in some “self-bounding” cases.

The final Section 4 illustrates how to apply the results from the preceding section on a number of concrete cases. Section 4.1 contains the novel results on Polyak–Ruppert mentioned above.

Section 4.2 provides a concrete example of an SDE case with explicit conditions on the drift and diffusion coefficients as well as the observable function. Using known results on gradient bounds for $P_{s,t}$ when the drift coefficient is “contractive at infinity” characterized by constants $\rho ,\kappa$, we show that for any initial law $\nu$ satisfying a $T_1\left(C\right)$ transport inequality and Lipschitz function $f$, we have

$$\mathbb{P}\left(\right.\frac{1}{T}{\int }_0^{T}f\left(X_t\right)dt-\frac{1}{T}{\int }_0^T{\mu }_t\left(f\right)dt\ge R+\rho \Vert f{\Vert }_{\mathrm{Lip}}\frac{1-e^{-\kappa T}}{\kappa T}{W}_1({\mu }_0,\nu )\left.\right)\le exp\left(-\frac{{\kappa }^2{R}^{2}T}{2{\rho }^2\Vert f{\Vert }_{\mathrm{Lip}}^2\left(1+C\frac{1-e^{-\kappa T}}{T}\right)}\right)$$

for a unique evolution system of measures ${\mu }_t$ (if the process has a stationary measure $\mu$ then ${\mu }_t=\mu$ for all $t$).

Section 4.3 provides some concrete examples using classical martingales as integrands: Brownian motion, the compensated Poisson process and compensated squared Brownian motion $B_t^2-t$.

Sections 4.4 Squared Ornstein–Uhlenbeck process, 4.4 Squared Ornstein–Uhlenbeck process treat the cases where the integrand $X$ is either the squared Ornstein–Uhlenbeck process or more generally the square of a Lipschitz function. These cases go beyond the scope of most previously published approaches to concentration inequalities for additive functionals. The final Section 4.6 presents a simple case of a highly non-elliptic SDE, which yields easily to the probabilistic methods presented here but is outside of the scope of previous approaches based for example on Poisson equations.

About the literature. In the Markovian setting, our approach is most closely related to the work of Joulin [25], and we recover and extend the results from that work (Proposition 3.7, Proposition 4.10). The cases of martingales and general square integrable processes do not seem to have been systematically studied in the literature.

Most previous results on concentration inequalities for functionals of the form $S_t$ have been obtained for time-homogeneous Markov processes using functional inequalities. The works [7], [19], [21] require the existence of a stationary measure and an initial distribution that has an integrable density with respect to the stationary measure. The same holds true for the combinatorial and perturbation arguments in the classic paper [28]. In [40] the authors establish concentration inequalities around the expectation using stochastic calculus and Girsanov’s theorem under strong contractivity conditions. Some concentration inequalities for inhomogeneous functionals have previously been established in [20]. A different approach using renewal processes has been used in the work [29] to establish concentration inequalities for functionals with bounded integrands.

For Markov processes, the mixing conditions in this work are most naturally formulated in terms of bounds on either the Lipschitz seminorm, gradient or squared field (carré du champs) operator of the semigroup. Bounds on the Lipschitz seminorm are closely related to contractivity in the $L^1$ transportation distance and can for instance be found in [1], [15] for elliptic diffusions, in [39] for the Riemannian case, in [16] for Langevin dynamics or in [23] for stochastic delay equations. See also [26], [31] for the discrete-time case and a large number of examples in both discrete and continuous time. Gradient estimates for semigroups can be obtained using Bismut-type formulas, see for example [12], [17], [36], the works [9], [10] for the non-autonomous case as well as the textbook [37]. Finally, in terms of the squared field operator, our mixing conditions are a relaxation of the Bakry–Emery curvature-dimension condition [2] since we allow for a prefactor strictly greater than $1$.

Notation. For a right-continuous process ${\left(X_t\right)}_{t\ge 0}$ with left limits we write $X_{t-}={lim}_{\epsilon \to 0^{+}}{X}_{t-\epsilon }$ and $\Delta X_t=X_t-X_{t-}$. For a $\sigma$-field $\mathcal{F}$ and a random variable $X$, we denote ${\mathbb{E}}^{\mathcal{F}}X$ the conditional expectation of $X$ with respect to $\mathcal{F}$.