Randomness is inherently imprecise

Abstract

We use the martingale-theoretic approach of game-theoretic probability to incorporate imprecision into the study of randomness. In particular, we define several notions of randomness associated with interval, rather than precise, forecasting systems, and study their properties. The richer mathematical structure that thus arises lets us, amongst other things, better understand and place existing results for the precise limit. When we focus on constant interval forecasts, we find that every sequence of binary outcomes has an associated filter of intervals it is random for. It may happen that none of these intervals is precise—a single real number—which justifies the title of this paper. We illustrate this by showing that randomness associated with non-stationary precise forecasting systems can be captured by a constant interval forecast, which must then be less precise: a gain in model simplicity is thus paid for by a loss in precision. But imprecise randomness can't always be explained away as a result of oversimplification: we show that there are sequences that are random for a constant interval forecast, but never random for any computable (more) precise forecasting system. We also show that the set of sequences that are random for a non-vacuous interval forecasting system is meagre, as it is for precise forecasting systems.

Introduction

This paper documents the first steps in our attempt to incorporate imprecision into the study of algorithmic randomness. What this means is that we want to allow for, give a precise mathematical meaning to, and study the mathematical consequences of, associating randomness with interval rather than precise probabilities and expectations. We will see that this is a non-trivial problem, argue that it leads to surprising conclusions about the nature of randomness, and discover that it opens up interesting and hitherto uncharted territory for mathematical and even philosophical investigation. We believe that our work provides (the beginnings of) a satisfactory answer to questions raised by a number of researchers [19], [20], [22], [61] about frequentist and ‘objective’ aspects of interval, or imprecise, probabilities.

To explain what it is we're after, consider an infinite sequence $\omega =(z_1,\dots ,z_n,\dots )$, whose components $z_k$ are either zero or one, and are typically considered as successive outcomes of some experiment. When do we call such a sequence random? There are many notions of randomness, and many of them have a number of equivalent definitions [1], [4]. We will focus here essentially on Martin-Löf randomness, computable randomness, and Schnorr randomness.

The randomness of a sequence ω is typically associated with a probability measure on the sample space of all such infinite sequences, or—which is essentially equivalent due to Ionescu Tulcea's extension theorem [5, Theorem II.9.2]—with a so-called forecasting system φ that associates with each finite sequence of outcomes $(x_1,\dots ,x_n)$ the (conditional) expectation $\phi (x_1,\dots ,x_n)=E(X_{n+1}|x_1,\dots ,x_n)$ for the next, as yet unknown, outcome $X_{n+1}$.^1,2 This $\phi (x_1,\dots ,x_n)$ is the (precise) forecast for the value of $X_{n+1}$ after observing the values $x_1,\dots ,x_n$ of the respective variables $X_1,\dots ,X_n$. The sequence ω is then typically called ‘random’ when it passes some countable number of randomness tests, where the collection of such randomness tests depends of the forecasting system φ.

An alternative and essentially equivalent approach to defining randomness, going back to Ville [54], sees each forecast $\phi (x_1,\dots ,x_n)$ as a fair price for—and therefore a commitment to bet on—the as yet unknown next outcome $X_{n+1}$ after observing the first n outcomes $x_1,\dots ,x_n$. The sequence ω is then ‘random’ when there is no ‘allowable’ strategy for getting infinitely rich by exploiting the bets made available by the forecasting system φ along the sequence, without borrowing. Betting strategies that are made available by the forecasting system φ are called supermartingales. Which supermartingales are considered ‘allowable’ differs in various approaches [1], [4], [18], [28], [39], but typically involves some (semi)computability requirement—we discuss relevant aspects of computability in Section 4. Technically speaking, randomness then requires that all allowable non-negative supermartingales (that start with unit value) should remain bounded on ω.

It is this last, martingale-theoretic, approach that seems to lend itself most easily to allowing for interval rather than precise forecasts, and therefore to allowing for ‘imprecision’ in the definition of randomness. As we explain in Sections 2 and 3, an interval, or ‘imprecise’, forecasting system φ associates with each finite sequence of outcomes $(x_1,\dots ,x_n)$ a (conditional) expectation interval $\phi (x_1,\dots ,x_n)$ for the next, as yet unknown, outcome $X_{n+1}$. The lower bound of this interval forecast represents a supremum acceptable buying price, and its upper bound an infimum acceptable selling price, for the next outcome $X_{n+1}$. This idea rests firmly on the common ground between Walley's [60] theory of coherent lower previsions and Shafer and Vovk's [45], [46] game-theoretic approach to probability that we have helped establish in recent years, through our research on imprecise stochastic processes [13], [16]; see also Refs. [2], [53] for more details on so-called ‘imprecise probabilities’. These theoretical developments allow us here to associate supermartingales with an interval forecasting system, and therefore in Section 5 to extend a number of existing notions of randomness to allow for interval, rather than precise, forecasts: we include in particular Martin-Löf randomness and computable randomness [1], [4], [18], [39]. In Section 6, we also extend Schnorr randomness [1], [4], [18], [39] to allow for interval forecasts. We then show in Section 7 that our approach allows us to extend to interval forecasting some of Dawid's [7] well-known work on calibration, and to establish a number of interesting ‘limiting frequencies’ or computable stochasticity results.

We believe the discussion becomes especially interesting in Section 8, where we start restricting our attention to constant, or stationary, interval forecasts. We see this as an extension of the more classical accounts of randomness, which typically consider a forecasting system with constant forecast $\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.$—corresponding to flipping a fair coin. As we have by now come to expect from our experience with so-called imprecise probability models, when we allow for interval forecasts, a mathematical structure appears that is much more interesting than the rather simpler case of precise forecasts would lead us to suspect. In the precise case, a given sequence may not be random for any stationary forecast, but as we will see, in the case of interval forecasting there typically is a filter of intervals that a given sequence is random for. Furthermore, as we show in Section 9 by means of explicit examples, this filter may not have a smallest element, and even when it does, this smallest element may be a non-vanishing interval: this is the first cornerstone for our argument that randomness is inherently imprecise.

The examples in Section 9 all involve sequences that are random for some computable non-stationary precise forecast, but can't be random for a stationary forecast unless it becomes interval-valued, or imprecise. This might lead to the suspicion that this imprecision is perhaps only an artefact, which results from looking at non-stationary phenomena through an imperfect stationary lens. We show in Section 10 that this suspicion is unfounded: there are sequences that are random for a stationary interval forecast, but that aren't random for any computable (more) precise forecast, be it stationary or not. This further corroborates our claim that randomness is, indeed, inherently imprecise.

Finally, in Section 11, we argue that ‘imprecise’ randomness is an interesting extension of the existing notions of ‘precise’ randomness, because it is equally rare: just as for precise stationary forecasts, the set of all sequences that are random for a non-vacuous stationary interval forecast is meagre. This, we will argue, indicates that the essential distinction lies not between precise and imprecise forecasts (or randomness), but between non-vacuous and vacuous ones, and provides further evidence for the essentially ‘imprecise’ nature of the randomness notion.

We conclude with a short discussion of the significance of our findings, and of possible avenues for further research. In order to maintain focus, we have decided to move all technical proofs of auxiliary results about computability and growth functions to an appendix. We have also, as much as possible, tried to make sure that our more complicated and technical proofs in the main text are preceded by informal arguments, in order to help the reader build some intuition about why and how they work.