A martingale analysis of first passage times of time-dependent Wiener diffusion models

https://doi.org/10.1016/j.jmp.2016.10.001Get rights and content

Highlights

  • Multistage processes model decisions involving time-varying stimuli.

  • Martingale theory provides derivations of classical reaction time properties.

  • Semi-analytic formulas for mean decision time and hitting probability are derived.

  • These derivations contribute to the study richer decision processes.

Abstract

Research in psychology and neuroscience has successfully modeled decision making as a process of noisy evidence accumulation to a decision bound. While there are several variants and implementations of this idea, the majority of these models make use of a noisy accumulation between two absorbing boundaries. A common assumption of these models is that decision parameters, e.g., the rate of accumulation (drift rate), remain fixed over the course of a decision, allowing the derivation of analytic formulas for the probabilities of hitting the upper or lower decision threshold, and the mean decision time. There is reason to believe, however, that many types of behavior would be better described by a model in which the parameters were allowed to vary over the course of the decision process.

In this paper, we use martingale theory to derive formulas for the mean decision time, hitting probabilities, and first passage time (FPT) densities of a Wiener process with time-varying drift between two time-varying absorbing boundaries. This model was first studied by Ratcliff (1980) in the two-stage form, and here we consider the same model for an arbitrary number of stages (i.e. intervals of time during which parameters are constant). Our calculations enable direct computation of mean decision times and hitting probabilities for the associated multistage process. We also provide a review of how martingale theory may be used to analyze similar models employing Wiener processes by re-deriving some classical results. In concert with a variety of numerical tools already available, the current derivations should encourage mathematical analysis of more complex models of decision making with time-varying evidence.

Introduction

Continuous time stochastic processes modeling a particle’s diffusion (with drift) towards one of two absorbing boundaries have been used in a wide variety of applications including statistical physics (Farkas & Fulop, 2001), finance (Lin, 1998), economics (Webb, 2015), and health science (Horrocks & Thompson, 2004). Varieties of such models have also been applied extensively within psychology and neuroscience to describe both the behavior and neural activity associated with decision processes involved in perception, memory, attention, and cognitive control (Brunton et al., 2013, Diederich and Oswald, 2014, Diederich and Oswald, 2016, Feng et al., 2009, Gold and Shadlen, 2001, Gold and Shadlen, 2007, Heath, 1992, Ratcliff and McKoon, 2008, Ratcliff and Rouder, 1998, Shadlen and Newsome, 2001, Simen et al., 2009); for reviews see (Bogacz et al., 2006, Busemeyer and Diederich, 2010, Ratcliff and Smith, 2004).

In these stochastic accumulation decision models, the state variable x(t) is thought to represent the amount of accumulated noisy evidence at time t for decisions represented by the two absorbing boundaries, that we refer to as the upper (+) and lower () thresholds (boundaries). The evidence x(t) evolves in time according to a biased random walk with Gaussian increments, which may be written as dx(t)Normal(μdt,σ2dt), and a decision is said to be made at the random time τ, the smallest time t for which x(t) hits either the upper threshold (x(τ)=+ζ) or the lower threshold (x(τ)=ζ), also known as the first passage time (FPT). The resulting decision dynamics are thus described by the FPT of the underlying model. In studying these processes one is often interested in relating the mean decision time and the probability of hitting a certain threshold (e.g. the probability of making a certain decision) to empirical data. For example, these metrics can offer valuable insight into how actions and cognitive processes might maximize reward rate, which is a simple function of the FPT properties (Bogacz et al., 2006).

However, not all decisions can be properly modeled if parameters are fixed throughout the duration of the decision process. Certain contexts can be better described by a model whose parameters change with time. In this article we analyze the time-dependent version of the Wiener-process-with-drift between two absorbing boundaries, building on recent work that is focused on similar time-varying random walk models (Diederich and Oswald, 2014, Hubner et al., 2010). After reviewing how martingale theory can be used to analyze and re-derive the classical FPT results for the time independent case, we calculate results for the time-dependent case. The main theoretical results are presented in Section  5.2, where we provide closed form expressions for threshold-hitting probabilities and expected decision times. In Appendix D, we also describe how our methods can be applied to the more general Ornstein–Uhlenbeck (O–U) processes, which are similar to the Wiener diffusion processes albeit with an additional “leak” term. We conclude with a summary of the results and a discussion of how the present work interfaces with other similar analyses of time-varying random walk models.

Section snippets

Notation and terminology

Here we introduce the notation and terminology for describing the model we analyze, which is a Wiener process with (time-dependent) piecewise constant parameters. This simple stochastic model, and others close to it, have been studied before (Bogacz et al., 2006, Diederich and Busemeyer, 2003, Diederich and Busemeyer, 2006, Diederich and Oswald, 2014, Heath, 1992, Ratcliff, 1980, Smith, 2000, Wagenmakers et al., 2007), although the reader should note that our parameterization differs from that

Martingale theory applied to the single-stage model

In this section, we give an introduction to the basic properties of martingales and the optional stopping theorem, which are the key mathematical tools used in calculating our main results in Section  5. For readers who are less familiar with martingale methods, we first derive the mean decision time, hitting probabilities, and FPT densities for the single- and two-stage models. These analyses provide an alternate approach to deriving these classical results as compared to other non-martingale

Analysis of the two-stage model

In this section, we use the tools developed in Section  3 in order to analyze the two-stage process originally presented and analyzed in Ratcliff (1980). While our calculations lead to equivalent formulas for the first passage time densities, a martingale argument provides us with additional closed form expressions for the probability of hitting a particular threshold and expected decision times. Computations of these FPT statistics using the results of Ratcliff (1980) require numerical

Analysis of the multistage model

In this section we derive first passage time (FPT) properties of the multistage process defined in Section  2.2 using an approach similar to that employed throughout Section  4. The model is viewed as n modified processes in sequence such that for each process (stage), the initial condition is a random variable and only the decisions made before a deadline are considered. For the ith stage process with a known distribution of initial condition Xi1, we derive properties of the FPT conditioned

Time-varying thresholds for the multistage process

The results in Section  5 were obtained under the assumption that the thresholds are constant throughout each stage. Now suppose that the thresholds for the ith stage are ±ζi, i.e., piecewise constant thresholds. If the upper thresholds decrease at time ti (i.e.  ζi+1<ζi) and x(ti) is in the interval (ζi+1,ζi), then the path is absorbed by the upper boundary, and the probability of this instantaneous absorption is calculated by integrating (18) from ζi+1 to ζi. Likewise, the probability of

Numerical examples

In this section we apply our calculations from Sections  5 Analysis of the multistage model, 6 Time-varying thresholds for the multistage process to a variety of numerical experiments. In doing so, we compare the theoretical predictions obtained from the analysis in this paper with the numerical values obtained through Monte-Carlo simulations, thereby numerically verifying our derivations above. We also provide examples illustrating time pressure or changes in attention over the course of a

Discussion

In this paper we analyze the first passage time properties of a Wiener process between two absorbing boundaries with piecewise constant (time-dependent) parameters, which we call a multistage model or multistage process. Our main theoretical results, collected in Section  5, add to previous work on analyzing time-dependent random walk models in psychology and neuroscience. Broadly speaking, these can be split into three approaches. One approach is the integral equation approach introduced and

Acknowledgments

We thank the editor, Philip Smith, and the referees for helpful comments, especially regarding this article’s exposition and discussion. We thank Ryan Webb for the reference to Smith (2000) and the associated code. We thank Phil Holmes and Patrick Simen for helpful comments and discussions. This work was supported in part by the C.V. Starr Foundation (AS), the Princeton University Insley-Blair Pyne Fund, ONR grant N00014-14-1-0635, ARO grant W911NF-14-1-0431 (VS and NEL), and NIH Brain

References (64)

  • R. Ratcliff et al.

    Diffusion decision model: Current issues and history

    Trends in Cognitive Sciences

    (2016)
  • P.L. Smith

    A note on the distribution of response times for a random walk with Gaussian increments

    Journal of Mathematical Psychology

    (1990)
  • P.L. Smith

    Stochastic dynamic models of response time and accuracy: A foundational primer

    Journal of Mathematical Psychology

    (2000)
  • C. Voskuilen et al.

    Comparing fixed and collapsing boundary versions of the diffusion model

    Journal of Mathematical Psychology

    (2016)
  • X.-J. Wang

    Probabilistic decision making by slow reverberation in cortical circuits

    Neuron

    (2002)
  • C.N. White et al.

    Diffusion models of the flanker task: Dzhaniscrete versus gradual attentional selection

    Cognitive Psychology

    (2011)
  • R. Bogacz et al.

    The physics of optimal decision making: A formal analysis of models of performance in two-alternative forced-choice tasks

    Psychological Review

    (2006)
  • A.N. Borodin et al.

    Handbook of brownian motion: facts and formulae

    (2002)
  • B.W. Brunton et al.

    Rats and humans can optimally accumulate evidence for decision-making

    Science

    (2013)
  • J.R. Busemeyer et al.

    Cognitive modeling

    (2010)
  • P. Cisek et al.

    Decisions in changing conditions: The urgency-gating model

    The Journal of Neuroscience

    (2009)
  • D.R. Cox et al.

    The theory of stochastic processes

    (1965)
  • A. Diederich et al.

    Modeling the effects of payoff on response bias in a perceptual discrimination task: Bound-change, drift-rate-change, or two-stage-processing hypothesis

    Perception & Psychophysics

    (2006)
  • A. Diederich et al.

    Sequential sampling model for multiattribute choice alternatives with random attention time and processing order

    Frontiers in Human Neuroscience

    (2014)
  • J.-L. Doob

    Stochastic processes

    (1953)
  • R. Douady

    Closed form formulas for exotic options and their lifetime distribution

    International Journal of Theoretical and Applied Finance

    (1999)
  • Drugowitsch, J. (2014). C++ diffusion model toolset with Python and Matlab interfaces. GitHub repository:...
  • J. Drugowitsch et al.

    The cost of accumulating evidence in perceptual decision making

    The Journal of Neuroscience

    (2012)
  • R. Durrett

    Probability: theory and examples

    (2010)
  • Z. Farkas et al.

    One-dimensional drift-diffusion between two absorbing boundaries: Application to granular segregation

    Journal of Physics A: Mathematical and General

    (2001)
  • W. Feller

    An introduction to probability theory and its applications. Vol. 1

    (1968)
  • S. Feng et al.

    Can monkeys choose optimally when faced with noisy stimuli and unequal rewards

    PLoS Computational Biology

    (2009)
  • Cited by (17)

    • Computation of time probability distributions for the occurrence of uncertain future events

      2021, Mechanical Systems and Signal Processing
      Citation Excerpt :

      Efforts on finding analytical expressions for FPT probability distributions have been carried out on many disciplines and application domains such as in chemistry [15,16], physics [17,18], biology [19,20], neurobiology [21,22], epidemiology [23], psychology [24], finance [25,26], economy [27,28], reliability theory [29,30], among others [1,2]. Nonetheless, it is important to emphasize the fact that most of these research efforts have focused on continuous-time [31–37], rather than discrete-time systems [27,38–40] (except the case of autoregressive models [41–44,39,45–48]). In continuous-time systems, the FPT probability distribution constitutes the solution to particular Stochastic Differential Equation (SDE) with boundary conditions, which is typically solved using transformations [49–51] or on eigenfunction expansions [32,50] (most of the times numerically approximated).

    • Audiovisual detection at different intensities and delays

      2019, Journal of Mathematical Psychology
    • Associative memory retrieval modulates upcoming perceptual decisions

      2023, Cognitive, Affective and Behavioral Neuroscience
    View all citing articles on Scopus
    View full text