A martingale analysis of first passage times of time-dependent Wiener diffusion models
Introduction
Continuous time stochastic processes modeling a particle’s diffusion (with drift) towards one of two absorbing boundaries have been used in a wide variety of applications including statistical physics (Farkas & Fulop, 2001), finance (Lin, 1998), economics (Webb, 2015), and health science (Horrocks & Thompson, 2004). Varieties of such models have also been applied extensively within psychology and neuroscience to describe both the behavior and neural activity associated with decision processes involved in perception, memory, attention, and cognitive control (Brunton et al., 2013, Diederich and Oswald, 2014, Diederich and Oswald, 2016, Feng et al., 2009, Gold and Shadlen, 2001, Gold and Shadlen, 2007, Heath, 1992, Ratcliff and McKoon, 2008, Ratcliff and Rouder, 1998, Shadlen and Newsome, 2001, Simen et al., 2009); for reviews see (Bogacz et al., 2006, Busemeyer and Diederich, 2010, Ratcliff and Smith, 2004).
In these stochastic accumulation decision models, the state variable is thought to represent the amount of accumulated noisy evidence at time for decisions represented by the two absorbing boundaries, that we refer to as the upper () and lower () thresholds (boundaries). The evidence evolves in time according to a biased random walk with Gaussian increments, which may be written as , and a decision is said to be made at the random time , the smallest time for which hits either the upper threshold () or the lower threshold (), also known as the first passage time (FPT). The resulting decision dynamics are thus described by the FPT of the underlying model. In studying these processes one is often interested in relating the mean decision time and the probability of hitting a certain threshold (e.g. the probability of making a certain decision) to empirical data. For example, these metrics can offer valuable insight into how actions and cognitive processes might maximize reward rate, which is a simple function of the FPT properties (Bogacz et al., 2006).
However, not all decisions can be properly modeled if parameters are fixed throughout the duration of the decision process. Certain contexts can be better described by a model whose parameters change with time. In this article we analyze the time-dependent version of the Wiener-process-with-drift between two absorbing boundaries, building on recent work that is focused on similar time-varying random walk models (Diederich and Oswald, 2014, Hubner et al., 2010). After reviewing how martingale theory can be used to analyze and re-derive the classical FPT results for the time independent case, we calculate results for the time-dependent case. The main theoretical results are presented in Section 5.2, where we provide closed form expressions for threshold-hitting probabilities and expected decision times. In Appendix D, we also describe how our methods can be applied to the more general Ornstein–Uhlenbeck (O–U) processes, which are similar to the Wiener diffusion processes albeit with an additional “leak” term. We conclude with a summary of the results and a discussion of how the present work interfaces with other similar analyses of time-varying random walk models.
Section snippets
Notation and terminology
Here we introduce the notation and terminology for describing the model we analyze, which is a Wiener process with (time-dependent) piecewise constant parameters. This simple stochastic model, and others close to it, have been studied before (Bogacz et al., 2006, Diederich and Busemeyer, 2003, Diederich and Busemeyer, 2006, Diederich and Oswald, 2014, Heath, 1992, Ratcliff, 1980, Smith, 2000, Wagenmakers et al., 2007), although the reader should note that our parameterization differs from that
Martingale theory applied to the single-stage model
In this section, we give an introduction to the basic properties of martingales and the optional stopping theorem, which are the key mathematical tools used in calculating our main results in Section 5. For readers who are less familiar with martingale methods, we first derive the mean decision time, hitting probabilities, and FPT densities for the single- and two-stage models. These analyses provide an alternate approach to deriving these classical results as compared to other non-martingale
Analysis of the two-stage model
In this section, we use the tools developed in Section 3 in order to analyze the two-stage process originally presented and analyzed in Ratcliff (1980). While our calculations lead to equivalent formulas for the first passage time densities, a martingale argument provides us with additional closed form expressions for the probability of hitting a particular threshold and expected decision times. Computations of these FPT statistics using the results of Ratcliff (1980) require numerical
Analysis of the multistage model
In this section we derive first passage time (FPT) properties of the multistage process defined in Section 2.2 using an approach similar to that employed throughout Section 4. The model is viewed as modified processes in sequence such that for each process (stage), the initial condition is a random variable and only the decisions made before a deadline are considered. For the th stage process with a known distribution of initial condition , we derive properties of the FPT conditioned
Time-varying thresholds for the multistage process
The results in Section 5 were obtained under the assumption that the thresholds are constant throughout each stage. Now suppose that the thresholds for the th stage are , i.e., piecewise constant thresholds. If the upper thresholds decrease at time (i.e. ) and is in the interval , then the path is absorbed by the upper boundary, and the probability of this instantaneous absorption is calculated by integrating (18) from to . Likewise, the probability of
Numerical examples
In this section we apply our calculations from Sections 5 Analysis of the multistage model, 6 Time-varying thresholds for the multistage process to a variety of numerical experiments. In doing so, we compare the theoretical predictions obtained from the analysis in this paper with the numerical values obtained through Monte-Carlo simulations, thereby numerically verifying our derivations above. We also provide examples illustrating time pressure or changes in attention over the course of a
Discussion
In this paper we analyze the first passage time properties of a Wiener process between two absorbing boundaries with piecewise constant (time-dependent) parameters, which we call a multistage model or multistage process. Our main theoretical results, collected in Section 5, add to previous work on analyzing time-dependent random walk models in psychology and neuroscience. Broadly speaking, these can be split into three approaches. One approach is the integral equation approach introduced and
Acknowledgments
We thank the editor, Philip Smith, and the referees for helpful comments, especially regarding this article’s exposition and discussion. We thank Ryan Webb for the reference to Smith (2000) and the associated code. We thank Phil Holmes and Patrick Simen for helpful comments and discussions. This work was supported in part by the C.V. Starr Foundation (AS), the Princeton University Insley-Blair Pyne Fund, ONR grant N00014-14-1-0635, ARO grant W911NF-14-1-0431 (VS and NEL), and NIH Brain
References (64)
- et al.
Fast and accurate calculations for cumulative first-passage time distributions in Wiener diffusion models
Journal of Mathematical Psychology
(2012) Optimal decision-making theories: linking neurobiology with behaviour
Trends in Cognitive Sciences
(2007)- et al.
Simple matrix methods for analyzing diffusion models of choice probability, choice response time, and simple response time
Journal of Mathematical Psychology
(2003) - et al.
Multi-stage sequential sampling models with finite or infinite time horizon and variable boundaries
Journal of Mathematical Psychology
(2016) - et al.
Neural computations that underlie decisions about sensory stimuli
Trends in Cognitive Sciences
(2001) - et al.
Even faster and even more accurate first-passage time densities and distributions for the Wiener diffusion model
Journal of Mathematical Psychology
(2014) A general nonstationary diffusion model for two-choice decision-making
Mathematical Social Sciences
(1992)The relative judgment theory of two choice response time
Journal of Mathematical Psychology
(1975)- et al.
Fast and accurate calculations for first-passage times in Wiener diffusion models
Journal of Mathematical Psychology
(2009) A note on modeling accumulation of information when the rate of accumulation changes over time
Journal of Mathematical Psychology
(1980)
Diffusion decision model: Current issues and history
Trends in Cognitive Sciences
A note on the distribution of response times for a random walk with Gaussian increments
Journal of Mathematical Psychology
Stochastic dynamic models of response time and accuracy: A foundational primer
Journal of Mathematical Psychology
Comparing fixed and collapsing boundary versions of the diffusion model
Journal of Mathematical Psychology
Probabilistic decision making by slow reverberation in cortical circuits
Neuron
Diffusion models of the flanker task: Dzhaniscrete versus gradual attentional selection
Cognitive Psychology
The physics of optimal decision making: A formal analysis of models of performance in two-alternative forced-choice tasks
Psychological Review
Handbook of brownian motion: facts and formulae
Rats and humans can optimally accumulate evidence for decision-making
Science
Cognitive modeling
Decisions in changing conditions: The urgency-gating model
The Journal of Neuroscience
The theory of stochastic processes
Modeling the effects of payoff on response bias in a perceptual discrimination task: Bound-change, drift-rate-change, or two-stage-processing hypothesis
Perception & Psychophysics
Sequential sampling model for multiattribute choice alternatives with random attention time and processing order
Frontiers in Human Neuroscience
Stochastic processes
Closed form formulas for exotic options and their lifetime distribution
International Journal of Theoretical and Applied Finance
The cost of accumulating evidence in perceptual decision making
The Journal of Neuroscience
Probability: theory and examples
One-dimensional drift-diffusion between two absorbing boundaries: Application to granular segregation
Journal of Physics A: Mathematical and General
An introduction to probability theory and its applications. Vol. 1
Can monkeys choose optimally when faced with noisy stimuli and unequal rewards
PLoS Computational Biology
Cited by (17)
Randomness accelerates the dynamic clearing process of the COVID-19 outbreaks in China
2023, Mathematical BiosciencesComputation of time probability distributions for the occurrence of uncertain future events
2021, Mechanical Systems and Signal ProcessingCitation Excerpt :Efforts on finding analytical expressions for FPT probability distributions have been carried out on many disciplines and application domains such as in chemistry [15,16], physics [17,18], biology [19,20], neurobiology [21,22], epidemiology [23], psychology [24], finance [25,26], economy [27,28], reliability theory [29,30], among others [1,2]. Nonetheless, it is important to emphasize the fact that most of these research efforts have focused on continuous-time [31–37], rather than discrete-time systems [27,38–40] (except the case of autoregressive models [41–44,39,45–48]). In continuous-time systems, the FPT probability distribution constitutes the solution to particular Stochastic Differential Equation (SDE) with boundary conditions, which is typically solved using transformations [49–51] or on eigenfunction expansions [32,50] (most of the times numerically approximated).
Decision with multiple alternatives: Geometric models in higher dimensions — the cube model
2019, Journal of Mathematical PsychologyAudiovisual detection at different intensities and delays
2019, Journal of Mathematical PsychologyAssociative memory retrieval modulates upcoming perceptual decisions
2023, Cognitive, Affective and Behavioral NeuroscienceAsynchrony rescues statistically optimal group decisions from information cascades through emergent leaders
2023, Royal Society Open Science