A penalty scheme and policy iteration for nonlocal HJB variational inequalities with monotone nonlinearities

https://doi.org/10.1016/j.camwa.2021.04.011Get rights and content

Abstract

We propose a class of numerical schemes for nonlocal HJB variational inequalities (HJBVIs) with monotone nonlinearities arising from mixed optimal stopping and control of processes with infinite activity jumps, where the objective is specified by a monotone recursive preference. The solution and free boundary of the HJBVI are constructed from a sequence of penalized equations, for which the penalization error is estimated. The penalized equation is then discretized by a class of semi-implicit monotone approximations. We present a novel analysis technique for the well-posedness of the discrete equation, and demonstrate the convergence of the scheme, which subsequently gives a constructive proof for the existence of a solution to the penalized equation and variational inequality. We further propose an efficient iterative algorithm with local superlinear convergence for solving the discrete equation. Numerical experiments are presented for an optimal investment problem under ambiguity and a two-dimensional recursive consumption-portfolio allocation problem.

Introduction

In this article, we consider the following nonlocal Hamilton-Jacobi-Bellman variational inequality (HJBVI) on QT=(0,T]×Rd with T>0:0=F(t,x,u,Du,D2u,{Kαu}αA,{Bαu}αA)={min{uζ,ut+infαA(Lαufα(t,x,u,(σα)TDu,Bαu))},(t,x)QT,u(0,x)g(x),xRd, with the operators Lα:=Aα+Kα and Bα satisfying for ϕC1,2(Q¯T) thatAαϕ(t,x)=12tr(σα(t,x)(σα(t,x))TD2ϕ(t,x))+bα(t,x)Dϕ(t,x),Kαϕ(t,x)=E(ϕ(t,x+ηα(t,x,e))ϕ(t,x)ηα(t,x,e)Dϕ(t,x))ν(de),Bαϕ(t,x)=Em(ϕ(t,x+ηα(t,x,e))ϕ(t,x))γ(t,x,e)ν(de), where u:Q¯TR is an unknown function, A is a given nonempty compact metric space, ν is a given σ-finite measure on E=Rn{0} satisfying E(1|e|2)ν(de)<, m:RR, g:RdR, ζ:Q¯TR, γ:Q¯T×ER are given continuous functions, for each αA, bα:Q¯TRd, σα:Q¯TRd×d, ηα:Q¯T×ERd, fα:Q¯T×R×Rd×RR are given continuous functions, and for each ϕC1,2(Q¯T) and (t,x)Q¯T, Dϕ(t,x) and D2ϕ(t,x) are the gradient and Hessian of ϕ at (t,x), respectively. We shall specify the precise conditions of the coefficients of (1.1) in Assumption 1.

Note that the integral operators Kα and Bα create the non-locality of (1.1), which indicates that the value of the solution u at each point (t,x)QT evolutes based on the weighted average of others values of u with respect to the measure ν. These integral operators arise naturally as the generators of pure jump Lévy processes with Lévy measure ν (see e.g. [10]). Here we allow ν to be a singular measure such that ν({eE||e|<1})=, which is associated with infinite activity Lévy processes, i.e., Lévy processes that admit infinite number of jumps in each time interval.

We shall allow the function fα, called the driver of (1.1), to be monotone, possibly non-Fréchet-differentiable, and of arbitrary growth in its third component. Equation (1.1) with monotone nonlinearies extends the classical HJBVIs with linear drivers, i.e., fα(t,x,y,z,k)α(t,x)rα(t,x)y for some functions α,rα:Q¯TR, and play an important role in modern finance, including the following: models for American options in a market with constrained portfolios [18], [17], [2], recursive utility optimization problems [20], and robust pricing and risk measures under probability model uncertainty [3], [37], [34]. We remark that imposing merely monotonicity assumptions on the drivers allows us to consider several important non-smooth Lipschitz drivers stemming from robust pricing [37], [19], [34] and non-Lipschitz drivers arising in stochastic recursive control problems (e.g. the Epstein-Zin preference in [20], [29], [33]), while including an extra nonlinearity in the operator Bα enables us to incorporate ambiguity in the jump processes [37]. In particular, we present two worked-out examples in Section 6, where the first one describes the situation that an investor chooses the optimal wealth allocation and liquidation time of an asset, whose price process has infinite activity jumps, by taking potential model misspecification into consideration, while the second one describes the situation that an agent chooses their optimal consumption and investment strategies based on the (nonlinear) Epstein-Zin utility. These problems lead naturally to HJBVIs of the type (1.1) with the key features present (i.e., singular non-local term, non-smooth/non-Lipschitz driver, control optimization, obstacle term). As the solution to (1.1) is in general not known analytically, it is important to construct effective and robust numerical schemes for solving these fully nonlinear equations.

To the best of our knowledge, there is no published numerical scheme covering the generality of (1.1). However, there is a vast literature on monotone approximations for local HJB equations (e.g., [11], [4], [30], [12]) and on monotone finite-difference quadrature schemes for nonlocal HJB equations (e.g., [6], [5]). For works covering specific extensions, we refer the reader to [22], [24], [28] for penalty approximations to elliptic variational inequalities, to [25], [16], [23] for penalty approximations to parabolic variational inequalities (with applications to American option pricing problems), to [7], [39] for an application of policy iteration together with penalization to solve HJB obstacle problems with linear drivers, to [15] for schemes to HJB obstacle problems with Lipschitz drivers based on piecewise constant policy time stepping, and to [40] for applying policy iteration to solve (finite-dimensional) static HJB equations with Fréchet-differentiable concave drivers and finite control sets.

In this paper, we shall construct a class of monotone schemes for solving (1.1) with a monotone (possibly non-Fréchet-differentiable) driver and a compact set of controls. Note that monotonicity of the scheme is crucial, since it is well-known that non-monotone schemes may fail to converge or even converge to false “solutions” [12]. By Godunov's Theorem [21], in general, one can expect a monotone scheme to be at most first-order accurate.

Recently, a class of non-monotone “filtered” schemes has been proposed and analyzed in [8] for parabolic HJB equations (without the integral operators Kα and Bα), which combines monotone schemes with high-order non-monotone discretizations, and exhibits an overall high order convergence behaviour for smooth enough solutions. However, as pointed out in [8, Remark 2.8], it is more challenging to establish the well-posedness and convergence of such schemes in a general setting due to the loss of monotonicity. The presence of nonlocal operators Kα and Bα and the non-smooth non-Lipschitz driver fα further complicates the construction of high-order time and space discretizations of (1.1). Hence, we focus on monotone discretizations of (1.1) in this paper.

We emphasize that the non-Lipschitz setting of the drivers prevents us from adopting the standard Banach fixed-point arguments (see e.g. [12], [15]) to establish the well-posedness and stability of the discrete approximations of (1.1), and hence new analysis techniques are required. Moreover, the non-differentiability and non-convexity of the driver in y and its nonlinear dependence on z and k also introduce substantial difficulties in designing efficient iterative algorithms for solving the discrete equations.

The main contributions of this work are as follows:

  • We formulate a penalty approximation to (1.1) with a monotone driver, and demonstrate that, as the penalty parameter tends to infinity, the solution of the penalized equation converges to the solution of (1.1) monotonically from below, at a rate depending explicitly on the regularity of the obstacle, which extends the results in [28] to nonlocal equations with monotone drivers and time-dependent obstacles. These convergence results further lead us to a convergent approximation of the free boundary of (1.1), which to our best knowledge is new, even in the classical cases with linear drivers.

  • We propose a class of semi-implicit monotone approximations to the penalized equations, which enjoy a stability condition independent of the penalty parameter. We further present a novel analysis technique for the well-posedness of the resulting (infinite-dimensional) discrete equation by constructing Lipschitz approximations of the monotone driver via smoothing and truncation. The convergence of the scheme is demonstrated, which subsequently gives a constructive proof for the existence of a bounded viscosity solution to the penalized equations and the HJBVI (1.1).

  • For practical implementations, we propose an efficient iterative algorithm for a localized discrete equation with a slantly differentiable driver, and demonstrate the local superlinear convergence for the value functions, which extends the results obtained in the cases with linear drivers (see [7], [39]) or with Fréchet-differentiable convex drivers and finite control sets [40].

  • Numerical examples for an optimal investment problem for jump-diffusion models under ambiguity and a two-dimensional recursive consumption-portfolio allocation problem with stochastic volatility models are included to investigate the convergence order of the scheme with respect to different discretization parameters.

We organize this paper as follows. Section 2 gives standard definitions and assumptions on the HJBVI (1.1). We shall propose a penalty approximation to the HJBVI in Section 3 and study its convergence properties. Then we derive a class of fully discrete monotone schemes for the penalized equations in Section 4.1, and establish their convergence in Section 4.2. A Newton-type iterative method with local superlinear convergence is constructed in Section 5 to solve the resulting discrete equations. Numerical examples for an optimal investment problem under ambiguity and a recursive consumption-portfolio allocation problem are presented in Section 6 to illustrate the effectiveness of our algorithms.

Section snippets

Main assumptions and preliminaries

In this section, we state our main assumptions on the coefficients of (1.1) and introduce related concepts of solutions. We start by collecting some useful notation which is needed frequently throughout this work.

For any given function g:Rd1Rd2, we define by g+:=max(g,0) and g:=max(g,0) the (component-wise) positive and negative part of g, respectively. Also for a function f:Q¯T:=[0,T]×RdRd1×d2 we define the following (semi-)norms:|f|0=sup(t,x)Q¯T|f(t,x)|,|f|1=supt[0,T],x,xRd,xx|f(

Penalty approximations for the HJBVI

In this section, we shall propose a penalty approximation for the HJBVI (1.1), which is an extension of the ideas used for local HJB obstacle problems (with linear drivers) in [28], [39] and for American options in [23].

For any given parameter ρ0, we shall consider the following penalized problem:0=Fρ(x,u,Du,D2u,{Kαu}αA,{Bαu}αA)={utρ+infαA(Lαuρfα(x,uρ,(σα)TDuρ,Bαuρ))ρ(ζuρ)+,xQT,uρ(0,x)g(x),xRd, which will be interpreted in the viscosity sense similar to Definition 2.1 by virtue of

Discrete approximations for penalized equations

In this section, we propose a class of semi-implicit monotone approximations for solving the penalized equation (3.1) with a fixed penalty parameter ρ0. We shall construct the schemes in Section 4.1 and perform their analysis in Section 4.2. In order to derive more accurate estimates for the truncation error and the stability condition of the schemes, throughout this section we shall impose the following condition on the Lévy measure:

Assumption 2

The Lévy measure ν admits a density k(e) with the following

Policy iteration for the discrete equation

In this section, we propose an efficient method for solving the discrete problem based on policy iteration. We shall further demonstrate local superlinear convergence by interpreting the scheme as a nonsmooth Newton method. Since in practice one usually truncates the discrete equation (4.19) by localizing it onto a chosen bounded computational domain, and specifying the behaviour of the solution outside the domain, we shall consider the following finite-dimensional problem: for any given unRM,

Numerical experiments

In this section, we demonstrate the effectiveness of the schemes through numerical experiments. We present two examples, an optimal investment problem with model uncertainty, and a consumption-portfolio allocation problem with non-Lipschitz recursive utilities. Both examples are related to non-standard HJB equations, where the first example contains non-smooth convex/concave nonlinearities, while the second one involves monotone drivers of polynomial growth.

Conclusions

This paper constructs numerical approximations to the solution and free boundary of HJB variational inequalities with monotone drivers, which arise from mixed stochastic control/optimal stopping problems with recursive preferences and infinite activity jumps. The schemes are based on the penalty method, monotone discretizations and policy iteration. We prove the convergence of the numerical scheme and illustrate the theoretical results with some numerical examples including an optimal

References (40)

  • I. Biswas et al.

    On the rate of convergence for monotone numerical schemes for nonlocal Isaacs' equations

    SIAM J. Numer. Anal.

    (2019)
  • I.H. Biswas et al.

    Difference-quadrature schemes for nonlinear degenerate parabolic integro-PDE

    SIAM J. Numer. Anal.

    (2010)
  • O. Bokanowski et al.

    Some convergence results for Howard's algorithm

    SIAM J. Numer. Anal.

    (2009)
  • O. Bokanowski et al.

    High-order filtered schemes for time-dependent second order HJB equations

    Modél. Math. Anal. Numér.

    (2018)
  • X. Chen et al.

    Smoothing methods and semismooth methods for nondifferentiable operator equations

    SIAM J. Numer. Anal.

    (2000)
  • R. Cont et al.

    Financial Modelling with Jump Processes

    (2003)
  • M.G. Crandall et al.

    Two approximations of solutions of Hamilton-Jacobi equations

    Math. Comput.

    (1984)
  • K. Debrabant et al.

    Semi-Lagrangian schemes for linear and fully nonlinear diffusion equations

    Math. Comput.

    (2012)
  • R. Dumitrescu et al.

    A weak dynamic programming principle for combined optimal stopping/ stochastic control with Ef-expectations

    SIAM J. Control Optim.

    (2016)
  • R. Dumitrescu et al.

    Mixed generalized Dynkin game and stochastic control in a Markovian framework

    Stochastics

    (2016)
  • View full text