A penalty scheme and policy iteration for nonlocal HJB variational inequalities with monotone nonlinearities
Introduction
In this article, we consider the following nonlocal Hamilton-Jacobi-Bellman variational inequality (HJBVI) on with : with the operators and satisfying for that where is an unknown function, A is a given nonempty compact metric space, ν is a given σ-finite measure on satisfying , , , , are given continuous functions, for each , , , , are given continuous functions, and for each and , and are the gradient and Hessian of ϕ at , respectively. We shall specify the precise conditions of the coefficients of (1.1) in Assumption 1.
Note that the integral operators and create the non-locality of (1.1), which indicates that the value of the solution u at each point evolutes based on the weighted average of others values of u with respect to the measure ν. These integral operators arise naturally as the generators of pure jump Lévy processes with Lévy measure ν (see e.g. [10]). Here we allow ν to be a singular measure such that , which is associated with infinite activity Lévy processes, i.e., Lévy processes that admit infinite number of jumps in each time interval.
We shall allow the function , called the driver of (1.1), to be monotone, possibly non-Fréchet-differentiable, and of arbitrary growth in its third component. Equation (1.1) with monotone nonlinearies extends the classical HJBVIs with linear drivers, i.e., for some functions , and play an important role in modern finance, including the following: models for American options in a market with constrained portfolios [18], [17], [2], recursive utility optimization problems [20], and robust pricing and risk measures under probability model uncertainty [3], [37], [34]. We remark that imposing merely monotonicity assumptions on the drivers allows us to consider several important non-smooth Lipschitz drivers stemming from robust pricing [37], [19], [34] and non-Lipschitz drivers arising in stochastic recursive control problems (e.g. the Epstein-Zin preference in [20], [29], [33]), while including an extra nonlinearity in the operator enables us to incorporate ambiguity in the jump processes [37]. In particular, we present two worked-out examples in Section 6, where the first one describes the situation that an investor chooses the optimal wealth allocation and liquidation time of an asset, whose price process has infinite activity jumps, by taking potential model misspecification into consideration, while the second one describes the situation that an agent chooses their optimal consumption and investment strategies based on the (nonlinear) Epstein-Zin utility. These problems lead naturally to HJBVIs of the type (1.1) with the key features present (i.e., singular non-local term, non-smooth/non-Lipschitz driver, control optimization, obstacle term). As the solution to (1.1) is in general not known analytically, it is important to construct effective and robust numerical schemes for solving these fully nonlinear equations.
To the best of our knowledge, there is no published numerical scheme covering the generality of (1.1). However, there is a vast literature on monotone approximations for local HJB equations (e.g., [11], [4], [30], [12]) and on monotone finite-difference quadrature schemes for nonlocal HJB equations (e.g., [6], [5]). For works covering specific extensions, we refer the reader to [22], [24], [28] for penalty approximations to elliptic variational inequalities, to [25], [16], [23] for penalty approximations to parabolic variational inequalities (with applications to American option pricing problems), to [7], [39] for an application of policy iteration together with penalization to solve HJB obstacle problems with linear drivers, to [15] for schemes to HJB obstacle problems with Lipschitz drivers based on piecewise constant policy time stepping, and to [40] for applying policy iteration to solve (finite-dimensional) static HJB equations with Fréchet-differentiable concave drivers and finite control sets.
In this paper, we shall construct a class of monotone schemes for solving (1.1) with a monotone (possibly non-Fréchet-differentiable) driver and a compact set of controls. Note that monotonicity of the scheme is crucial, since it is well-known that non-monotone schemes may fail to converge or even converge to false “solutions” [12]. By Godunov's Theorem [21], in general, one can expect a monotone scheme to be at most first-order accurate.
Recently, a class of non-monotone “filtered” schemes has been proposed and analyzed in [8] for parabolic HJB equations (without the integral operators and ), which combines monotone schemes with high-order non-monotone discretizations, and exhibits an overall high order convergence behaviour for smooth enough solutions. However, as pointed out in [8, Remark 2.8], it is more challenging to establish the well-posedness and convergence of such schemes in a general setting due to the loss of monotonicity. The presence of nonlocal operators and and the non-smooth non-Lipschitz driver further complicates the construction of high-order time and space discretizations of (1.1). Hence, we focus on monotone discretizations of (1.1) in this paper.
We emphasize that the non-Lipschitz setting of the drivers prevents us from adopting the standard Banach fixed-point arguments (see e.g. [12], [15]) to establish the well-posedness and stability of the discrete approximations of (1.1), and hence new analysis techniques are required. Moreover, the non-differentiability and non-convexity of the driver in y and its nonlinear dependence on z and k also introduce substantial difficulties in designing efficient iterative algorithms for solving the discrete equations.
The main contributions of this work are as follows:
- •
We formulate a penalty approximation to (1.1) with a monotone driver, and demonstrate that, as the penalty parameter tends to infinity, the solution of the penalized equation converges to the solution of (1.1) monotonically from below, at a rate depending explicitly on the regularity of the obstacle, which extends the results in [28] to nonlocal equations with monotone drivers and time-dependent obstacles. These convergence results further lead us to a convergent approximation of the free boundary of (1.1), which to our best knowledge is new, even in the classical cases with linear drivers.
- •
We propose a class of semi-implicit monotone approximations to the penalized equations, which enjoy a stability condition independent of the penalty parameter. We further present a novel analysis technique for the well-posedness of the resulting (infinite-dimensional) discrete equation by constructing Lipschitz approximations of the monotone driver via smoothing and truncation. The convergence of the scheme is demonstrated, which subsequently gives a constructive proof for the existence of a bounded viscosity solution to the penalized equations and the HJBVI (1.1).
- •
For practical implementations, we propose an efficient iterative algorithm for a localized discrete equation with a slantly differentiable driver, and demonstrate the local superlinear convergence for the value functions, which extends the results obtained in the cases with linear drivers (see [7], [39]) or with Fréchet-differentiable convex drivers and finite control sets [40].
- •
Numerical examples for an optimal investment problem for jump-diffusion models under ambiguity and a two-dimensional recursive consumption-portfolio allocation problem with stochastic volatility models are included to investigate the convergence order of the scheme with respect to different discretization parameters.
We organize this paper as follows. Section 2 gives standard definitions and assumptions on the HJBVI (1.1). We shall propose a penalty approximation to the HJBVI in Section 3 and study its convergence properties. Then we derive a class of fully discrete monotone schemes for the penalized equations in Section 4.1, and establish their convergence in Section 4.2. A Newton-type iterative method with local superlinear convergence is constructed in Section 5 to solve the resulting discrete equations. Numerical examples for an optimal investment problem under ambiguity and a recursive consumption-portfolio allocation problem are presented in Section 6 to illustrate the effectiveness of our algorithms.
Section snippets
Main assumptions and preliminaries
In this section, we state our main assumptions on the coefficients of (1.1) and introduce related concepts of solutions. We start by collecting some useful notation which is needed frequently throughout this work.
For any given function , we define by and the (component-wise) positive and negative part of g, respectively. Also for a function we define the following (semi-)norms:
Penalty approximations for the HJBVI
In this section, we shall propose a penalty approximation for the HJBVI (1.1), which is an extension of the ideas used for local HJB obstacle problems (with linear drivers) in [28], [39] and for American options in [23].
For any given parameter , we shall consider the following penalized problem: which will be interpreted in the viscosity sense similar to Definition 2.1 by virtue of
Discrete approximations for penalized equations
In this section, we propose a class of semi-implicit monotone approximations for solving the penalized equation (3.1) with a fixed penalty parameter . We shall construct the schemes in Section 4.1 and perform their analysis in Section 4.2. In order to derive more accurate estimates for the truncation error and the stability condition of the schemes, throughout this section we shall impose the following condition on the Lévy measure: Assumption 2 The Lévy measure ν admits a density with the following
Policy iteration for the discrete equation
In this section, we propose an efficient method for solving the discrete problem based on policy iteration. We shall further demonstrate local superlinear convergence by interpreting the scheme as a nonsmooth Newton method. Since in practice one usually truncates the discrete equation (4.19) by localizing it onto a chosen bounded computational domain, and specifying the behaviour of the solution outside the domain, we shall consider the following finite-dimensional problem: for any given ,
Numerical experiments
In this section, we demonstrate the effectiveness of the schemes through numerical experiments. We present two examples, an optimal investment problem with model uncertainty, and a consumption-portfolio allocation problem with non-Lipschitz recursive utilities. Both examples are related to non-standard HJB equations, where the first example contains non-smooth convex/concave nonlinearities, while the second one involves monotone drivers of polynomial growth.
Conclusions
This paper constructs numerical approximations to the solution and free boundary of HJB variational inequalities with monotone drivers, which arise from mixed stochastic control/optimal stopping problems with recursive preferences and infinite activity jumps. The schemes are based on the penalty method, monotone discretizations and policy iteration. We prove the convergence of the numerical scheme and illustrate the theoretical results with some numerical examples including an optimal
References (40)
- et al.
Parabolic variational inequalities: the Lagrange multiplier approach
J. Math. Pures Appl.
(2006) - et al.
Continuous dependence estimates for viscosity solutions of integro-PDEs
J. Differ. Equ.
(2005) - et al.
BSDEs with jumps, optimization and applications to dynamic risk measures
Stoch. Process. Appl.
(2013) Backward stochastic differential equations with jumps and related non-linear expectations
Stoch. Process. Appl.
(2006)On diagonal dominance arguments for bounding
Linear Algebra Appl.
(1976)- et al.
A semismooth Newton method for a kind of HJB equation
Comput. Math. Appl.
(2017) - et al.
Infinite Dimensional Analysis: A Hitchhiker's Guide
(2006) The Obstacle Problem for Nonlinear Integro-differential Equations Arising in Option Pricing
(2000)Nonlinear integro-differential evolution problems arising in option pricing: a viscosity solution approach
Differ. Integral Equ.
(2003)- et al.
Convergence of approximation schemes for fully nonlinear second order equations
Asymptot. Anal.
(1991)