A McKean–Vlasov approach to distributed electricity generation development

Abstract

This paper analyses the interaction between centralised carbon emissive technologies and distributed intermittent non-emissive technologies. In our model, there is a representative consumer who can satisfy her electricity demand by investing in distributed generation (solar panels) and by buying power from a centralised firm at a price the firm sets. Distributed generation is intermittent and induces an externality cost to the consumer. The firm provides non-random electricity generation subject to a carbon tax and to transmission costs. The objective of the consumer is to satisfy her demand while minimising investment costs, payments to the firm and intermittency costs. The objective of the firm is to satisfy the consumer’s residual demand while minimising investment costs, demand deviation costs, and maximising the payments from the consumer. We formulate the investment decisions as McKean–Vlasov control problems with stochastic coefficients. We provide explicit, price model-free solutions to the optimal decision problems faced by each player, the solution of the Pareto optimum, and the Stackelberg equilibrium where the firm is the leader. We find that, from the social planner’s point of view, the carbon tax or transmission costs are necessary to justify a positive share of distributed capacity in the long-term, whatever the respective investment costs of both technologies are. The Stackelberg equilibrium is far from the Pareto equilibrium and leads to an over-investment in distributed energy and to a much higher price for centralised energy.

A Appendix

1.1 A.1 Proof of Theorem 3.1

Given a candidate in the form of (3.7), our goal is to set the coefficients $K,\Lambda ,Y,R$ so as to satisfy the condition in (3.9). By applying the Itô formula and completing the square with respect to the control, we get (3.10), where the coefficients are defined by

$$\begin{aligned} F(k, \dot{k})= & {} \dot{k} - \frac{b^2k^2}{\gamma } + (\sigma ^2 - \rho )k + \eta , \\ G(k,\lambda , \dot{\lambda })= & {} \dot{\lambda } -\frac{b^2 \lambda ^2}{\gamma } -\rho \lambda +\sigma ^2k, \\ H_t(k, \lambda , y, \bar{y}, \dot{y})= & {} \dot{y} - \rho y -\frac{b^2 k}{\gamma } (y - \bar{y}) - \frac{b \lambda }{\gamma } (c+ b \bar{y}) - (1+\theta ) P_t, \\ M(y, r, \dot{r})= & {} \dot{r} - \rho r - \frac{1}{4\gamma } (c+ b y)^2, \\ A(x, \bar{x}, k , \lambda , y)= & {} - \frac{b k}{\gamma }(x - \bar{x}) -\frac{b \lambda }{\gamma } \bar{x} - \frac{b}{2\gamma } y - \frac{c}{2\gamma }. \end{aligned}$$

Condition (3.11) then leads to the following equations for the coefficients:

$$\begin{aligned} dK_t= & {} \Big ( \frac{b^2}{\gamma }K_t^2 + (\rho - \sigma ^2) K_t -\eta \Big ) dt, \\ d\Lambda _t= & {} \Big (\frac{b^2}{\gamma } \Lambda _t^2 +\rho \Lambda _t -\sigma ^2 K_t \Big ) dt, \\ dY_t= & {} \Big ( \rho Y_t + \frac{b^2 K_t}{\gamma }(Y_t-\mathbb {E}[Y_t]) +\frac{b\Lambda _t}{\gamma }(c+b\mathbb {E}[Y_t])+(1+\theta )P_t\Big )dt + Z^Y_t dW^0_t, \\ dR_t= & {} \Big ( \rho R_t + \frac{1}{4\gamma } (c+ b Y_t)^2 \Big ) dt + Z^R_t dW^0_t. \end{aligned}$$

The first and second equations admit constant strictly positive solutions $K_t \equiv K$ and $\Lambda _t \equiv \Lambda$, with $K,\Lambda$ as in (3.15). The third and the fourth equations are linear Backward Stochastic Differential Equations (BSDEs), with the immediate solution (for the coefficient $Y_t$, first compute $Y_t - \mathbb {E}[Y_t]$) given by:

$$\begin{aligned} Y_t= & {} - (1+\theta ) \int _t^\infty e^{-(\rho + b^2K/\gamma )(s-t)} \mathbb {E}[ P_s| \mathcal {F}^0_t] ds \nonumber \\&- (1+\theta ) \int _t^\infty \Big ( e^{-(\rho + b^2\Lambda /\gamma )(s-t)} -e^{-(\rho + b^2K/\gamma )(s-t)} \Big ) \mathbb {E}[P_s] ds -\frac{bc\Lambda }{\rho \gamma + b^2\Lambda }, \nonumber \\ R_t= & {} - \frac{1}{4\gamma } \int _t^\infty e^{-\rho (s-t)} \mathbb {E}[(bY_s + c)^2| \mathcal {F}^0_t] ds. \end{aligned}$$

(A.1)

Furthermore, $Y_t$ is well-defined by the Holder inequality and (3.5), whereas (A.2) will imply that $R_t<\infty$. We get (3.13) by plugging (A.1) into (3.12), while (3.14) immediately follows by (2.1) and (3.13). By the procedure above, conditions (ii) and (iii) in Lemma 3.1 are verified (we will later prove that $\hat{\alpha }$ actually lies in ${\mathcal {A}}$). As for condition (i), we have to prove that

$$\begin{aligned} \lim _{t \rightarrow \infty } \mathbb {E}\big [ e^{-\rho t} (|Y_t|^2 + |R_t|)\big ] \rightarrow 0. \end{aligned}$$

It immediately follows from the definition that $\mathbb {E}[ e^{-\rho t} |R_t|] \rightarrow 0$ as $t \rightarrow \infty$; as for the process Y, we prove the stronger condition

$$\begin{aligned} \mathbb {E}\Big [ \int _0^\infty e^{-\rho t} |Y_t|^2 dt \Big ] < \infty . \end{aligned}$$

(A.2)

By Jensen’s inequality, Fubini’s theorem, the law of iterated conditional expectations, and (3.5), we have:

$$\begin{aligned}&\mathbb {E}\Big [ \int _0^\infty e^{-\rho t} \left( \int _t^\infty e^{-(\rho + b^2 K / \gamma )(s-t)} \mathbb {E}[P_s|{\mathcal {F}}_t^0] ds \right) ^2 dt \Big ] \\&\quad \le \frac{1}{\rho + b^2 K / \gamma } \int _0^\infty \int _t^\infty e^{-\rho t} e^{-(\rho + b^2 K / \gamma )(s-t)} \mathbb {E}[|P_s|^2] ds dt \\&\quad \le \frac{1}{(\rho + b^2 K / \gamma )b^2 K / \gamma } \int _0^\infty e^{-\rho s} \mathbb {E}[|P_s|^2] ds < \infty . \end{aligned}$$

We deal with the other terms in $Y_t$ by the same arguments and obtain the required integrability condition (A.2). Let us finally show that $\hat{\alpha } \in {\mathcal {A}}$, i.e. $\int _0^\infty e^{-\rho t}\mathbb {E}[|\hat{\alpha }_t|^2] dt < \infty$. Recall that $\hat{\alpha }$ is written as

$$\begin{aligned} \hat{\alpha }_t = - \frac{b K}{\gamma }\hat{X}_t + U_t, \end{aligned}$$

with

$$\begin{aligned} U_t := \frac{b (K - \Lambda )}{\gamma } \mathbb {E}[\hat{X}_t] -\frac{b}{2\gamma } Y_t - \frac{c}{2\gamma }. \end{aligned}$$

By Itô’s formula to (2.1) for $\alpha = \hat{\alpha }$, Young’s inequality, and Gronwall’s lemma, we have:

$$\begin{aligned} \mathbb {E}[ e^{-\rho t} |\hat{X}_t|^2 ] \le \Big ( x_0^2 + \frac{b}{\epsilon } \int _0^t e^{-\rho s} \mathbb {E}[|U_s|^2] ds \Big ) e^{-(\rho - \sigma ^2 - b\epsilon )t}, \;\;\; t \ge 0, \end{aligned}$$

for all $\epsilon > 0$. For $\epsilon$ small enough, and under (3.3), we have $\rho - \sigma ^2 - b\epsilon > 0$, and thus it suffices to show that $\int _0^\infty e^{-\rho t} \mathbb {E}[|U_t|^2] dt < \infty$ to ensure the square-integrability condition: $\int _0^\infty e^{-\rho t} \mathbb {E}[|\hat{X}_t|^2] dt < \infty$, and so $\hat{\alpha } \in {\mathcal {A}}$. From the integrability condition (A.2) on Y, we then have to check that $\int _0^\infty e^{-\rho t} \big (\mathbb {E}[\hat{X}_t]\big )^2 dt < \infty$, hence by (3.14) that (we set $B = b^2\Lambda /\gamma$)

$$\begin{aligned} I := \int _0^\infty e^{-\rho t} \bigg ( e^{-Bt} \int _0^t e^{Bs} \int _s^\infty e^{-(\rho + B)(u-s)} \mathbb {E}[P_u] du \, ds \bigg )^2 dt <\infty . \end{aligned}$$

By Fubini-Tonelli we can rewrite I as:

$$\begin{aligned} I = \frac{1}{(\rho +2B)^2} \int _0^\infty e^{-(\rho + 2B) t} \bigg (\int _0^\infty \big (e^{(\rho + 2B) \min \{t,u\}}-1\big ) e^{-(\rho + B)u} \mathbb {E}[P_u] du \bigg )^2 dt; \end{aligned}$$

by writing $e^{-(\rho + B)u} = e^{-2\epsilon u}e^{-(\rho + B -2\epsilon )u}$ for $\epsilon >0$, by Jensen’s inequality w.r.t. the measure $e^{-2\epsilon u} du$ and by Fubini-Tonelli, we get:

$$\begin{aligned} I \le \frac{1}{\epsilon (\rho +2B)^2} \int _0^\infty \left( \int _0^\infty e^{-(\rho + 2B)(t+u-2\min \{t,u\})} dt \right) e^{-(\rho -2\epsilon )u}\mathbb {E}[P_u^2] du, \end{aligned}$$

which finally leads to

$$\begin{aligned} I \le \frac{2}{\epsilon (\rho + 2B)^3} \int _0^\infty e^{-(\rho - 2\epsilon )u}\mathbb {E}[P_u^2] du. \end{aligned}$$

Hence, from (3.5) and for $\epsilon$ small enough we have $I < \infty$ and then $\hat{\alpha } \in {\mathcal {A}}$. $\square$

1.2 A.2 Proof of Theorem 3.2

A suitable adaptation of Lemma 3.1 holds. We look for a candidate in the form of:

$$\begin{aligned} v_t(q) = K_t q^2 + Y_t q + R_t, \end{aligned}$$

(A.3)

where the dynamics of the coefficients K, Y, R are given by

$$\begin{aligned} dK_t = \dot{K}_t dt, \qquad dY_t = \dot{Y}_t dt + Z_t^Y dW_t, \qquad dR_t = \dot{R}_t dt + Z_t^R dW_t, \end{aligned}$$

for some deterministic process $\dot{K}$ and $\mathbb {F}^0$-adapted processes $\dot{Y}$, $\dot{R}$, $Z^Y$, $Z^R$. As in “Appendix A.1”, we have assumed the quadratic coefficient to be deterministic. Moreover, since the randomness comes from the $\mathbb {F}$-adapted process $X^\alpha$, the stochastic part in Y, R only depends on W.

By applying Itô’s formula to $S_t^\nu = e^{-\rho t} v_t(Q_t^\nu ) +\int _0^t e^{-\rho s} \ell _s(Q^\nu _s,\nu _s)ds$ and completing the square, we get $d\mathbb {E}[S_t^\nu ] = e^{-\rho t} \mathbb {E}[ {\mathcal {D}}_t^\nu ] dt$, with

$$\begin{aligned} \mathbb {E}[\mathcal {D}^\nu _t]= & {} \mathbb {E}\Big [ h \big (\nu _t - A(Q^\nu _y, K_t, Y_t) \big )^2 +F(K_t, \dot{K}_t) (Q^\nu _t)^2 \\&\quad \quad + H_t(K_t,Y_t,\dot{Y}_t) Q^\nu _t + M(Y_t, R_t, \dot{R}_t) \Big ], \end{aligned}$$

where the coefficients are defined by

$$\begin{aligned} F(k, \dot{k})= & {} \dot{k} - \frac{k^2}{h} - \rho k + \lambda , \\ H_t(k, y, \dot{y})= & {} \dot{y} - \big ( \rho + \frac{k}{h} \big ) y -(2\lambda -\pi _1/D)\big (D - X^\alpha _t\big ), \\ M(y, r, \dot{r})= & {} \dot{r} - \rho r - \frac{y^2}{4h}, \\ A(q, k, y)= & {} - \frac{k}{h} q - \frac{y}{2h}. \end{aligned}$$

Setting $\mathbb {E}[\mathcal {D}^\nu _t] \ge 0$ for each $\nu$ and $\mathbb {E}[\mathcal {D}^\nu _t] = 0$ for $\nu = \hat{\nu }$ provides the optimal control

$$\begin{aligned} \hat{\nu }_t = A(Q^{\hat{\nu }}_t, K_t, Y_t) \end{aligned}$$

(A.4)

and the following equations for the coefficients:

$$\begin{aligned} dK_t= & {} \bigg ( \frac{K_t^2}{h} + \rho K_t - \lambda \bigg ) dt, \\ dY_t= & {} \bigg ( \Big (\rho + \frac{K_t}{h}\Big ) Y_t + 2\lambda \Big (1-\frac{\pi _1}{2\lambda D}\Big )\big (D - X^\alpha _t\big ) \bigg ) dt + Z^Y_t dW_t, \\ dR_t= & {} \bigg ( \rho R_t + \frac{1}{4h}Y^2_t \bigg ) dt + Z^R_t dW_t. \end{aligned}$$

The first equation admits a constant solution $K_t \equiv \tilde{K}$, with $\tilde{K}$ as in (3.21), while the second and third equations are linear BSDEs, with immediate solutions:

$$\begin{aligned} \begin{aligned}&Y_t = - 2\lambda \Big (1-\frac{\pi _1}{2\lambda D}\Big ) \int _t^\infty e^{-(\rho + \tilde{K}/h)(s-t)} \mathbb {E}[ D - X^\alpha _s| \mathcal {F}_t] ds, \\&R_t = - \frac{1}{4h} \int _t^\infty e^{-\rho (s-t)} \mathbb {E}[Y_s^2| \mathcal {F}_t] ds. \end{aligned} \end{aligned}$$

We get (3.19) by (A.4) and (A.2), whereas (3.20) immediately follows by (3.19) and (2.3). Moreover, the conditions (i) and (iii) in Lemma 3.1 are verified by standard estimates as in “Appendix A.1”. Finally, we get the limit result by the same computations as the ones in Proposition 3.1. $\square$

1.3 A.3 Proof of Theorem 3.3

A suitable adaptation of Lemma 3.1 holds. We use vector notations and a small change of variable by setting:

$$\begin{aligned} \delta _t = \begin{pmatrix} b \alpha _t \\ \nu _t \end{pmatrix}, \qquad Z^\delta _t =\begin{pmatrix} X^\alpha _t \\ Q^\nu _t \end{pmatrix}, \end{aligned}$$

so that the dynamics of $Z^\delta$ are written as

$$\begin{aligned} dZ^\delta _t = \delta _t dt + S Z^\delta _t dW_t, \qquad S= \begin{pmatrix} \sigma &{} 0 \\ 0 &{} 0 \\ \end{pmatrix}. \end{aligned}$$

(A.5)

The payoff is rewritten as $\tilde{g}_t(Z^\delta _t, \mathbb {E}[Z^\delta _t], \delta _t)$, where

$$\begin{aligned} \tilde{g}_t(z, \bar{z}, d) = (z- \bar{z})'Q(z-\bar{z}) + \bar{z}' (Q +\tilde{Q}) \bar{z} + T_t' z + d'N d + d'U, \end{aligned}$$

where $'$ denotes transposition and the coefficients are defined by (notice that $Q,\tilde{Q}, N, U$ are constant, T is stochastic and $\mathbb {F}^0$-adapted)

$$\begin{aligned} Q= & {} \begin{pmatrix} \lambda +\eta -\frac{\pi _0}{D} &{} \lambda -\frac{\pi _1}{2D} \\ \lambda -\frac{\pi _1}{2D} &{} \lambda \\ \end{pmatrix}, \qquad \tilde{Q}=\begin{pmatrix} -\eta &{} 0 \\ 0 &{} 0 \\ \end{pmatrix}, \\ T_t= & {} \begin{pmatrix} -2\lambda D - \theta P_t + \pi _0 \\ -2\lambda D + \pi _1\\ \end{pmatrix}, \qquad N=\begin{pmatrix} \frac{\gamma }{b^2} &{} 0 \\ 0 &{} h \\ \end{pmatrix}, \qquad U=\begin{pmatrix} \frac{c}{b} \\ 0 \\ \end{pmatrix}. \end{aligned}$$

Correspondingly, we consider candidates in the form of:

$$\begin{aligned} v_t(z,\bar{z}) = (z-\bar{z})' K_t (z-\bar{z}) + \bar{z}' \Lambda _t \bar{z} + Y_t' z + R_t, \end{aligned}$$

(A.6)

where the dynamics of the coefficients $K,\Lambda , Y,R$ are given by

$$\begin{aligned} dK_t {=} \dot{K}_t dt, \qquad d\Lambda _t {=} \dot{\Lambda }_t dt, \qquad dY_t {=} \dot{Y}_t dt + Z_t^Y dW^0_t, \qquad dR_t {=}\dot{R}_t dt + Z_t^R dW^0_t, \end{aligned}$$

for some symmetric deterministic processes $\dot{K}, \dot{\Lambda }$ and $\mathbb {F}^0$-adapted processes $\dot{Y}$, $\dot{R}$, $Z^Y$, $Z^R$. As in “Appendix A.1”, we have assumed the quadratic coefficient to be deterministic. Moreover, since the randomness only comes from the $\mathbb {F}^0$-adapted process T, the stochastic part in Y, R only depends on $W^0$.

By applying Itô’s formula to $S_t^\delta = e^{-\rho t} v_t(Z_t^\delta , \mathbb {E}[Z_t^\delta ]) + \int _0^t e^{-\rho s} \tilde{g}_s(Z^\delta _s, \mathbb {E}[Z^\delta _s], \delta _s)ds$ and completing the square, we get $d\mathbb {E}[S_t^\delta ] = e^{-\rho t} \mathbb {E}[ {\mathcal {D}}_t^\delta ] dt$, with

$$\begin{aligned} \mathbb {E}[{\mathcal {D}}_t^\delta ]= & {} \mathbb {E}\Big [ \big ( \delta _t - A(Z_t^\delta ,\mathbb {E}[Z_t^\delta ], K_t,\Lambda _t,Y_t) \big )' N \big ( \delta _t - A(Z_t^\delta ,\mathbb {E}[Z_t^\delta ], K_t,\Lambda _t,Y_t) \big ) \nonumber \\ \end{aligned}$$

(A.7)

$$\begin{aligned}&+\, \big (Z_t^\delta - \mathbb {E}[Z_t^\delta ] \big )' F(K_t, \dot{K}_t) \big (Z_t^\delta - \mathbb {E}[Z_t^\delta ] \big ) \nonumber \\&+\, \mathbb {E}[Z_t^\delta ]' G(K_t, \Lambda _t, \dot{\Lambda }_t) \mathbb {E}[Z_t^\delta ] \nonumber \\&+\, H_t(K_t, \Lambda _t, Y_t,\mathbb {E}[Y_t],\dot{Y}_t)' Z_t^\delta + M(Y_t, R_t, \dot{R}_t) \Big ], \end{aligned}$$

(A.8)

where the coefficients are defined by

$$\begin{aligned} F(k, \dot{k})= & {} \dot{k} - kN^{-1}k + SkS - \rho k + Q, \\ G(k,\lambda , \dot{\lambda })= & {} \dot{\lambda } -\lambda N^{-1} \lambda -\rho \lambda + SkS + Q + \tilde{Q}, \\ H_t(k, \lambda , y, \bar{y}, \dot{y})= & {} \dot{y} - \rho y - k N^{-1} (y - \bar{y}) - \lambda N^{-1} (U + \bar{y}) + T_t, \\ M(y, r, \dot{r})= & {} \dot{r} - \rho r - \frac{1}{4} (U + y)'N^{-1}(U+y), \\ A(z, \bar{z}, k, \lambda , y)= & {} - N^{-1}k(z - \bar{z}) - N^{-1} \lambda \bar{z} - N^{-1}(U+y)/2. \end{aligned}$$

Setting $\mathbb {E}[\mathcal {D}^\delta _t] \ge 0$ for each $\delta$ and $\mathbb {E}[\mathcal {D}^\delta _t] = 0$ for $\delta = \delta ^{*}$ provides the optimal control

$$\begin{aligned} \delta ^{*}_t = A(Z_t^{\delta ^{*}},\mathbb {E}[Z_t^{\delta ^{*}}],K_t,\Lambda _t,Y_t) \end{aligned}$$

(A.9)

and the following equations for the coefficients

$$\begin{aligned} dK_t= & {} \Big ( K_tN^{-1}K_t - SK_tS +\rho K_t - Q\Big ) dt, \\ d\Lambda _t= & {} \Big (\Lambda _tN^{-1}\Lambda _t + \rho \Lambda _t - SK_tS - Q - \tilde{Q} \Big ) dt, \\ dY_t= & {} \Big (\rho Y_t + K_t N^{-1} (Y_t - \mathbb {E}[Y_t]) +\Lambda _t N^{-1}(U + \mathbb {E}[Y_t]) - T_t \Big ) dt + Z^Y_t dW^0_t,\\ dR_t= & {} \Big (\rho R_t + \frac{1}{4}(U + Y_t)' N^{-1}(U + Y_t) \Big ) dt+ Z^R_t dW^0_t. \end{aligned}$$

The first and the second equation admit constant solutions $K_t \equiv K$ and $\Lambda _t \equiv \Lambda$: there exists a unique couple of symmetric positive-definite matrices $(K,\Lambda )$ which solves the equations

$$\begin{aligned} KN^{-1}K - SKS +\rho K - Q= & {} 0, \end{aligned}$$

(A.10)

$$\begin{aligned} \Lambda N^{-1}\Lambda +\rho \Lambda -SKS-Q-\tilde{Q}= & {} 0, \end{aligned}$$

(A.11)

i.e. the systems (3.27) (we postpone the proof to the end of this “Appendix”). The third and the fourth equations are linear BSDEs, with immediate solution (for the coefficient $Y_t$, first compute $Y_t - \mathbb {E}[Y_t]$):

$$\begin{aligned} Y_t= & {} \int _t^\infty e^{-(\rho \,\text {Id} + K N^{-1})(s-t)} \mathbb {E}[ T_s| \mathcal {F}^0_t] ds \nonumber \\&+ \int _t^\infty \Big ( e^{-(\rho \,\text {Id} + \Lambda N^{-1})(s-t)} -e^{-(\rho \,\text {Id} + K N^{-1})(s-t)} \Big )\mathbb {E}[T_s] ds\nonumber \\&+ (\rho \,\text {Id} + \Lambda N^{-1})^{-1}(\rho U) - U. \nonumber \\ R_t= & {} - \frac{1}{4} \int _t^\infty e^{-\rho (s-t)} \mathbb {E}[(U + Y_s)' N^{-1}(U + Y_s)| \mathcal {F}^0_t] ds. \end{aligned}$$

(A.12)

Since $K,\Lambda >0$, the matrices $\rho \,\text {Id} + K N^{-1}$ and $\rho \,\text {Id} + \Lambda N^{-1}$ are positive-definite and invertible; then, by arguing as in “Appendix A.1”, the processes $Y_t$ and $R_t$ are well-defined. We get (3.26) by plugging (A.12) into (A.9) and setting $\tilde{Y} = Y+U$, whereas (3.28) immediately follows by (A.5) and (3.26). Moreover, the conditions (i) and (iii) in Lemma 3.1 are verified by standard estimates as in “Appendix A.1”. Further, we get the limit result by the same computations as the ones in Proposition 3.1. $\square$

Lemma A.1

There exists a unique couple of symmetric positive-definite matrices $(K,\Lambda )$ which solves the Eqs. (A.10)–(A.11).

Proof

Uniqueness immediately follows from (A.6). To prove the existence of the solutions, we link (A.10)–(A.11) to suitable control problems. For $T \in \mathbb {R}^+ \cup \{\infty \}$ and $x\in \mathbb {R}^2$, we consider

$$\begin{aligned}&V_T(x)= \inf _{u \in \mathcal {U}_T} \mathbb {E}\left[ \int _0^T e^{-\rho s} (X'_s Q X_s + u'_s N^{-1} u_s) ds\right] , \\&\mathcal {U}_T = \Big \{\mathbb {R}^2\text {-valued adapted} \ u = \{u_s\}_{s \in [0,T]}\ \text {s.t. } \mathbb {E}\Big [ \int _0^T e^{-\rho s} |u_s|^2 ds \Big ] < \infty \Big \}, \\&dX_s = u_s ds + S X_s dW_s, \qquad X_0=x. \end{aligned}$$

By arguing as in (3.4), it is easy to see that $u \in \mathcal {U}_T$ implies $\mathbb {E}\big [ \int _0^T e^{-\rho s} |X_s|^2 ds \big ] < \infty$, so that the problems are well-defined. If T is finite, we know [see (Yong and Zhou 1999, Sections 6.6 and 6.7), with a straightforward adaptation of the arguments to include the discount factor] that there exists a unique solution $\{K_{t;T}\}_{t \in [0,T]}$ to

$$\begin{aligned} {\left\{ \begin{array}{ll} \frac{d}{dt} K_{t;T} = \rho K_{t;T} - SK_{t;T}S - Q + K_{t;T} N^{-1} K_{t;T}, \\ K_{T;T}=0, \end{array}\right. } \end{aligned}$$

(A.13)

and that for every $x \in \mathbb {R}^2$ we have

$$\begin{aligned} V_T(x) = x'K_{0;T}x. \end{aligned}$$

It is easy to see that $V_T \rightarrow V_\infty$ as $T \rightarrow \infty$; as a consequence, there exists

$$\begin{aligned} \lim _{T \rightarrow \infty } K_{0;T} =: \tilde{K}. \end{aligned}$$

By a classical argument and since the functions $\{K_{t;T}\}_{t \in [0,T]}$ solve (A.13), $\tilde{K}$ is a solution to (A.10). Also, $\tilde{K}$ is symmetric as it is the limit of symmetric matrices. Moreover, notice that $N^{-1}>0$ and assume that $Q>0$; by standard arguments in control theory, it follows that $x'\tilde{K} x = V_\infty (x)>0$ for each $x \ne 0$, so that $\tilde{K}>0$. Hence, we have proved that (A.10) admits a symmetric positive-definite solution provided that $Q>0$, which is true by (3.25).

By similar arguments, (A.11) admits a symmetric positive-definite solution provided that $SKS + Q + \tilde{Q}>0$, which is true by (3.25), since $K^{11}>0$. $\square$