On Lemke processibility of LCP formulations for solving discounted switching control stochastic games

Abstract

Schultz (J Optim Theory Appl 73(1):89–99, 1992) formulated 2-person, zero-sum, discounted switching control stochastic games as a linear complementarity problem (LCP) and discussed computational results. It remained open to prove or disprove Lemke-processibility of this LCP. We settle this question by providing a counter example to show that Lemke’s algorithm does not always successfully process this LCP.We propose a new LCP formulation with the aim of making the underlying matrix belong to the classes R$_{0}$ and E$_{0}$, which would imply Lemke processibility. While the underlying matrix in the new formulation is not $E_0$, we show that it is an R$_{0}$-matrix. Successful processing of Lemke’s algorithm depends on the choice of the initial vector d. Because of the special structure of the LCP in our context, we may, in fact, be able to find a suitable d such that our LCPs are processible by Lemke’s algorithm. We leave this open.

Appendix A

In this appendix, we describe our reformulation of Schultz’s LCP formulation.

$w - Mz = q = -c$ where $M = \left[ \begin{array}{cc} R &{} B \\ A &{} 0 \\ \end{array}\right]$, where

$$\begin{aligned} R= & {} \left[ \begin{array}{cccccccc} &{} &{} &{} &{} -R(1) &{} 0_{m_{1}\times n_{2}} &{} \cdots &{} 0_{m_{1}\times n_{N}} \\ \\ &{} 0_{m\times n} &{} &{} &{} 0_{m_{2}\times n_{1}} &{} -R(2) &{} \cdots &{} 0_{m_{2}\times n_{N}} \\ &{} &{} &{} &{} \vdots &{} \vdots &{} &{} \vdots \\ &{} &{} &{} &{} 0_{m_{N}\times n_{1}} &{} 0_{m_{N}\times n_{2}} &{} \cdots &{} -R(N) \\ \\ R^t(1) &{} 0_{n_{1}\times m_{2}} &{} \cdots &{} 0_{n_{1}\times m_{N}} &{} &{} &{} &{} \\ \\ 0_{n_{2}\times m_{1}} &{} R^t(2) &{} \cdots &{} 0_{n_{2}\times m_{N}} &{} &{} 0_{n\times m} &{} &{} \\ \vdots &{} \vdots &{} &{} \vdots &{} &{} &{} &{} \\ 0_{n_{N}\times m_{1}} &{} 0_{n_{N}\times m_{2}} &{} \cdots &{} R^t(N) &{} &{} &{} &{} \end{array}\right] \\ B= & {} \left[ \begin{array}{cccc} (B_{1}-B_{2}) &{} (B_{2}-B_{1}) &{} B_{3} &{} -B_{3} \\ \end{array} \right] where\\ B_{1}= & {} \left[ \begin{array}{cccccccc} e_{m_{1}} &{} 0_{m_{1}} &{} 0_{m_{1}} &{} \cdots &{} 0_{m_{1}} &{} 0_{m_{1}} &{} \cdots &{} 0_{m_{1}} \\ \\ 0_{m_{1}} &{} e_{m_{2}} &{} 0_{m_{2}} &{} \cdots &{} 0_{m_{2}} &{} 0_{m_{2}} &{} \cdots &{} 0_{m_{2}} \\ \vdots &{} \vdots &{} \vdots &{} &{} \vdots &{} \vdots &{} &{} \vdots \\ 0_{m_{N_{1}}} &{} 0_{m_{N_{1}}} &{} 0_{m_{N_{1}}} &{} \cdots &{} e_{m_{N_{1}}} &{} 0_{m_{N_{1}}} &{} \cdots &{} 0_{m_{N_{1}}} \\ \\ 0_{m_{P_{2}}} &{} 0_{m_{P_{2}}} &{} 0_{m_{P_{2}}} &{} \cdots &{} 0_{m_{P_{2}}} &{} 0_{m_{P_{2}}} &{} \cdots &{} 0_{m_{P_{2}}} \\ \\ 0_{n_{P_{1}}} &{} 0_{n_{P_{1}}} &{} 0_{n_{P_{1}}} &{} \cdots &{} 0_{n_{P_{1}}} &{} 0_{n_{P_{1}}} &{} \cdots &{} 0_{n_{P_{1}}} \\ \\ 0_{n_{N_{1}+1}} &{} 0_{n_{N_{1}+1}} &{} 0_{n_{N_{1}+1}} &{} \cdots &{} 0_{n_{N_{1}+1}} &{} -e_{n_{N_{1}+1}} &{} \cdots &{} 0_{n_{N_{1}+1}} \\ \vdots &{} \vdots &{} \vdots &{} &{} \vdots &{} \vdots &{} &{} \vdots \\ 0_{n_{N}} &{} 0_{n_{N}} &{} 0_{n_{N}} &{} \cdots &{} 0_{n_{N}} &{} 0_{n_{N}} &{} \cdots &{} -e_{n_{N}} \\ \end{array} \right] \end{aligned}$$

where ${m_{P_{2}}}$ is the number of actions for player 1 in player 2 controlled states. That is, ${m_{P_{2}}} = \sum _{s_{2} \in S_{2}}m_{s_{2}}$. Similarly, ${n_{P_{1}}}$ is the number of actions for player 2 in player 1 controlled states. That is, ${n_{P_{1}}} = \sum _{s_{1} \in S_{1}}n_{s_{1}}$.

$$\begin{aligned} B_{2}= & {} \left[ \begin{array}{cccccc} \beta Q_{1}(1,1) &{} \cdots &{} \beta Q_{1}(1,N_{1}) &{} \beta Q_{1}(1,N_{1}+1) &{} \cdots &{} \beta Q_{1}(1,N) \\ \\ \beta Q_{1}(2,1) &{} \cdots &{} \beta Q_{1}(2,N_{1}) &{} \beta Q_{1}(2,N_{1}+1) &{} \cdots &{} \beta Q_{1}(2,N) \\ \vdots &{} &{} \vdots &{} \vdots &{} &{} \vdots \\ \beta Q_{1}(N_{1},1) &{} \cdots &{} \beta Q_{1}(N_{1},N_{1}) &{} \beta Q_{1}(N_{1},N_{1}+1) &{} \cdots &{} \beta Q_{1}(N_{1},N) \\ \\ 0_{m_{N_{1}+1}} &{} \cdots &{} 0_{m_{N_{1}+1}} &{} -e_{m_{N_{1}+1}} &{} \cdots &{} 0_{m_{N_{1}+1}} \\ \vdots &{} &{} \vdots &{} \vdots &{} &{} \vdots \\ 0_{m_{N}} &{} \cdots &{} 0_{m_{N}} &{} 0_{m_{N}} &{} \cdots &{} -e_{m_{N}}\\ \\ e_{n_{1}} &{} \cdots &{} 0_{n_{1}} &{} 0_{n_{1}} &{} \cdots &{} 0_{n_{1}} \\ \vdots &{} &{} \vdots &{} \vdots &{} &{} \vdots \\ 0_{n_{N_{1}}} &{} \cdots &{} e_{n_{N_{1}}} &{} 0_{n_{N_{1}}} &{} \cdots &{} 0_{n_{N_{1}}} \\ \\ -\beta Q_{2}(N_{1}+1,1) &{} \cdots &{} -\beta Q_{2}(N_{1}+1,N_{1}) &{} -\beta Q_{2}(N_{1}+1,N_{1}+1) &{} \cdots &{} -\beta Q_{2}(N_{1}+1,N) \\ \vdots &{} &{} \vdots &{} \vdots &{} &{} \vdots \\ -\beta Q_{2}(N,1) &{} \cdots &{} -\beta Q_{2}(N,N_{1}) &{} -\beta Q_{2}(N,N_{1}+1) &{} \cdots &{} -\beta Q_{2}(N,N) \\ \end{array} \right] \\ B_{3}= & {} \left[ \begin{array}{cccccc} 0_{m} &{} \cdots &{} 0_{m} &{} 0_{m} &{} \cdots &{} 0_{m} \\ \\ 0_{m_{N_{1}+1}} &{} \cdots &{} 0_{m_{N_{1}+1}} &{} -e_{m_{N_{1}+1}} &{} \cdots &{} 0_{m_{N_{1}+1}} \\ \vdots &{} &{} \vdots &{} \vdots &{} &{} \vdots \\ 0_{m_{N}} &{} \cdots &{} 0_{m_{N}} &{} 0_{m_{N}} &{} \cdots &{} -e_{m_{N}}\\ \\ e_{n_{1}} &{} \cdots &{} 0_{n_{1}} &{} 0_{n_{1}} &{} \cdots &{} 0_{n_{1}} \\ \vdots &{} &{} \vdots &{} \vdots &{} &{} \vdots \\ 0_{n_{N_{1}}} &{} \cdots &{} e_{n_{N_{1}}} &{} 0_{n_{N_{1}}} &{} \cdots &{} 0_{n_{N_{1}}} \\ \\ 0_{n} &{} \cdots &{} 0_{n} &{} 0_{n} &{} \cdots &{} 0_{n} \\ \\ \end{array} \right] \\ A= & {} \left[ \begin{array}{cccccccccccc} -e^t_{m_{1}} &{} \cdots &{} 0^t_{m_{N_{1}}} &{} 0^t_{m_{N_{1}+1}} &{} \cdots &{} 0^t_{m_{N}} &{} e^t_{n_{1}} &{} \cdots &{} 0^t_{n_{N_{1}}} &{} 0^t_{n_{N_{1}+1}} &{} \cdots &{} 0^t_{n_{N}}\\ \\ e^t_{m_{1}} &{} \cdots &{} 0^t_{m_{N_{1}}} &{} 0^t_{m_{N_{1}+1}} &{} \cdots &{} 0^t_{m_{N}} &{} -e^t_{n_{1}} &{} \cdots &{} 0^t_{n_{N_{1}}} &{} 0^t_{n_{N_{1}+1}} &{} \cdots &{} 0^t_{n_{N}} \\ \vdots &{} &{} \vdots &{} \vdots &{} &{} \vdots &{} \vdots &{} &{} \vdots &{} \vdots &{} &{} \vdots \\ 0^t_{m_{1}} &{} \cdots &{} 0^t_{m_{N_{1}}} &{} 0^t_{m_{N_{1}+1}} &{} \cdots &{} -e^t_{m_{N}} &{} 0^t_{n_{1}} &{} \cdots &{} 0^t_{n_{N_{1}}} &{} 0^t_{n_{N_{1}+1}} &{} \cdots &{} e^t_{n_{N}}\\ \\ 0^t_{m_{1}} &{} \cdots &{} 0^t_{m_{N_{1}}} &{} 0^t_{m_{N_{1}+1}} &{} \cdots &{} e^t_{m_{N}} &{} 0^t_{n_{1}} &{} \cdots &{} 0^t_{n_{N_{1}}} &{} 0^t_{n_{N_{1}+1}} &{} \cdots &{} -e^t_{n_{N}} \\ \\ 0^t_{m_{1}} &{} \cdots &{} 0^t_{m_{N_{1}}} &{} -e^t_{m_{N_{1}+1}} &{} \cdots &{} 0^t_{m_{N}} &{} 0^t_{n_{1}} &{} \cdots &{} 0^t_{n_{N_{1}}} &{} 0^t_{n_{N_{1}+1}} &{} \cdots &{} 0^t_{n_{N}}\\ \\ 0^t_{m_{1}} &{} \cdots &{} 0^t_{m_{N_{1}}} &{} e^t_{m_{N_{1}+1}} &{} \cdots &{} 0^t_{m_{N}} &{} 0^t_{n_{1}} &{} \cdots &{} 0^t_{n_{N_{1}}} &{} 0^t_{n_{N_{1}+1}} &{} \cdots &{} 0^t_{n_{N}} \\ \vdots &{} &{} \vdots &{} \vdots &{} &{} \vdots &{} \vdots &{} &{} \vdots &{} \vdots &{} &{} \vdots \\ 0^t_{m_{1}} &{} \cdots &{} 0^t_{m_{N_{1}}} &{} 0^t_{m_{N_{1}+1}} &{} \cdots &{} -e^t_{m_{N}} &{} 0^t_{n_{1}} &{} \cdots &{} 0^t_{n_{N_{1}}} &{} 0^t_{n_{N_{1}+1}} &{} \cdots &{} 0^t_{n_{N}} \\ \\ 0^t_{m_{1}} &{} \cdots &{} 0^t_{m_{N_{1}}} &{} 0^t_{m_{N_{1}+1}} &{} \cdots &{} e^t_{m_{N}} &{} 0^t_{n_{1}} &{} \cdots &{} 0^t_{n_{N_{1}}} &{} 0^t_{n_{N_{1}+1}} &{} \cdots &{} 0^t_{n_{N}} \\ \\ 0^t_{m_{1}} &{} \cdots &{} 0^t_{m_{N_{1}}} &{} 0^t_{m_{N_{1}+1}} &{} \cdots &{} 0^t_{m_{N}} &{} e^t_{n_{1}} &{} \cdots &{} 0^t_{n_{N_{1}}} &{} 0^t_{n_{N_{1}+1}} &{} \cdots &{} 0^t_{n_{N}}\\ \\ 0^t_{m_{1}} &{} \cdots &{} 0^t_{m_{N_{1}}} &{} 0^t_{m_{N_{1}+1}} &{} \cdots &{} 0^t_{m_{N}} &{} -e^t_{n_{1}} &{} \cdots &{} 0^t_{n_{N_{1}}} &{} 0^t_{n_{N_{1}+1}} &{} \cdots &{} 0^t_{n_{N}} \\ \vdots &{} &{} \vdots &{} \vdots &{} &{} \vdots &{} \vdots &{} &{} \vdots &{} \vdots &{} &{} \vdots \\ 0^t_{m_{1}} &{} \cdots &{} 0^t_{m_{N_{1}}} &{} 0^t_{m_{N_{1}+1}} &{} \cdots &{} 0^t_{m_{N}} &{} 0^t_{n_{1}} &{} \cdots &{} e^t_{n_{N_{1}}} &{} 0^t_{n_{N_{1}+1}} &{} \cdots &{} 0^t_{n_{N}} \\ \\ 0^t_{m_{1}} &{} \cdots &{} 0^t_{m_{N_{1}}} &{} 0^t_{m_{N_{1}+1}} &{} \cdots &{} 0^t_{m_{N}} &{} 0^t_{n_{1}} &{} \cdots &{} -e^t_{n_{N_{1}}} &{} 0^t_{n_{N_{1}+1}} &{} \cdots &{} 0^t_{n_{N}} \\ \end{array} \right] \\ c= & {} \left[ \begin{array}{c} 0_{2N+m+n} \\ \\ -e^*_{2N_{2}} \\ \\ e^*_{2N_{1}} \\ \end{array} \right] \end{aligned}$$

where $e^*_{2k}$ is the 2k-dimensional column vector of alternating 1’s and $-1$’s, starting with 1.