Abstract
The basic aim of this paper is to provide a fundamental tool, the resolvent decomposition theorem, in the construction theory of denumerable Markov processes. We present a detailed analytic proof of this extremely useful tool and explain its clear probabilistic interpretation. We then apply this tool to investigate the basic problems of existence and uniqueness criteria for denumerable Markov processes with instantaneous states to which few results have been obtained even until now. Although the complete answers regarding these existence and uniqueness criteria will be given in a subsequent paper, we shall, in this paper, present part solutions of these very important problems that are closely linked with the subtle Williams S and N conditions.
Similar content being viewed by others
1 Introduction
The basic aim of this paper is to provide a fundamental tool, the resolvent decomposition theorem, in the construction theory of continuous time Markov chains (CTMC). This extremely useful theorem has a very clear probabilistic interpretation. It is just the Laplace transform version of first-entrance–last-exit decomposition law.
Let \(\{X_t; t\ge 0\}\) be a homogeneous continuous time Markov chain defined on the countable state space \({\mathbf {E}}=\{e_1, e_2, e_3, \ldots \}\) and let \(P(t) = \{p_{ij}(t); i, j\in {\mathbf {E}}, t \ge 0\}\) be its transition function. Then, in matrix form, this family of real matrix functions P(t) satisfies
and
where I denotes the identity matrix on \({\mathbf {E}}\times {\mathbf {E}}\) and \({\mathbf {1}}\) denotes the column vector on \({\mathbf {E}}\) whose components are all 1. In the following, \({\mathbf {1}}\) will always denote the column vector with an appropriate dimension (either finite or infinite) whose components are all \({\mathbf {1}}\). If the equality holds in (1.2) for all \(t \ge 0\), then the transition function P(t) is called honest.
One of the fundamental results in the theory of CTMC is that if \({\mathbf {P}}(t)\) satisfies (1.1)–(1.4), then the following limit exists
where the matrix \(Q = \{q_{ij}; i, j \in {\mathbf {E}}\}\) satisfies
and
In the following, we shall always denote \(q_i = -q_{ii}\ (i \in {\mathbf {E}})\). Note that in (1.7), \(q_{ii}\) may not be finite. If \(q_i = -q_{ii}< +\infty \), then the state \(i \in {\mathbf {E}}\) is called stable, while if \(q_i = +\infty \) then the state \(i \in {\mathbf {E}}\) is called instantaneous or unstable. If all states are stable, then the matrix Q is called stable, otherwise the Q is called unstable.
The limiting matrix Q given in (1.5) is usually called the intensity matrix of the transition function P(t). However, from now on we shall follow Anderson’s [1] convenient usage by using another term. In particular, the limiting matrix Q in (1.5) will be called a q-matrix and a transition function P(t) will be called a q-function. We shall also follow Anderson’s convention by calling a transition function P(t) as a Q-function if the q-matrix Q is specified. Furthermore, if a matrix Q on \({\mathbf {E}}\times {\mathbf {E}}\) satisfies (1.6)–(1.8), then this Q will be called a pre-q-matrix. Hence the above basic result could be simply referred to as “a q-matrix must be a pre-q-matrix.” However, the converse may not be always true. In fact, the basic question in the construction theory of CTMC is to investigate the reverse problems. More specifically, the basic questions in the construction theory of CTMC are as follows:
-
Question 1 (Existence) Under what conditions will a given pre-q-matrix become a q-matrix, i.e., under what conditions does there exist a Q-function for a given pre-q-matrix?
-
Question 2 (Uniqueness) If the given Q is a q-matrix, then under what conditions will there exist only one corresponding Q-function?
-
Question 3 (Construction) How do we construct all the Q-functions for a given q-matrix Q?
-
Question 4 (Property) How do we study properties of the Q-function P(t) and the corresponding CTMC \(\{X_t; t \ge 0\}\) in terms of the given q-matrix Q?
No doubt, answering the above questions is of great significance. This is particularly important due to the fact that in most of theoretic and practical problems only the pre-q-matrix Q is known.
In order to answer these important questions, in many cases it will be more convenient to study the so-called resolvents rather than transition functions. For this purpose, define
and denote \({\mathbf {R}}(\lambda ) = \{r_{ij}(\lambda ); i, j \in {\mathbf {E}}, \lambda > 0\}\), and then this \(R(\lambda )\) is usually called a resolvent function, or simply a resolvent.
The follow conclusion shows that it is important to introduce the concept of resolvent in the construction theory of CTMC.
Proposition 1.1
A family of real functions \({\mathbf {R}}(\lambda ) = \{r_{ij}(\lambda );\, i, j \in {\mathbf {E}}, \lambda > 0\}\) is a resolvent function of a transition function P(t) if and only if the following conditions hold
and P(t) is honest, i.e., \(P(t) {\mathbf {1}} = {\mathbf {1}}\) if and only if
holds. Moreover, (1.5) is true if and only if
Similar as transition functions, we shall use Q-resolvent and q-resolvent, respectively, to denote the resolvent function when the q-matrix is specified or not, respectively.
From the view point of construction theory of CTMC, the resolvent is more convenient than the transient function and therefore in the following we shall concentrate on discussing resolvent functions. In particular, the above-mentioned four basic questions may apply to Q-resolvent for a given q-matrix Q.
If the given q-matrix Q is stable, then the above questions were firstly systematically studied by J.L. Doob and W. Feller in the 1940s and then continuously investigated by many world-leading probabilitists including D.G. Kendall, G.E.H. Reuter, D. Willimas, J.F.C. Kingman, Samuel Karlin and K.L. Chung. In particular, Feller [14] showed that a stable pre-q-matrix must be a q-matrix and constructed a Q-resolvent (Q-function) for any given stable q-matrix Q, which has a minimal property and bears his name today. The uniqueness problem for stable chains was firstly considered by Doob [13] and then resolved by Reuter [27,28,29, 31] and Hou [16], respectively. Since then, the construction theory of stable CTMC has been flourished to a very high level. Until the publishing of Chung [12]’s foundation book, the theory of total stable q-processes was viewed, by and large, as completed, though many other important topics such as reversibility, ergodicity, quasi-stationary distributions, monotonicity, duality, coupling, large deviation, and spectral theory have emerged and flourished since then and lasted even until now. For more details, see Chen [11].
However, it may be surprised to know that few results have been obtained even until now if the pre-q-matrix is not stable, although great effort has been taken and could be traced back as early as in about 70 years ago. In fact, Kolmogorov [22] first posed an example with a single instantaneous state and asked for the existence of the corresponding process. This example was later firstly answered by Kendall and Reuter [20] and then deeply analyzed and generalized by Williams [35]. Later, Reuter [30] slightly generalized the Kolmogorov’s example.
On the other hand, the case when all states are instantaneous has also been receiving attention for a long time. For example, Blackwell [2] considered a very special example when all off-diagonal elements of the pre-q-matrix Q are all zero. See also Kendall [19]. Later, Williams [35] obtained an excellent result by giving a surprisingly simple criterion for the existence of totally instantaneous chains. Freedman’s book [15] seems the only book concentrating on discussing the unstable chains. See also Rogers and Williams [32, 33].
It may be hard to believe that the above results are essentially the only results obtained for unstable chains. In particular, it is really very hard to believe, though definitely true, that even until now the topic regarding mixing pre-q-matrix with more than one single instantaneous state remains totally blank. This reflects the fact that we are still lack of suitable tools and methods to tackle the unstable case.
The basic aim of this paper is just to fill this gap by providing an extremely useful tool, the so-called resolvent decomposition theorem. As we are aware, the idea of using such decomposition in the construction theory of continuous time Markov chains could be traced back at least to early seventies of last century when the paper of Lamb [23] was published in 1971. Also, a special case of this theorem was already provided in Chen and Renshaw [3, 4] and they have used this theorem to successfully tackle the unstable chain with a single instantaneous state. See, also, Chen and Renshaw [6, 8] and Chen et al. [10]. In some sense, the main aim of this paper is to generalize their result to the multi-state case and then use it to tackle the Markov chain with finitely many instantaneous states.
It should be noticed that to tackle unstable chains is not only of significance in the aspect of theory, but also important in the applications of continuous time Markov chains. In fact, Markov chains with instantaneous states have been revealed helpful in modeling many natural phenomena, even in social science. For example, in modern finance, it is usually the case that when financial crisis happens, some kind of “outside force” may strongly intervenes in order to “rescues” the economy from “depression.” Such “intervention” may be well modeled by the instantaneous chains since the action will be taken immediately when the “crisis” reaches some fixed “dangerous” level.
This paper concentrates on discussing a basic tool, the resolvent decomposition theorem, referred to as RDT in the following. The structure of this paper is as follows. In Sect. 2, the resolvent decomposition theorems will be stated and explained. But the detailed proof is postponed to the final section of the paper. In Sect. 3, we use this fundamental resolvent decomposition theorem tool to discuss the interesting conditions regarding CTMC with instantaneous states posted by David Williams. The more important application, i.e., to investigate the fundamental problems of existence and uniqueness of denumerable Markov process together with some important examples will be postponed to a sequent paper.
Before proceeding, it is necessary to mention two points in the terminologies used in this paper. The first one is that we shall use the synonyms “continuous time Markov chains” and “denumerable Markov process” to denote our model. Secondly, we shall use the term “Q-process” to refer either Q-function P(t), Q-resolvent \(R(\lambda )\) or the true process (i.e., the family of random variables) \(\{X_t(w); t\ge 0\}\). Such usage is, of course, commonly accepted and won’t cause any confusion. In case we need to distinguish them, we shall use the notations P(t), \(R(\lambda )\) or \(\{X_t; t \ge 0\}\) to emphasize the difference.
2 Resolvent Decomposition Theorems
Suppose P(t) is a transition function on the countable state space \({\mathbf {E}}\) with a q-matrix Q and resolvent \(R(\lambda )\), respectively. Let \({\mathbf {F}}\subset {\mathbf {E}}\) be a finite subset of \({\mathbf {E}}\) and denote \({\mathbf {G}}= {\mathbf {E}}{\backslash } {\mathbf {F}}\). In most of cases (but not always), we shall assume that the states in the set \({\mathbf {F}}\) are instantaneous and also that, without loss of any generality, we have re-arranged \({\mathbf {E}}= \{e_1, e_2, \ldots , e_n, e_{n+1}, \ldots \}\) such that \({\mathbf {F}}= \{e_1, e_2, \ldots , e_n\}\) contains the first n elements.
For later usage, let us assume that \(\Psi (\lambda )\ (\lambda > 0)\) is a (not necessarily honest) resolvent function defined on \({\mathbf {G}}\), i.e., \(\Psi (\lambda ) \ (\lambda > 0)\) satisfies (1.9) (1.10) and (1.11) with respect to \({\mathbf {G}}\). Further assume \(\eta (\lambda ) \ (\lambda > 0)\) and \(\xi (\lambda ) \ (\lambda > 0)\) are, for each \(\lambda > 0\), row vector and column vector on \({\mathbf {G}}\), respectively. Define
and
Then we have the following conclusion whose proof can be found in Sect. 7 of Chen and Renshaw [3].
Lemma 2.1
Suppose \(\eta (\lambda ) \in H_\Psi \) and \(\xi (\lambda ) \in K_\Psi \). Then
-
(i)
\(\eta (\lambda ) \equiv 0\)\((\forall \, \lambda > 0)\) if and only if \(\eta (\lambda _0) = 0\) for some \(\lambda _0 > 0\).
-
(ii)
\(\eta (\lambda )\) and \(\eta (\lambda ){\mathbf {1}}\) are decreasing functions of \(\lambda > 0\) and satisfy \(\eta (\lambda ) \downarrow 0\)\((\lambda \uparrow \infty )\) and \(\eta (\lambda ){\mathbf {1}} \downarrow 0\ (\lambda \uparrow \infty )\).
-
(iii)
\(\lambda \eta (\lambda ){\mathbf {1}}\) is an increasing function of \(\lambda > 0\) and thus \(\lim \nolimits _{\lambda \rightarrow \infty } \lambda \eta (\lambda )1\) exists but may be an infinite value.
-
(iv)
\(\eta = \eta (\mu ) + \mu \eta (\mu ) \Psi \ (\mu > 0)\) and \(\eta = \eta (\lambda ) + \lambda \eta \Psi (\lambda )\ (\lambda > 0)\) where the limits \(\eta = \lim \nolimits _{\lambda \rightarrow 0^+}\eta (\lambda )\) and \(\Psi = \lim \nolimits _{\lambda \rightarrow 0^+} \Psi (\lambda )\) both exist but may be an infinite value.
-
(v)
\(\xi (\lambda ) \equiv 0 \ (\forall \, \lambda > 0)\) if and only if \(\xi (\lambda _0) = 0\) for some \(\lambda _0 > 0\).
-
(vi)
\(\xi (\lambda ) \downarrow 0\) as \(\lambda \rightarrow + \infty \) and \(\xi (\lambda ) \uparrow \xi \) as \(\lambda \rightarrow 0^+\) and that \({\mathbf {0}} \le \xi \le {\mathbf {1}}\).
-
(vii)
\(\xi - \xi (\lambda ) = \lambda \Psi (\lambda )\xi \) and \(\xi - \xi (\mu ) = \mu \Psi \xi (\mu )\) where \(\xi = \lim \nolimits _{\lambda \rightarrow 0^+} \xi (\lambda )\) as in (vi) and \(\Psi = \lim \limits _{\lambda \rightarrow 0} \Psi (\lambda )\).
Now, in addition to \(\eta (\lambda ) \in H_\Psi \) and \(\xi (\lambda ) \in K_\Psi \) we further assume that \(\xi (\lambda ) \le {\mathbf {1}} - \lambda \Psi (\lambda ){\mathbf {1}}\) and then \(\eta (\lambda )\) and \(\xi (\lambda )\) enjoy more interesting properties as revealed in the following two lemmas whose proof can also be found in Sect. 7 of Chen and Renshaw [3].
Lemma 2.2
Suppose \(\eta (\lambda ) \in H_\Psi \) and \(\xi (\lambda ) \in K_\Psi \) and that \(\xi (\lambda ) \le {\mathbf {1}} - \lambda \Psi (\lambda ){\mathbf {1}}\). Then
-
(i)
$$\begin{aligned} (\lambda -\mu ) \left\langle \eta (\mu ), \xi (\lambda )\right\rangle = \lambda \left\langle \eta (\lambda ), \xi \right\rangle - \mu \left\langle \eta (\mu ), \xi \right\rangle . \end{aligned}$$(2.1)
-
(ii)
All the three functions \(\lambda \left\langle \eta (\lambda ), {\mathbf {1}}\right\rangle \), \(\lambda \left\langle \eta (\lambda ), \xi \right\rangle \) and \(\lambda \left\langle \eta (\lambda ), {\mathbf {1}}-\xi \right\rangle \) are increasing functions of \(\lambda > 0\) and thus the limits \(\lim \nolimits _{\lambda \rightarrow \infty } \lambda \left\langle \eta (\lambda ), {\mathbf {1}}\right\rangle \), \(\lim \nolimits _{\lambda \rightarrow \infty } \lambda \left\langle \eta (\lambda ), \xi \right\rangle \) and \(\lim \nolimits _{\lambda \rightarrow \infty } \lambda \left\langle \eta (\lambda ), {\mathbf {1}}-\xi \right\rangle \) exist but may be an infinite value.
-
(iii)
\(\lim \nolimits _{\lambda \rightarrow 0} \lambda \left\langle \eta (\lambda ), \xi \right\rangle = 0\),
where the notations such as \(\left\langle \eta (\mu ), \xi (\lambda )\right\rangle \), say, in the above expressions denote the inner product of the row vector \(\eta (\mu )\) and the column vector \(\xi (\lambda )\) on E.
Lemma 2.3
Suppose \(\eta (\lambda )\in H_\Psi \) and let \(\xi ^{(0)}(\lambda ) = {\mathbf {1}} - \lambda \Psi (\lambda ) {\mathbf {1}}\). Then \(\xi ^{(0)}(\lambda ) \in K_{\Psi }\) and thus enjoys all the properties expressed in Lemmas 2.1 and 2.2. Moreover, we have that
where \(\xi ^{(0)} = \lim \nolimits _{\lambda \rightarrow 0} \xi ^{(0)} (\lambda )\).
Moreover, suppose for each \(i\in {\mathbf {F}}\) we have a row vector \(\eta ^{(i)}(\lambda )\) on \({\mathbf {G}}\) such that \(\eta ^{(i)}(\lambda ) \in H_\Psi \ (i \in {\mathbf {F}})\). We pack these row vectors on \({\mathbf {G}}\) together as a finite-dimensional column vector whose components are row vectors on \({\mathbf {G}}\). We still denote it as \(\varvec{\eta }(\lambda )\). Hence \(\varvec{\eta }(\lambda ) = \{\eta ^{(i)}_j(\lambda ); i \in {\mathbf {F}}, j\in {\mathbf {G}}\}\) is, in fact, an \({\mathbf {F}}\times {\mathbf {G}}\) matrix. For convenience, we also express it as \(\varvec{\eta }(\lambda ) = \{\eta _{ij}(\lambda ); i \in {\mathbf {F}}, j\in {\mathbf {G}}\}\). Now, all the above information could be simply packed into a matrix equation
Similarly, if for each \(i\in {\mathbf {F}}\) we have a column vector \(\xi ^{(i)}(\lambda )\) on \({\mathbf {G}}\) such that \(\xi ^{(i)}(\lambda ) \in K_\Psi \), then we can get a finite-dimensional row vector, denoted by \(\varvec{\xi }(\lambda )\), whose components are column vectors on \({\mathbf {G}}\). That is that \(\varvec{\xi }(\lambda ) = \{\xi ^{(i)}_j (\lambda ); i \in {\mathbf {G}}, j\in {\mathbf {F}}\}\), or \(\varvec{\xi }(\lambda ) = \{\varvec{\xi }_{ij}(\lambda ); i \in {\mathbf {G}}, j\in {\mathbf {F}}\}\) is a \({\mathbf {G}}\times {\mathbf {F}}\) matrix satisfying the matrix equation
Now, the “inner product” of \(\varvec{\eta }(\lambda )\) and \(\varvec{\xi }(\lambda )\) is an \({\mathbf {F}}\times {\mathbf {F}}\) matrix. We shall denote it as \([\varvec{\eta }(\lambda ), \varvec{\xi }(\lambda )]\).
We can now restate Lemma 2.2 as the following useful and convenient conclusion.
Lemma 2.4
Suppose \(\varvec{\eta }(\lambda ) = \{\eta _{ij}(\lambda ); i \in {\mathbf {F}}, j \in {\mathbf {G}}\}\) satisfies (2.2) and \(\varvec{\xi }(\lambda ) = \{\xi _{ij}(\lambda ); i \in {\mathbf {G}}, j \in {\mathbf {F}}\}\) satisfies (2.3). Further assume that \(\varvec{\xi }(\lambda ){\mathbf {1}} \le {\mathbf {1}} - \lambda \Psi (\lambda ){\mathbf {1}}\). Then
-
(i)
Both \(\varvec{\eta }(\lambda )\) and \(\varvec{\xi }(\lambda )\) are increasing matrix function of \(\lambda > 0\) and thus the limit \(\varvec{\xi }= \lim \limits _{\lambda \rightarrow 0}\varvec{\xi }(\lambda )\) exists and \(\varvec{\xi }\) is an \({\mathbf {F}}\times {\mathbf {G}}\) bounded matrix (bounded by 1, say).
-
(ii)
$$\begin{aligned} (\lambda -\mu ) [\varvec{\eta }(\mu ), \varvec{\xi }(\lambda )] = \lambda [\varvec{\eta }(\lambda ), \varvec{\xi }] - \mu [\varvec{\eta }(\mu ), \varvec{\xi }]. \end{aligned}$$(2.4)
Proof
It follows directly from Lemma 2.2. \(\square \)
Note that (2.1) and(2.4) are very similar, but the meaning is different. Indeed, both sides of (2.1) are just scalars, while both sides of (2.4) are \({\mathbf {F}}\times {\mathbf {F}}\) matrices. For example, suppose \({\mathbf {F}}=\{a, b\}\) where \(a\ne b\), then the meaning of the simple form (2.4) is
We are now ready to state our main conclusions in this paper. Since we are mainly interested in the honest transition function, we shall assume that \({\mathbf {P}}(t)\) or, equivalently, \({\mathbf {R}}(\lambda )\) is honest. Then we have the following basic theorem.
Theorem 2.1
Suppose \(Q=\{q_{ij}; i, j \in {\mathbf {E}}\}\) is a q-matrix on the state space \({\mathbf {E}}\) (that is that there exists a transition function P(t) such that \(P'(0) = Q\)). Suppose further that \({\mathbf {R}}(\lambda ) = \{r_{ij}(\lambda ); i, j \in {\mathbf {E}}, \lambda > 0\}\) is an honest Q-resolvent. Let \({\mathbf {F}}\) be a finite subset of \({\mathbf {E}}\) and denote \({\mathbf {G}}= {\mathbf {E}}{\backslash } {\mathbf {F}}\). Then \({\mathbf {R}}(\lambda )\) can be uniquely decomposed as follows
where
-
(i)
\({\mathbf {A}}(\lambda )\) is the restriction of \({\mathbf {R}}(\lambda )\) on \({\mathbf {F}}\times {\mathbf {F}}\), i.e., \({\mathbf {A}}(\lambda ) = \{r_{ij} (\lambda ); i, j \in {\mathbf {F}}\}\) and
$$\begin{aligned} |{\mathbf {A}}(\lambda )|> 0 \quad (\forall \, \lambda > 0) \end{aligned}$$(2.6)and thus \({\mathbf {A}}(\lambda )\) is invertible for each \(\lambda > 0\).
-
(ii)
\(\varvec{\Psi }(\lambda ) = \{\psi _{ij}(\lambda ); i, j \in {\mathbf {G}}\}\) is a \({Q}_{{\mathbf {G}}{\mathbf {G}}}\)-resolvent (usually dishonest) where \({Q}_{{\mathbf {G}}{\mathbf {G}}} = (q_{ij}; i, j \in {\mathbf {G}})\) is the restriction of Q on \({\mathbf {G}}\times {\mathbf {G}}\).
-
(iii)
\(\varvec{\eta }(\lambda ) = \{\eta _{ij}(\lambda ); i \in {\mathbf {F}}, j \in {\mathbf {G}}\}\) satisfies
$$\begin{aligned} \varvec{\eta }(\lambda ) - \varvec{\eta }(\mu ) = (\mu - \lambda ) \varvec{\eta }(\lambda ) \varvec{\Psi }(\mu ), \quad (\forall \, \lambda , \mu > 0) \end{aligned}$$(2.7)and
$$\begin{aligned} 0 \le \eta _{i \cdot } (\lambda ) \in l_1 \quad (\forall \, i \in {\mathbf {F}}, \forall \, \lambda > 0) . \end{aligned}$$(2.8) -
(iv)
\(\varvec{\xi }(\lambda ) = \{\xi _{ij} (\lambda ); i \in {\mathbf {G}}, j \in {\mathbf {F}}\}\) satisfies
$$\begin{aligned} \varvec{\xi }(\lambda ) - \varvec{\xi }(\mu ) = (\mu - \lambda ) \varvec{\Psi }(\lambda ) \varvec{\xi }(\mu ), \quad (\forall \, \lambda , \mu > 0) \end{aligned}$$(2.9)and
$$\begin{aligned} 0 \le \xi _{ij} (\lambda ) < \infty \ \text{ with }\ \xi (\lambda ) {\mathbf {1}} = {\mathbf {1}} - \lambda \varvec{\Psi }(\lambda ){\mathbf {1}} \end{aligned}$$(2.10)and also
$$\begin{aligned} \xi _{ij}(\lambda )\ \uparrow \xi _{ij} \le 1 \ \text{ as }\ \lambda \rightarrow 0 \quad (\forall \, i \in {\mathbf {F}}, j \in {\mathbf {G}}) . \end{aligned}$$(Here \({\mathbf {1}}\) is a column vector whose elements are all 1 and the dimension of which depends. For example, the first \({\mathbf {1}}\) in (2.10) is a finite-dimensional vector of \({\mathbf {F}}\), while the other two in (2.10) are infinite-dimensional vector on \({\mathbf {G}}\))
-
(v)
$$\begin{aligned} \lim \limits _{\lambda \rightarrow \infty } \lambda \varvec{\eta }(\lambda )= & {} {Q}_{FG} \end{aligned}$$(2.11)$$\begin{aligned} \lim \limits _{\lambda \rightarrow \infty } \lambda \varvec{\xi } (\lambda )= & {} {Q}_{GF} \end{aligned}$$(2.12)
where \({Q}_{FG} = (q_{ij}, i \in {\mathbf {F}}, j \in {\mathbf {G}})\) and \({Q}_{GF} = (q_{ij}, i \in {\mathbf {G}}, j \in {\mathbf {F}})\) are the restriction of Q on \({\mathbf {F}}\times {\mathbf {G}}\) and \({\mathbf {G}}\times {\mathbf {F}}\), respectively.
-
(vi)
$$\begin{aligned} \lim \limits _{\lambda \rightarrow \infty } \lambda \sum _{j \in {\mathbf {G}}} \eta _{ij} (\lambda ) (1-\xi _{ji}) < + \infty \quad (\forall \, i \in {\mathbf {F}}) . \end{aligned}$$(2.13)
-
(vii)
There exists a constant matrix \({\mathbf {C}}= \{c_{ij}; i, j \in {\mathbf {F}}\}\) such that
$$\begin{aligned} {\mathbf {A}}^{-1} (\lambda ) = {\mathbf {C}}+ \lambda {\mathbf {I}}+ \lambda [\varvec{\eta }(\lambda ), \varvec{\xi }] \end{aligned}$$(2.14)and thus the right-hand side of (2.14) is invertible. Moreover,
$$\begin{aligned} c_{ii}\ge & {} 0 \quad c_{ij} \le 0 \quad (i \ne j)\ (i, j \in {\mathbf {F}}) \end{aligned}$$(2.15)$$\begin{aligned} -c_{ij}= & {} q_{ij} + \lim \limits _{\lambda \rightarrow \infty } \lambda \sum _{k \in {\mathbf {G}}} \eta _{ik} (\lambda ) \xi _{kj} \quad (\forall \, i, j \in {\mathbf {F}}, i \ne j) \end{aligned}$$(2.16)$$\begin{aligned} \sum _{j\in {\mathbf {F}}} c_{ij}= & {} \lambda \sum \limits _{k\in {\mathbf {G}}} \eta _{ik}(\lambda ) \left( 1-\sum \limits _{j\in {\mathbf {F}}} \xi _{kj}\right) , \quad (\forall \, i \in {\mathbf {F}}) \end{aligned}$$(2.17)where the right-hand side of the above expression (2.17) is independent of \(\lambda > 0\).
-
(viii)
If \(i \in {\mathbf {F}}\) is unstable, i.e., \(q_i = +\infty \), then
$$\begin{aligned} \lim \limits _{\lambda \rightarrow \infty } \lambda \sum _{k\in {\mathbf {G}}} \eta _{ik}(\lambda )\xi _{ki} = + \infty \quad (\forall \, i \in {\mathbf {F}}) \end{aligned}$$(2.18)or, equivalently
$$\begin{aligned} \lim \limits _{\lambda \rightarrow \infty } \lambda \sum _{k\in {\mathbf {G}}} \eta _{ik}(\lambda ) = + \infty \quad (\forall \, i \in {\mathbf {F}}) \end{aligned}$$(2.19)while if \(i \in {\mathbf {F}}\) is stable, i.e., \(q_i < \infty \), then
$$\begin{aligned} \lim \limits _{\lambda \rightarrow \infty } \lambda \sum _{k\in {\mathbf {G}}} \eta _{ik}(\lambda )\xi _{ki} < + \infty \end{aligned}$$(2.20)and
$$\begin{aligned} q_i = c_{ii} + \lim \limits _{\lambda \rightarrow \infty } \lambda \sum _{k\in {\mathbf {G}}} \eta _{ik}(\lambda )\xi _{ki} \end{aligned}$$(2.21)
here, \(\varvec{\xi } = \{\xi _{ij}; i \in {\mathbf {G}}, j \in {\mathbf {F}}\}\) in (2.14)–(2.18) and (2.20)–(2.21) is
(Recall the monotone property of \(\varvec{\xi }(\lambda )\) stated in (iv) above and thus the limit \(\varvec{\xi }\) in (2.22) does exist).
Remark 2.1
If the transition function \({\mathbf {P}}(t)\) or resolvent \({\mathbf {R}}(\lambda )\) may not be honest, then Theorem 2.1 still holds. The only difference is that the equality in (2.17) becomes the following (2.24) and the second equality in (2.10) become inequalities as the following (2.23), i.e.,
and
However, it is important to note that, different from the expression (2.17) which is independent of \(\lambda > 0\), the right-hand side of the above (2.24) does depend on \(\lambda > 0\) and thus a limit here is necessary.
This extremely useful theorem has a very clear probabilistic meaning. It is just the Laplace transform version of the first-entrance and the last-exit decomposition theorem. Indeed, \(\varvec{\xi }(\lambda )\) and \(\varvec{\eta }(\lambda )\) are simply the Laplace transforms of the first-entrance law to, and last-exit law from, the subset \({\mathbf {F}}\) of the corresponding Markov chain and \(\varvec{\Psi }(\lambda )\) is just the taboo-resolvent. See Chung [12] for the celebrated idea of taboo probability. This idea has been extensively developed by Syski [34], though this latter book has concentrated on the Feller minimal chains and thus the q-matrix concerned is stable. It should also be emphasized that the \({\mathbf {A}}(\lambda )\) in (2.5) and (2.6) is just the Laplace transform of the “transition function” of a quasi-Markov chain, a theory brilliantly developed by Kingman [21], in which the “Markov characterization problem” was tackled and solved.
Surely, the idea of decomposition theorem 2.1, though not in the completed and subtle form as stated here, has a long history which can be traced back at least to Neveu [24,25,26]. Based on Neveu and Chung’s works, Williams systematically studied and raised it to a considerable high level, see Rogers and Williams [33].
However, it seems that people have paid little attention to the converse of Theorem 2.1, which, in our opinion, is more important and has much more applications, particularly, in the study of unstable chains. That is that we have the following conclusion.
Theorem 2.2
Let \(Q = (q_{ij}; i, j \in {\mathbf {E}})\) be a pre-q-matrix on \({\mathbf {E}}\) and \({\mathbf {F}}\) is a finite subset of \({\mathbf {E}}\). Suppose there exist two functions \(\varvec{\eta }(\lambda )\) and \(\varvec{\xi }(\lambda )\) of \(\lambda > 0\) and a \(Q_{{\mathbf {G}}{\mathbf {G}}}\)-resolvent \(\Psi (\lambda )\) together with a constant matrix \({\mathbf {C}}=\{c_{ij}; i,j\in {\mathbf {F}}\}\) such that (2.7)–(2.13) (the equalities in (2.10) and (2.17) could be relaxed as the inequalities in (2.23) and (2.24)) and (2.15)–(2.21) are satisfied, where \({\mathbf {G}}= {\mathbf {E}}{\backslash } {\mathbf {F}}\) and \(Q_{{\mathbf {G}}{\mathbf {G}}}\) is the restriction of Q on \({\mathbf {G}}\times {\mathbf {G}}\), then Q is a q-matrix, that is that there exists a Q-function. Moreover, if the above \(\varvec{\Psi }(\lambda )\), \(\varvec{\xi }(\lambda )\) and \(\varvec{\eta }(\lambda )\) further satisfy
and for all stable \(i \in {\mathbf {F}}\)
then there exists an honest Q-function. The corresponding Q-resolvent (honest and dishonest) can be constructed by using the given \(\varvec{\Psi }(\lambda )\), \(\varvec{\eta }(\lambda )\), \(\varvec{\xi }(\lambda )\), the constant matrix \({\mathbf {C}}\) in terms of (2.5) and (2.14)–(2.22).
The important thing here in Theorem 2.2 is that it not only gives the existence conditions but also yields uniqueness criteria. It also provides a method to construct the q-resolvents, by which the property of the corresponding q-processes can be analyzed. This makes Theorems 2.1 and 2.2 very useful even for the stable q-processes. In particular, if the underling \(Q_{{\mathbf {G}}{\mathbf {G}}}\) resolvent \(\Psi (\lambda )\) is known, then the property of the Q-process may be easily derived. This idea has stimulated some new research works, see, for example, Chen and Renshaw [7] in which the underlying structure is an M / M / 1 (queue), and Chen and Renshaw [5, 9] when the underlying structure is a simple Markov branching process.
In case that \({\mathbf {F}}\) is a single element set of \({\mathbf {E}}\), Theorems 2.1 and 2.2 have been fully proved in Chen and Renshaw [3]. Their proof could be extended to our current general case by using the induction principle, see Hou et al. [18]. However, we prefer to provide a more simple and direct proof here, see the last section of this paper.
Note that in both Theorems 2.1 and 2.2, we do not impose any condition for the pre-q-matrix Q. That is that the pre-q-matrix Q is arbitrary. It may be stable, totally instantaneous or a mixed type (i.e., with both stable and instantaneous states).
In the following, we shall mainly be concerned with the case that all states in \({\mathbf {G}}\) are stable, though the states in the finite set \({\mathbf {F}}\) are arbitrary. Now, suppose that \(P(t) = \{p_{ij}(t); i, j\in {\mathbf {G}}, t \ge 0\}\) is a \(Q_{{\mathbf {G}}{\mathbf {G}}}\)-function where \(Q_{{\mathbf {G}}{\mathbf {G}}}\) is a stable q-matrix (that is that for all \(i \in {\mathbf {G}}\), \(q_i < \infty \)). Following Yang [36] (see also Hou and Guo [17] or Anderson [1]), we shall call P(t) is a B-type \(Q_{{\mathbf {G}}{\mathbf {G}}}\) transition function if
or an F-type \(Q_{{\mathbf {G}}{\mathbf {G}}}\) transition function if
Note that the resolvent forms of (2.27) and (2.28) are sometimes more convenient. That is the that a \(Q_{{\mathbf {G}}{\mathbf {G}}}\)-resolvent \(\Psi (\lambda )\) is called a B-type or an F-type \(Q_{{\mathbf {G}}{\mathbf {G}}}\)-resolvent if
or
holds, respectively.
It is well known that if \(Q_{{\mathbf {G}}{\mathbf {G}}}\) is stable, then there always exists a \(Q_{{\mathbf {G}}{\mathbf {G}}}\)-function (\(Q_{{\mathbf {G}}{\mathbf {G}}}\)-resolvent) that satisfies both (2.27) and (2.28) (or, equivalently, (2.29) and (2.30)) and also possesses the minimal property which is usually called the Feller minimal one. In case we need to emphasize this Feller minimal one, we shall usually use \({\mathbf {F}}(t)\) or \(\Phi (\lambda )\) to denote this Feller minimal transition function or resolvent, respectively.
If Q is not stable on \({\mathbf {E}}\) but \(Q_{{\mathbf {G}}{\mathbf {G}}}\) is stable, then although we are able to define neither the B-type nor the F-type transition function, we may define the so-called almost B-type (F-type) as follows.
Without loss of any generality, let us assume that all the states in the finite set \({\mathbf {F}}\) are instantaneous, i.e., \({\mathbf {F}}= \{i\in {\mathbf {E}}; q_i=+\infty \}\), and all the states in \({\mathbf {G}}= {\mathbf {E}}{\backslash } {\mathbf {F}}\) are stable. We then define a Q-function on \({\mathbf {E}}\) as almost B-type if
or almost F-type if
Furthermore, a Q-function is called an almost \(B\cap F\) type if both (2.31) and 2.32) hold. Again, it is more convenient if the resolvent form is used. That is that a Q-resolvent \(\varvec{R}(\lambda ) = \{r_{ij}(\lambda ); i, j \in {\mathbf {E}}\}\) is almost B-type if
or, almost F-type if
Furthermore, a Q-resolvent \(\varvec{R}(\lambda ) = \{r_{ij}(\lambda ); i, j \in {\mathbf {E}}\}\) is almost \(B\cap F\)-type if both (2.33) and (2.34) hold.
Now, we have the following simple but useful conclusion.
Theorem 2.3
Let \(Q= \{q_{ij}; i, j \in {\mathbf {E}}\}\) be a pre-q-matrix on \({\mathbf {E}}\) with a finite set \({\mathbf {F}}= \{i \in {\mathbf {E}}, q_i = +\infty \}\) and \({\mathbf {G}}= \{i \in {\mathbf {E}}, q_i < +\infty \}\) here \({\mathbf {G}}= {\mathbf {E}}{\backslash } {\mathbf {F}}\).
-
(i)
\({\mathbf {R}}(\lambda )\) is an almost B-type Q-resolvent if and only if the restricting \(Q_{{\mathbf {G}}{\mathbf {G}}}\)-resolvent \(\Psi (\lambda )\) is B-type.
-
(ii)
\({\mathbf {R}}(\lambda )\) is an almost F-type Q-resolvent if and only if the restricting \(Q_{{\mathbf {G}}{\mathbf {G}}}\)-resolvent \(\Psi (\lambda )\) is F-type.
-
(iii)
\({\mathbf {R}}(\lambda )\) is an almost \(B\cap F\)-type Q-resolvent if and only if the restricting \(Q_{{\mathbf {G}}{\mathbf {G}}}\)-resolvent \(\Psi (\lambda )\) is \(B\cap F\)-type.
Note that Theorems 2.1 and 2.2 hold true even if Q is stable. Although in this case, they do not provide further information regarding the existence of Q-function (which is trivial since we always have the Feller minimal one), Theorems 2.1 and 2.2 still give very useful information regarding the structure of the Q-functions (Q-resolvents). In particular, if the stable Q is regular, then such structure takes a very simple form which makes it very useful in realizing the properties of the Q-process. For this reason, we state it here for reference. Its proof is omitted since it is merely a direct corollary of Theorems 2.1 and 2.2.
Let Q be a regular (and thus conservative and stable) q-matrix defined on a countable state space \({\mathbf {E}}\). Let \({\mathbf {F}}\) be a finite subset of \({\mathbf {E}}\) and \({\mathbf {G}}={\mathbf {E}}{\setminus } {\mathbf {F}}\). We write Q as
where \(Q_{FF}\) is the restriction of Q to \({\mathbf {F}}\times {\mathbf {F}}\), \(Q_{{\mathbf {F}}{\mathbf {G}}}\) is the restriction of Q to \({\mathbf {F}}\times {\mathbf {G}}\), etc. Now, let \({\mathbf {R}}(\lambda )=\{r_{ij}(\lambda ),\, \lambda >0, i, j \in {\mathbf {E}}\}\) be the minimal Q-resolvent (the resolvent of the minimal Q-process), and observe that \({\mathbf {R}}(\lambda )\) can be written similarly as
Since Q is regular, the \({\mathbf {R}}(\lambda )\) is honest in that \(\lambda R(\lambda ) {\mathbf {1}}={\mathbf {1}}\), where \({\mathbf {1}}\) is the column vector in \({\mathbf {E}}\) whose elements are all 1 .
Theorem 2.4
Suppose Q is a regular q-matrix defined on \({\mathbf {E}}={\mathbf {F}}\cup {\mathbf {G}}\). Then, the (honest) minimal Q-resolvent \({\mathbf {R}}(\lambda )\) can be written as
where (i) \({\mathbf {A}}(\lambda )=R_{FF}(\lambda )\), (ii) \(\Psi (\lambda )\) is the minimal \(Q_{GG}\)-resolvent (which is dishonest except for the trivial case \(Q_{GF}\equiv 0\)), (iii) \(\varvec{\eta } (\lambda )=Q_{FG}\Psi (\lambda )\) and (iv) \(\varvec{\xi } (\lambda )=\Psi (\lambda )Q_{GF}\). Moreover, each element of \(\xi (\lambda )\) is a bounded (by 1, say) decreasing function of \(\lambda >0\) and thus the limit \(\varvec{\xi } (\lambda )\uparrow \varvec{\xi }\) exists as \(\lambda \downarrow 0\). The matrix function \(A(\lambda )\) is invertible for each \(\lambda >0\). Furthermore, after writing the row vector \(\varvec{\eta }(\lambda ) = (\eta ^{(a)}(\lambda ); a \in {\mathbf {F}})\) and the column vector \(\varvec{\xi } =(\xi ^{(b)}, b\in {\mathbf {F}})\), where we have, respectively, denoted the row vector \(\eta ^{(\alpha )}(\lambda )=\{\eta _{\alpha j}(\lambda ),\, j\in G\}\), \((\alpha \in {\mathbf {F}})\) and the column vector \(\xi ^{(\beta )}=\{\xi _{j\beta },\, j\in {\mathbf {G}}\}\), \((\beta \in {\mathbf {F}})\), then we have
where the constant matrix \(\varvec{C} = \{c_{ij}; i, j \in {\mathbf {F}})\) takes the form
3 Williams’ Conditions for q-Matrices
In his study of totally instantaneous Markov chains, D. Williams has obtained the following famous result regarding the totally instantaneous Markov chains.
Proposition 3.1
Suppose Q is a totally instantaneous q-matrix, i.e., for all \(i\in {\mathbf {E}}\), \(q_i = +\infty \). Then there exists a Q-function \({\mathbf {P}}(t)\) if and only if the following two conditions hold
-
(i)
$$\begin{aligned} \sum _{j\in {\mathbf {E}}{\backslash } \{a, b\}} (q_{aj} \wedge q_{bj}) < \infty \quad (\forall \, a\in {\mathbf {E}}, b \in {\mathbf {E}}, a\ne b) . \end{aligned}$$(3.1)
-
(ii)
There exists an infinite subset K of \({\mathbf {E}}\) such that
$$\begin{aligned} \sum _{j\in K{\backslash } \{i\}} q_{ij} < + \infty \quad (\forall \, i \in {\mathbf {E}}) . \end{aligned}$$(3.2)
Conditions (i) and (ii) in the above are usually referred to as (N) and (S) conditions, respectively.
Note that both (N) and (S) conditions also hold trivially true for stable Q-matrices. Naturally, we are interested in knowing whether they would be true for Q-matrices with finitely many instantaneous states. Of course, we only want to know whether they are necessary conditions since it is obvious that they cannot be sufficient. Note that Williams’ proof was probabilistic. We are thus also interested in answering the following questions. Can we give an analytic and simple proof of these conditions? Are these conditions always necessary for existence of Q-functions? Are there any relationships between the (N) condition and (S) condition?
The following conclusion shows that (N)-condition is in fact always necessary.
Theorem 3.1
If \(Q=\{q_{ij}; i, j \in {\mathbf {E}}\}\) is a q-matrix, i.e., if there exists a Q-function, then the N-condition (3.1) holds.
Proof
Suppose \(a\in {\mathbf {E}}\), \(b \in {\mathbf {E}}\) and \(a\ne b\). Without loss of generality, we may assume that \(a\in {\mathbf {F}}\) and \(b\in {\mathbf {F}}\) in Theorem 2.1 since otherwise we could enlarge \({\mathbf {F}}\). Then by (2.13) we have
where \(\eta ^{(a)} (\lambda ) = \{\eta _{aj} (\lambda ); j \in {\mathbf {G}}\}\) and \(\xi ^{(a)} = \{\xi _{ja}; j \in {\mathbf {G}}\}\). Similarly, we have
Hence
and
It follows from (3.3) and (3.4) that
But by (2.10) we have
Now, by letting \(\lambda \rightarrow 0\) in the above inequality we immediately get that
and thus by using (3.5) we obtain
Applying Fatou’s lemma in (3.6) and noting (2.11) and (2.12) together with the fact that \({\mathbf {F}}\) is a finite set then immediately yield the (N)-condition (3.1). \(\square \)
In fact, a much more strong form of N-condition can be obtained by simply applying Theorem 3.1.
Theorem 3.2
Suppose Q is a q-matrix on \({\mathbf {E}}\times {\mathbf {E}}\). Let \({\mathbf {E}}_1\) and \({\mathbf {E}}_2\) be two disjoint finite subsets of \({\mathbf {E}}\), i.e., \({\mathbf {E}}_1 \subset {\mathbf {E}}\), \({\mathbf {E}}_2 \subset {\mathbf {E}}\) and \({\mathbf {E}}_1 \cap {\mathbf {E}}_2 = \phi \). Then
and, in particular
Proof
We only need to prove (3.7) since (3.8) is an easy corollary of (3.7). In order to show (3.7), we first see that by Theorem 3.1 we have
Since \({\mathbf {E}}_1 \cap {\mathbf {E}}_2 = \phi \), we have \(\sum \limits _{b\in {\mathbf {E}}_2} \xi ^{(b)} \le 1 - \xi ^{(a)}\) and thus as a direct consequence of (3.9) we have that
Note also that it is easy to see that the limit in the left-hand side of (3.10) does exist. Now, since \({\mathbf {E}}_1\) is a finite subset of \({\mathbf {E}}\) we get by (3.10) that
and hence
On the other hand, by using
we obtain
Note that the vector \({\mathbf {1}}-\sum \nolimits _{b\in {\mathbf {E}}_2} \xi ^{(b)}\) in (3.12) is actually independent of \(b\in {\mathbf {E}}_2\) and thus by using (3.12) and the finiteness of \({\mathbf {E}}_2\) we obtain
and hence
Combining (3.13) and (3.11) immediately yields
Applying Fatou’s lemma in (3.14) immediately yields the required (3.7). \(\square \)
As a direct consequence of Theorem 3.2, we have the following corollary.
Corollary 3.1
Suppose Q is a q-matrix on \({\mathbf {E}}\times {\mathbf {E}}\). Let \({\mathbf {E}}_1\) be a finite subset of E. Then for any \(a\in {\mathbf {E}}{\backslash } {\mathbf {E}}_1\), we have
Note that the Williams’ (N)-condition is a direct consequence of (3.15). We see that by the above conclusions we have had much more deep understanding of the (N)-condition. We now turn to consider the (S)-condition.
If the q-matrix Q is stable, then (S)-condition holds trivially. On the other hand, if the q-matrix Q has only one instantaneous state, then the (S)-condition may not hold. A counter example is the famous Kolmogorov q-matrix, see Reuter [30] or Chen and Renshaw [4]. Hence in the following we shall assume that the given pre-q-matrix Q has at least two instantaneous states.
We now first show that for any given q-matrix Q that has at least two instantaneous states, then a weaker version of the (S)-condition does hold.
Theorem 3.3
Suppose that Q is a q-matrix on \({\mathbf {E}}\). Further suppose that Q possesses at least two instantaneous states. Then for any \(i \in {\mathbf {E}}\), there exists an infinite subset \(K_i\subset {\mathbf {E}}\) such that
Compared with the original Williams’ condition, we see that the difference is that in our Theorem 3.3, the infinite subset \(K_i\) may depend on i. We shall prove Theorem 3.3 by using two lemmas which deal with the two different cases.
Lemma 3.1
Suppose \(Q=\{q_{ij}; i, j \in {\mathbf {E}}\}\) is a q-matrix on \({\mathbf {E}}\times {\mathbf {E}}\). Further assume that \(a\in {\mathbf {E}}\), \(b\in {\mathbf {E}}\), \(a\ne b\) and that \(\sum \limits _{j\ne a} q_{aj} = q_a = +\infty \) and \(\sum \limits _{j\ne b} q_{bj} = q_b = +\infty \). Then there exist two infinite subsets \(K_a\) and \(K_b\) of \({\mathbf {E}}\) such that
Proof
By Theorem 3.1 we have
Now, define
Obviously, \(K_a \cup K_b \cup K_c = {\mathbf {E}}{\backslash }\{a, b\}\) and they are disjoint. Hence (3.18) may be rewritten as
We now claim that \(K_a\) is an infinite subset. Indeed if it is a finite subset of \({\mathbf {E}}\), then of course we have \(\sum \nolimits _{j\in K_a} q_{bj} < + \infty \). This together with (3.19) yields \(\sum \nolimits _{j\in {\mathbf {E}}{\backslash }\{a, b\}} q_{bj} < + \infty \), a contradiction to our assumption \(\sum \nolimits _{j\in {\mathbf {E}}_b{\backslash }\{a, b\}} q_{bj} = + \infty \). Similarly, we can prove \(K_b\) is also an infinite subset of \({\mathbf {E}}\). \(\square \)
Lemma 3.2
Suppose \(Q = \{q_{ij}; i, j \in {\mathbf {E}}\}\) is a q-matrix on \({\mathbf {E}}\times {\mathbf {E}}\) and \(a \in {\mathbf {E}}\), \(b \in {\mathbf {E}}\). Further assume that
and
Then, there exists an infinite subset \(K_a \subset {\mathbf {E}}\) such that
Proof
First note that it is easily seen that (3.22) holds true if and only if \(\liminf \nolimits _{j\rightarrow +\infty } q_{aj} = 0\) and thus if the conclusion does not hold, then we must have
We now derive a contradiction. Indeed, if (3.23) is true, then the vector \(\alpha : = \{q_{aj}; j \in {\mathbf {E}}{\backslash }\{0\}\}\) has at most finitely many \({\mathbf {0}}\)’s. Without loss of any generality, we may assume that \(q_{aj} > 0\) for all \(j \in {\mathbf {E}}{\backslash }\{a\}\). Now, by Theorem 3.1, we have
and then by applying Fatou’s lemma and noting \(\lim \nolimits _{\lambda \rightarrow +\infty } \lambda \eta ^{(a)}(\lambda ) = \alpha \), we obtain
It follows that
Using (3.23), we see that (3.24) implies that \(\lim \nolimits _{j\rightarrow +\infty } \xi ^{(b)}_j = 0\). Hence there exist a finite subset \({\mathbf {E}}_1\) (which is independent of \(\lambda > 0\)) and a constant \(\delta > 0\) such that
Since \(\lambda \left\langle \eta ^{(b)}(\lambda ), \,1-\xi ^{(b)}\right\rangle \) is an increasing function of \(\lambda \) when \(\lambda \) tends to \(+\infty \), and that \(\lim \limits _{\lambda \rightarrow +\infty } \lambda \left\langle \eta ^{(b)}(\lambda ),\, 1-\xi ^{(b)}\right\rangle < + \infty \), it follows that there exists a constant \(c > 0\) such that
It follows, by also using (3.25), that
or
Hence
Noting that \({\mathbf {E}}_1\) is a finite subset of \({\mathbf {E}}\) and also \({\mathbf {E}}_1\) does not depend on \(\lambda > 0\) and thus (3.26) implies that
which contradicts with \(q_a = +\infty \), see (2.18). \(\square \)
By Lemmas 3.1 and 3.2, we see that Theorem 3.3 has been proven. In other words, if Q is a q-matrix with at least two instantaneous states, then it must satisfy the weaker form of (S)-condition as stated in Theorem 3.3. We are now interested in knowing whether the original Williams’ condition is true or not. Interestingly, this question can be answered completely. In fact, if the q-matrix Q has infinitely many instantaneous states, then the Williams’ condition always hold. However, if the q-matrix Q has only finitely many instantaneous states, then the (S)-condition of Williams may not hold true. In a sequent paper we shall give an example to show that for any positive integer n we could construct a q-matrix such that the q-matrix Q has exactly n instantaneous states that does not satisfy the original (S)-condition. In spite of this, we could, interestingly, prove that (S)-condition is “nearly” true. The exact meaning is the following conclusion.
Theorem 3.4
Suppose \(Q=\{q_{ij}; i, j \in {\mathbf {E}}\}\) is a q-matrix on \({\mathbf {E}}\times {\mathbf {E}}\) with \(n \ge 2\) instantaneous states. Then for any finite subset \({\mathbf {E}}_1\) of \({\mathbf {E}}\), there exists an infinite subset K of \({\mathbf {E}}\) such that
provided that \(|{\mathbf {E}}_1| < n\) where \(|{\mathbf {E}}_1|\) denote the cardinal number of \({\mathbf {E}}_1\).
Proof
Without loss of generality, we could assume that
since otherwise we may discuss a more smaller \({\mathbf {E}}_1\). Now, by Corollary 3.1 we have
Now, if \(\sum \nolimits _{j\ne a} q_{aj} = + \infty \), then the conclusion follows from Lemma 3.1, while if \(\sum \nolimits _{j\ne a} q_{aj} < + \infty \), then the conclusion follows from Lemma 3.2. \(\square \)
Finally, we consider the case that the given q-matrix Q has infinitely many of instantaneous states. For this case, the (S)-condition (3.2) will always hold true as the following conclusion shows. This confirms the Williams’ theorem regarding totally instantaneous states.
Theorem 3.5
Suppose \(Q=\{q_{ij}; i, j \in {\mathbf {E}}\}\) is a q-matrix with infinitely many instantaneous states. Then the (S)-condition holds, i.e., there exists an infinite subset K of \({\mathbf {E}}\) such that
Proof
First note that the (S)-condition is equivalent to
Hence to complete the proof of this theorem we only need to prove that if there exists a positive integer \(n \ge 2\) such that
then Q can only have finitely many of instantaneous states.
To this end, we just need to show that if \(b > n\), then b must be a stable state. Indeed, by Theorem 3.1 and \(b > n\) we have
Therefore
Applying Fatou’s lemma then yields
and thus
By (3.27) we then obtain \(\lim \limits _{j\rightarrow +\infty } \xi ^{(b)}_j = 0\). Now, similarly as we did in Lemma 3.2 we may easily prove that the state b is stable. \(\square \)
4 Proofs of the Resolvent Decomposition Theorems
Our main aim of this section is to prove the resolvent decomposition theorems stated in Sect. 2.
Proof of Theorem 2.1
For notational convenience, we rearrange the states in \({\mathbf {E}}\) such that the states in the finite set \({\mathbf {F}}\) are in the beginning. Considering \({\mathbf {F}}\cup {\mathbf {G}}= {\mathbf {E}}\) and \({\mathbf {F}}\cap {\mathbf {G}}= \phi \), we could write the transition function P(t) in matrix block form as
where the meaning of the four block matrix is self-explained. Now, define Chung’s taboo probability
for \(i, j\in {\mathbf {G}}\) and let \({}_{\mathbf {F}}P(t)=\{{}_{\mathbf {F}}p_{ij}(t); i, j\in {\mathbf {G}}\}\), Then \({\mathbf {P}}(t)\) can be written as
where \(P_{{\mathbf {G}}{\mathbf {G}}}(t) - {}_{\mathbf {F}}P(t) \ge 0\) (pointwise) for obvious reasons.
By the general last-exit (from set \({\mathbf {F}}\)) decomposition, we have
where the term \(p_{ik}(s) g_{kj}(t-s)\mathrm{d}s\) represents, intuitively, the probability that \(X(t)=j\) and the last visit to k in F until time t occurred between s and \(s+\mathrm{d}s\) under the condition that \(X(0)=i\), see Chung [12]. Now, for notational convenience, we simply denoted the Laplace transforms matrices as
then
where
Similarly, by the general first-entrance (into set \({\mathbf {F}}\)) decomposition.
where the term \(f_{ik}(s) p_{kj}(t-s) \mathrm{d}s\) represents, intuitively, the probability that the Markov chain \(\{X_t\}\) first hits k between times s and \(s+\mathrm{d}s\) given that \(X(0)=i\). Hence paralleling to (4.4) we can get that
where \(\varvec{\xi } (\lambda ) = \{\xi _{ij}(\lambda ), i\in {\mathbf {G}}, j\in {\mathbf {F}}\}\) with the obvious meaning of \(\xi _{ij}(\lambda ) = \displaystyle {\int ^\infty _0} e^{-\lambda t} f_{ik} (t)\mathrm{d}t\).
Now, it is pretty clear that we can get that
If we further denote \({\mathbf {R}}(\lambda ) = \displaystyle {\int ^\infty _0} e^{-\lambda t} P(t) \mathrm{d}t\) and \(\varvec{\Psi }(\lambda )=\displaystyle {\int ^\infty _0} e^{-\lambda t} {}_{\mathbf {F}}P(t) \mathrm{d}t\), then the Laplace transform of (4.2) takes a simply form as
together with the following pointwise nonnegative properties for all \(\lambda > 0\)
Although it is fairly easy to prove now that the \(\varvec{\Psi }(\lambda )\) in (4.7) itself is a \(Q_{{\mathbf {G}}{\mathbf {G}}}\) resolvent where \(Q_{{\mathbf {G}}{\mathbf {G}}} = \{q_{ij}; i, j \in {\mathbf {G}}\}\), we shall not do so here for the reasons of getting more results and transparency. In fact, we shall prove our theorem in a few steps. Since we have represented the Q-resolvent \({\mathbf {R}}(\lambda )\) as in (4.7) together with the nonnegative property of the corresponding quantities, then firstly we claim that the \({\mathbf {R}}(\lambda )\) satisfies the resolvent equation (1.10) if and only if the following equations hold, simultaneously, for all \(\lambda > 0\) and \(\mu > 0\):
Indeed, substituting (4.7) into (1.10) immediately yields that (1.10) holds true if and only if the following equations hold true, simultaneously, for all \(\lambda > 0\) and \(\mu > 0\):
Substituting (4.10a) into (4.10b), (4.10c) and (4.10d) shows that the four equations (4.10a)–(4.10d) hold true if and only if the following four equations hold true, simultaneously, for all \(\lambda > 0\) and \(\mu > 0\),
Recall \({\mathbf {A}}(\lambda ) = \displaystyle {\int ^\infty _0 e^{-\lambda t} P_{{\mathbf {F}}{\mathbf {F}}}(t)\mathrm{d}t}\) and noting that \({\mathbf {A}}(\lambda )\) is a finite-dimensional matrix function and thus \({\mathbf {A}}(\lambda )\) is the resolvent functions of a quasi-Markov chain discussed in Kingman’s book [21] and thus by the same reference we know that \(|A(\lambda )| > 0\) (for all \(\lambda > 0\)) (see again Kingman [21]) where \(|A(\lambda )|\) denotes the determinant of \(A(\lambda )\). Hence for all \(\lambda > 0\) we know \({\mathbf {A}}^{-1}(\lambda )\) does exist. Now left-multiplicating \({\mathbf {A}}^{-1}(\lambda )\) on (4.11b) and right-multiplicating \({\mathbf {A}}^{-1}(\mu )\) on (4.11c) and then substituting the two resulting equations into (4.11d) immediately yield that the four equations (4.11a)–(4.11d) hold true for all \(\lambda > 0\) and \(\mu > 0\) if and only if the four equations in (4.9a)–(4.9d) hold true, simultaneously, for all \(\lambda > 0\) and \(\mu > 0\).
Secondly, we claim that the \({\mathbf {R}}(\lambda )\) represented in (4.7) satisfies (1.12) (recall for convenience and importance we only consider honest Q-resolvent in Theorem 2.1) if and only if the following two equations hold true, simultaneously, for all \(\lambda > 0\)
This is easy. Indeed, substituting (4.7) into (1.12) immediately yields that (1.12) holds true if and only if
and
hold true, simultaneously, for all \(\lambda > 0\). Now, it is very easy to see that both (4.13) and (4.14) hold true if and only if both (4.12a) and (4.12b) hold true, simultaneously, for all \(\lambda > 0\).
Thirdly, we claim that \({\mathbf {A}}(\lambda )\) satisfies (4.9a), (i.e., (4.11a)) if and only if there exists an \({\mathbf {F}}\times {\mathbf {F}}\) constant matrix \({\mathbf {C}}= \{c_{ij}; i, j \in {\mathbf {F}}\}\) such that the matrix \({\mathbf {C}}+ \lambda {\mathbf {I}}+ \lambda [\varvec{\eta }(\lambda ), \varvec{\xi }]\) is invertible and that
where \({\mathbf {I}}\) in (4.15) is the \({\mathbf {F}}\times {\mathbf {F}}\) identity matrix.
Indeed, by first left-multiplication of \({\mathbf {A}}^{-1}(\lambda )\) and then right-multiplication of \({\mathbf {A}}^{-1}(\mu )\) on (4.9a) and noting \(\varvec{\eta }(\lambda )\) and \(\varvec{\xi }(\lambda )\) are nonnegative (see (4.8b) and (4.8c)) and the fact that \({\mathbf {A}}^{-1}(\lambda )\) is a finite-dimensional matrix, we may apply associated law on (4.9a), and then we immediately get that
Applying (2.4) in the above then yields the fact that
which shows that \({\mathbf {A}}^{-1}(\lambda ) - \lambda {\mathbf {I}}- \lambda [\varvec{\eta }(\lambda ), \varvec{\xi }]\) is a constant matrix and thus could be expressed as \({\mathbf {C}}= \{c_{ij}; i, j \in {\mathbf {F}}\}\). Hence (4.15) follows. Conversely, if (4.15) holds, it is easy to see that (4.9a) holds true.
Fourthly, we show that \({\mathbf {R}}(\lambda )\) in (4.7) satisfies (1.11) if and only if the following four relations hold true.
where in the above four expressions, \({\mathbf {I}}\) and \({\mathbf {0}}\) are the identity matrix and 0 matrix, respectively. However, this verification is trivial and thus omitted.
Now, by (4.8a), (4.12b), (4.9d) and (4.17d) we see that \(\varvec{\Psi }(\lambda )\) is a resolvent function. We now further claim that it is a \(Q_{{\mathbf {G}}{\mathbf {G}}}\)-resolvent where \(Q_{{\mathbf {G}}{\mathbf {G}}}\) is the restriction of Q on \({\mathbf {G}}\times {\mathbf {G}}\). To this end, let us express the given q-matrix as the block form of
where the notations are self-explained. Now, it is easy to see that (1.13) holds true if and only if the following relations hold true simultaneously
Using (4.17b) and (4.18c) together with the finiteness of the set \({\mathbf {F}}\), we get that
and then
immediately follows from (4.18d) and (4.19). Therefore, Parts (i) and (ii) in Theorem 2.1 have been proven. Moreover, (4.18b) and (4.17a) immediately yield (2.11) and, similarly, (4.18c) and (4.17a) yield (2.12) and hence Parts (iii), (iv), and (v) in Theorem 2.1 have been also proven. In order to prove the other parts of Theorem 2.1, we go back to the proven (4.16) which shows that
which is just (2.14).
On the other hand, by using (4.18a) and (4.17a), it is fairly easy to show that
and thus by using (4.21) we obtain
or in component form
It follows from (4.23) that if \(i, j \in {\mathbf {F}}\) and \(i \ne j\), then \(c_{ij} \le 0\) and (2.16) holds.
Also, by left-multiplicating \(A^{-1}(\lambda )\) on (4.12a) yields
which, after substituting (4.15) into it, simplifies as
or, in component form,
Note that the second term in the right-hand side of the above (4.24) is independent of \(\lambda > 0\) which is guaranteed by Lemma 2.3. Note also that the proven fact that if \(i \ne j\), then \(c_{ij} \le 0\) and that \(\sum \nolimits _{j\in {\mathbf {F}}} \xi _{kj} \le 1\) for all \(k \in {\mathbf {F}}\) we see that by (4.24), \(c_{ii} \ge 0\ (\forall \, i \in {\mathbf {F}})\) and also, (4.24) just reads as (2.17). Hence the part (vii) of Theorem 2.1 is also proven.
Note also that the important fact (2.13) in Part (vi) follows directly from the just proven results (2.16) and (2.17) together with the fact that \({\mathbf {F}}\) is a finite set. Hence Part (vi) of Theorem 2.1 is proven as well.
Now, returning to (4.23) we see that for all \(i \in {\mathbf {F}}\) we have
Now, if \(i \in {\mathbf {F}}\) is unstable, i.e., if \(q_i = +\infty \), then by (4.25) and considering \(c_{ii}\) is a finite number we must have
which is just (2.18). The equivalency between (2.18) and (2.19) is obvious. Indeed, if (2.18) (i.e., (4.26)) is true, then considering \(0 \le \xi _{ki} \le 1 \ (\forall \, i \in {\mathbf {F}}, k \in {\mathbf {G}})\) we must have \(\lim \nolimits _{\lambda \rightarrow +\infty } \lambda \sum \nolimits _{k\in {\mathbf {G}}} \eta _{ik}(\lambda ) = +\infty \ (i \in {\mathbf {F}})\) which is (2.19). Conversely, if (2.19) is true, i.e., if \(\lim \nolimits _{\lambda \rightarrow +\infty } \lambda \sum \nolimits _{k\in {\mathbf {G}}} \eta _{ik}(\lambda ) = +\infty \) we must have (4.26) (i.e., (2.18)) since we always have \(\lim \nolimits _{\lambda \rightarrow +\infty } \lambda \sum \nolimits _{k\in {\mathbf {G}}} \eta _{ik}(\lambda ) \bigg (1-\sum \nolimits _{j\in {\mathbf {F}}} \xi _{ki}\bigg ) < +\infty \), here the last statement comes from the proven (2.17).
On the other hand, if \(q_i < + \infty \), then (4.25) immediately yields (2.20) and (2.21). Now, the last part of Theorem 2.1, i.e., part (viii) is also proven.
Finally, we prove the decomposition form (2.5) is unique. But this is easy. Indeed, if, in addition to (2.5), we have another decomposition form as
then we first must have \({\mathbf {A}}(\lambda ) = \varvec{{\widetilde{A}}}(\lambda )\) since both \({\mathbf {A}}(\lambda )\) and \(\varvec{{\widetilde{A}}}(\lambda )\) are the restriction of \({\mathbf {R}}(\lambda )\) on \({\mathbf {F}}\times {\mathbf {F}}\). Next, we then must have
Since \(A^{-1}(\lambda )\) exists and thus by (4.27) we get \(\varvec{\eta }(\lambda ) = \varvec{{\widetilde{\eta }}}(\lambda )\). Similarly by (4.28) we obtain \(\varvec{\xi }(\lambda ) = \varvec{{\widetilde{\xi }}}(\lambda )\). Then by (4.29) we further have \(\Psi (\lambda ) = {\widetilde{\Psi }}(\lambda )\). The proof of Theorem 2.1 is completed. \(\square \)
If one carefully checks the proof of Theorem 2.1, one would find that Theorem 2.2 has been essentially proved. Indeed, in the process of proving Theorem 2.1, we have constantly emphasized that each crucial step is both necessary and sufficient. In other words, based on the proof of Theorem 2.1 we could easily prove Theorem 2.2 by simply adding the detailed checking. But we shall omit such detailed checking here.
Proof of Theorem 2.3
We only need to prove (i), since the proof of (ii) is similar, while (iii) is a direct consequence of both (i) and (ii). For convenience, we use the block matrix form. That is that we express \({\mathbf {R}}(\lambda )\) as \(\left( \begin{array}{ll} R_{{\mathbf {F}}{\mathbf {F}}}(\lambda ), &{}\quad R_{{\mathbf {F}}{\mathbf {G}}}(\lambda )\\ R_{{\mathbf {G}}{\mathbf {F}}}(\lambda ), &{}\quad R_{{\mathbf {G}}{\mathbf {G}}}(\lambda ) \end{array} \right) \) where the meaning of the four block matrices is obvious. Similarly, we express Q as \(\left( \begin{array}{ll} Q_{{\mathbf {F}}{\mathbf {F}}}, &{}\quad Q_{{\mathbf {F}}{\mathbf {G}}}\\ Q_{{\mathbf {G}}{\mathbf {F}}}, &{}\quad Q_{{\mathbf {G}}{\mathbf {G}}} \end{array} \right) \). Then (2.33) can be represented as the following two relationships.
and
hold simultaneously. Now, substituting (2.5) into (4.30) and (4.31) and after some algebra shows that both of them hold if and only if both
and
hold. Now, substituting (4.32) into (4.33) immediately yields
which shows that \(\varvec{\Psi }(\lambda )\) is a B-type \(Q_{{\mathbf {G}}{\mathbf {G}}}\)-resolvent. To prove the converse, we need to prove that (4.34) implies both (4.30) and (4.31). To this end, we turn to Theorem 2.1 and by (2.5) we know that
By using (4.34), we see that the right-most expression in (4.35) can be simplifies as
Hence (4.35) is just
which shows that \((\lambda \, I - Q_{{\mathbf {G}}{\mathbf {G}}})\varvec{\xi }(\lambda )\) is a constant matrix, i.e., independent of \(\lambda > 0\). In order to obtain this constant matrix, we just consider \(\lim \nolimits _{\lambda \rightarrow \infty }(\lambda \, I - Q_{{\mathbf {G}}{\mathbf {G}}})\varvec{\xi }(\lambda )\). Note that \(\varvec{\xi }(\lambda ) \downarrow 0 \) as \(\lambda \uparrow \infty \) and that \(\lim \nolimits _{\lambda \rightarrow \infty } \lambda \, \varvec{\xi }(\lambda ) = Q_{{\mathbf {G}}{\mathbf {F}}}\), see (2.8). Hence this constant matrix is just \(Q_{{\mathbf {G}}{\mathbf {F}}}\) and hence
By right-multiplicating \({\mathbf {A}}(\lambda )\) on the above (4.36) immediately yields (4.32) and then (4.33) follows from the just proven (4.32) and (4.34). Hence both (4.32) and (4.33) hold true and thus both (4.30) and (4.31) hold, which ends the proof. \(\square \)
References
Anderson, W.: Continuous-Time Markov Chains: An Applications-Oriented Approach. Springer-Verlag, New York (1991)
Blackwell, D.: Another countable Markov process with only instantaneous states. Ann. Math. Stat. 29, 313–316 (1958)
Chen, A.Y., Renshaw, E.: Markov branching processes with instantaneous immigration. Probab. Theor. Rel. Fields 87, 209–240 (1990)
Chen, A.Y., Renshaw, E.: Existence and uniqueness criteria for conservative uni-instantaneous denumerable Markov processes. Probab. Theor. Rel. Fields 94, 427–456 (1993a)
Chen, A.Y., Renshaw, E.: Recurrence of Markov branching processes with immigration. Stoch. Process. Appl. 45, 231–242 (1993b)
Chen, A.Y., Renshaw, E.: Markov branching processes regulated by emigration and large immigration. Stoch. Process. Appl. 57, 339–359 (1995)
Chen, A.Y., Renshaw, E.: The M/M/1 queue with mass exodus and mass arrivals when empty. J. Appl. Probab. 34, 192–207 (1997)
Chen, A.Y., Renshaw, E.: Existence, recurrence and equilibrium properties of Markov branching processes with instantaneous immigration. Stoch. Process. Appl. 88, 177–193 (2000)
Chen, A.Y., Renshaw, E.: Markovian bulk-arriving queues with state-dependent control at idle time. Adv. Appl. Probab. 36, 499–524 (2004)
Chen, A.Y., Zhang, H.J., Liu, K., Rennolls, K.: Birth-death processes with disaster and instantaneous resurrection. Adv. Appl. Probab. 36, 267–292 (2004)
Chen, M.F.: From Markov Chains to Non-equilibrium Particle Systems. World Scientific, Singapore (1992)
Chung, K.L.: Markov Chains with Stationary Transition Probabilities, 2nd edn. Springer, New York (1967)
Doob, J.L.: Markov chains—denumerable case. Trans. Am. Math. Soc. 58, 455–473 (1945)
Feller, W.: On the integro-differential equations of purely discontinuous Markov processes. Trans. Am. Math. Soc. 48, 488–515 (1940)
Freedman, D.: Approximating Countable Markov Chains. Springer, Berlin (1983)
Hou, Z.T.: The uniqueness criterion for \(Q\)-processes (in Chinese). Sinica 2, 115–130 (1974)
Hou, Z.T., Guo, Q.F.: Homogeneous Denumerable Markov Processes. Springer-Verlag, Berlin (1988)
Hou, Z.T., et al.: The \(Q\)-Matrix Problem. Human Sciences Process, Changsha (1994)
Kendall, D.G.: A totally unstable denumerable Markov process. Q. J. Math. Oxford 9, 149–160 (1958)
Kendall, D.G., Reuter, G.E.H.: Some pathological Markov processes with a denumerable infinity of states and the associated semigroups of operators on \({\cal{L}}\). In: Proceedings of the International Congress of Mathematicians Amsterdam, vol. III, pp. 377–415, Amsterdam, North-Holland (1954)
Kingman, J.F.C.: Regenerative Phenomena. Wiley, London (1972)
Kolmogorov, A.N.: On the differentiability of the transition probabilities in stationary Markov processes with a denumerable number of states (in Russian). Moskov. Gos. Univ. Ucenye Zapiski. Mat. 148(4), 53–59 (1951)
Lamb, C.W.: Decomposition and construction of Markov chains. Z. Wahrscheinlichkeitstheorie view. Geb. 19, 213–224 (1971)
Neveu, J.: Lattice methods and submarkovian processes. In: Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, vol. 2, pp. 347–391. University of California Press, Berkeley, CA (1961)
Neveu, J.: Entrance, exit and fictitious states for Markov chains. In: Proceedings on Aarhus Colloquium Combinatorial Probability, pp. 64–68 (1962)
Neveu, J.: Sur les, etats d’entree ef les etats fictifs d’un processus de Markov. Ann. Inst. H. Poincare’ 17, 323–337 (1962b)
Reuter, G.E.H.: Denumerable Markov processes and the associated semigroup on \({\cal{L}}\). Acta. Math. 97, 1–46 (1957)
Reuter, G.E.H.: Denumerable Markov processes (II). J. Lond. Math. Soc. 34, 81–91 (1959)
Reuter, G.E.H.: Denumerable Markov processes (III). J. Lond. Math. Soc. 37, 63–73 (1962)
Reuter, G.E.H.: Remarks on a Markov chain example of Kolmogorov. Z. Wabrscheinlichkeits-theor. Verw. Geb. 13, 315–320 (1969)
Reuter, G.E.H.: Denumerable Markov processes (IV). On C.T. Hou’s uniqueness theorm for \(Q\)-semigroups. Z. Wabrscheinlichkeitstheor. Verw. Geb. 33, 309–315 (1976)
Rogers, L.C.G., Williams, D.: Construction and approximation of transition matrix functions. Adv. Appl. Probab. 18, 133–160 (1986)
Rogers, L.C.G., Williams, D.: Diffusions, Markov Processes and Martingales, vol. 1, 2nd edn. Wiley, Chichester (1994)
Syski, R.: Passage Times for Markov Chains. IOS Press, Amsterdam (1992)
Williams, D.: A note on the \(Q\)-matrices of Markov chains. Z. Wabrscheinlichkeitstheor. Verw. Geb. 7, 116–121 (1967)
Yang, X.Q.: The Construction Theory of Denumerable Markov Processes. Wiley, New York (1990)
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Chen, A. Resolvent Decomposition Theorems and Their Application in Denumerable Markov Processes with Instantaneous States. J Theor Probab 33, 2089–2118 (2020). https://doi.org/10.1007/s10959-019-00941-w
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10959-019-00941-w