Jeffrey Meets Kolmogorov

Abstract

Jeffrey conditionalization is a rule for updating degrees of belief in light of uncertain evidence. It is usually assumed that the partitions involved in Jeffrey conditionalization are finite and only contain positive-credence elements. But there are interesting examples, involving continuous quantities, in which this is not the case. Q1 Can Jeffrey conditionalization be generalized to accommodate continuous cases? Meanwhile, several authors, such as Kenny Easwaran and Michael Rescorla, have been interested in Kolmogorov’s theory of regular conditional distributions (rcds) as a possible framework for conditional probability which handles probability-zero events. However the theory faces a major shortcoming: it seems messy and ad hoc. Q2 Is there some axiomatic theory which would justify and constrain the use of rcds, thus serving as a possible foundation for conditional probability? These two questions appear unrelated, but they are not, and this paper answers both. We show that when one appropriately generalizes Jeffrey conditionalization as in Q1, one obtains a framework which necessitates the use of rcds. It is then a short step to develop a general theory which addresses Q2, which we call the theory of extensions. The theory is a formal model of conditioning which recovers Bayesian conditionalization, Jeffrey conditionalization, and conditionalization via rcds as special cases.

Appendices

Appendix A: Existence Results

1.1 A.1 Example 1 (Lake)

In Section 2.5, we claimed that a Jeffrey conditionalization on σY exists in this example. To prove this claim, we have to be more precise about the background probability space $({\varOmega },{\mathcal{F}}, P)$, which we left undefined in our discussion.

Recall that there are three quantities in the example: X denotes the distance from your boat to a particular buoy; Y denotes the distance from your boat to the pier; and Z denotes the distance between the buoy and the pier. Since Y, Z are assumed to be probabilistically independent according to P, and X is a function of Y and Z (so σX ⊂ σ{Y, Z}), a natural modeling choice for $({\varOmega },{\mathcal{F}},P)$ is the product space $([10,30]\times [80,120], {\mathcal{F}}_{Z}\otimes {\mathcal{F}}_{Y}, P)$ where ${\mathcal{F}}_{Y}$ and ${\mathcal{F}}_{Z}$ are the Borel algebras on [80, 120] and [10, 30] respectively, and P is the product measure P ∘ Z^− 1 ⊗ P ∘ Y^− 1 (recall that both P ∘ Y^− 1 and P ∘ Z^− 1 are uniform on their respective images). Note that, in this formal model, Y, Z are the projection maps that map ω to its respective coordinates.

With this set-up in mind, we now state the result.

Proposition 1

In Example 1, there exists a Jeffrey conditionalization of P given σY.

To prove this proposition we use the following theorem. As before, proofs are in Appendix C.

Definition 1

Given a probability space $({\varOmega },{\mathcal{F}},P)$ and two sub-sigma-algebras ${\mathcal{G}}_{1},{\mathcal{G}}_{2}\subset {\mathcal{F}}$, we say a map $Q: {\mathcal{G}}_{2}\times {\varOmega }\rightarrow [0,1]$ is a regular conditional distribution of $P|_{{\mathcal{G}}_{2}}$given ${\mathcal{G}}_{1}$ if it is the restriction of a regular conditional distribution of $P_{{\mathcal{G}}_{1}}$ to ${\mathcal{G}}_{2}\times {\varOmega }$.

Theorem 1

Let $({\varOmega },{\mathcal{F}},P)$be a product measure space where Ω = Ω₁ ×Ω₂, ${\mathcal{F}}={\mathcal{F}}_{1}\otimes {\mathcal{F}}_{2}$ (the smallest sigma-algebra generated by measurable rectangles of the form A ∩ B, where $A\in {\mathcal{F}}_{1}$and $B\in {\mathcal{F}}_{2}$). Let Q₁, Q₂ be conditional distribution of ${\mathcal{F}}_{1}, {\mathcal{F}}_{2}$given ${\mathcal{G}}$respectively. Define $Q:{\mathcal{F}}\times {\varOmega }\rightarrow \mathbb {R}$such that Q(., ω) is the unique product measure Q₁(., ω) ⊗ Q₂(., ω). ¹Then Q is regular and ${\mathcal{G}}$-measurable. In particular, if ${\mathcal{G}}={\mathcal{F}}_{2}$, then Q₂(., ω) = δ_ω and Q is a prcd.

1.2 A.2 Example 6 (Coin Machine)

In this example, we set Ω = {H, T}× [0, 1]. ${\mathcal{F}}$ is the field generated by sets of the form {H}× [0, a] and {T}× [0, b], where a, b ∈ [0, 1]. (Equivalently, ${\mathcal{F}}$ is the product measure ${\mathcal{F}}_{1}\otimes {\mathcal{F}}_{2}$ where ${\mathcal{F}}_{1}$ is the four-element algebra {∅,{H},{T},{H, T}} and ${\mathcal{F}}_{2}$ is the Borel algebra on [0, 1].)

The prior probability assignment $P:{\mathcal{F}}\rightarrow [0,1]$ is the unique probability distribution satisfying $P\{(H,a): a\leq b\}=\frac {b^{2}}{2}$. (Let $Y:{\varOmega }\rightarrow [0,1]$ be the projection map onto the second coordinate. Then note that P ∘ Y^− 1 is the uniform distribution on [0, 1], as desired.)

With this set-up in mind, we state the existence result.

Proposition 2

In Example 6, there exists a complete uniform extension $E^{P}:{\varDelta }(\sigma Y)\rightarrow {\varDelta }({\mathcal{F}})$satisfying $[E^{P}({\delta _{b}^{Y}})](H)=b$for all b ∈ [0, 1]. (Recall that ${\delta _{b}^{Y}}$is the Dirac measure concentrated on the point Y = b. Intuitively $[E^{P}({\delta _{b}^{Y}})]$is the conditional probability P(H|Y = b).)

Appendix B: Conditional Densities

Definition 2

Let $Y: {\varOmega } \rightarrow \mathbb {R}^{n}$ be an absolutely continuous random variable. A probability density function (pdf) for Y is a map $f_{Y}: \mathbb {R}^{n} \rightarrow \mathbb {R}$ which satisfies ${\int \limits }_{B} f_{Y}(y) dy = P(Y \in B)$ for every Borel set $B \subset \mathbb {R}^{n}$. A joint pdf for ${\mathcal{F}}$ and Y is a map $f_{Y}: {\mathcal{F}} \times \mathbb {R}^{n} \rightarrow \mathbb {R}$ which satisfies ${\int \limits }_{B} f_{Y}(F,y) dy = P(F \cap (Y \in B))$ for all $F \in {\mathcal{F}}$ and every Borel set $B \subset \mathbb {R}^{n}$.

Theorem 2 (Conditional Densities)

Let $Y: {\varOmega } \rightarrow \mathbb {R}^{n}$be an absolutely continuous random variable. Let f_Y be a pdf for Y, let S denote the set of values supported by f_Y(.), and let q_Y denote the measure q(Y ∈ .). Set ${\mathcal{G}} = \sigma Y$. Any Jeffrey conditionalization (or, equivalently, any uniform extension—see Section ??), $E^{P}: {\varTheta }({\mathcal{G}}) \rightarrow {\varDelta }({\mathcal{F}})$, can be written in the form

$$ E^{P}(q) =\underbrace{{\int}_{S} \frac{f_{Y}(\cdot,y)}{f_{Y}(y)} dq_{Y}(y)}_{\text{conditional density term}} + \underbrace{{\int}_{S^{c}} g(\cdot,y) dq_{Y}(y)}_{\text{extra term}} $$

for all $q \in {\varTheta }({\mathcal{G}})$, where P(Y ∈ S) = 1, f_Y is a joint pdf for ${\mathcal{F}}$and Y, and $g: {\mathcal{F}} \times \mathbb {R}^{n} \rightarrow \mathbb {R}$is some function. Furthermore, fix $F \in {\mathcal{F}}$. Then

$$ \frac{f_{Y}(F,b)}{f_{Y}(b)} = \lim_{\epsilon\rightarrow 0} \frac{P(F \cap Y \in B_{\epsilon}(b))}{P(Y \in B_{\epsilon}(b))} $$

for P-almost every value $b \in \mathbb {R}^{n}$.

Appendix C: Proofs

1.1 C.1 Propositions

Proof

(Proposition 1) It suffices to show that, for all ω ∈ G_i,

$$ \frac{Q(F \cap G_{i})}{q(G_{i})}= \frac{dQ(F \cap .)}{dq(.)}(\omega), \qquad \frac{P(F \cap G_{i})}{p(G_{i})}=\frac{dP(F \cap .)}{dp(.)}(\omega). $$

We will show it for Q; the reasoning for P is the same. Since the derivative $\frac {dQ(F \cap .)}{dq(.)}$ is ${\mathcal{G}}$-measurable and ${\mathcal{G}}$ is generated by π, it must be of the form ${\sum }_{j} c_{j} \cdot 1_{G_{j}}$. By the integral condition, $Q(F \cap G_{i})= {\int \limits }_{G_{i}} {\sum }_{j} c_{j} 1_{G_{j}}(\omega ) dq(\omega ) = {\sum }_{j} c_{j} ({\int \limits }_{G_{i}} 1_{G_{j}}(\omega ) dq(\omega )) = c_{i}\cdot q(G_{i})$, so $c_{i} = \frac {Q(F \cap G_{i})}{q(G_{i})}$ as desired. □

Proof

(Proposition 2) (⇒) Immediate from the definition of a Radon-Nikodym derivative. For (⇐) it suffices to show

$$ Q(F \cap G)={\int}_{G} \frac{dP(F\cap .)}{dp(.)} dq \qquad \forall G \in \mathcal{G}. $$

We use the fact $Q: {\mathcal{F}} \rightarrow [0,1]$ is a probability measure and $Q=q|_{{\mathcal{G}}}$. In particular we borrow reasoning from the proof of Theorem 2, below. First use the reasoning in (⇒) to show $\frac {dP(\cdot \cap .)}{dp(.)}(\omega ):{\mathcal{F}} \rightarrow [0,1]$ is a probability measure and $\frac {dP(G \cap .)}{dp(.)}(\omega )=1_{G}(\omega )$ for all $G \in {\mathcal{G}}$ and ω ∈Ω. Then use this, together with the reasoning in (⇐), to show ${\int \limits }_{G} \frac {dP(F\cap .)}{dp(.)} dq = {\int \limits } \frac {dP(F \cap G\cap .)}{dp(.)} dq$ which equals Q(F ∩ G) by assumption. □

Proof

(Proposition 3) Fix $F \in {\mathcal{F}}$ and $q \in {\varTheta }({\mathcal{G}})$. Note that q(.) = 0 ⇒ [E^P(q)](F ∩ .) = 0 by faithfulness and the fact E^P(q) is a probability measure. Thus the Radon-Nikodym theorem implies there is a density $K^{P}(F,q): {\varOmega } \rightarrow \mathbb {R}$ which satisfies both equations in the proposition. We then let K^P be the map from pairs (F, q) to the associated densities. □

Proof

(Proposition 4) Supposing E^P is uniform, we obtain the result using the same reasoning as in the proof of Proposition 1 combined with the result of Proposition 5 below. And for the converse note that this E^P is obviously uniform. □

Proof

(Proposition 5) (⇒) Let K^P(F, q) be the density $\frac {dP(F \cap .)}{dp(.)}$ satisfying rigidity. It does not depend on q at all, and conservativeness follows from the generalized law of total probability. (⇐) Let $K^{P}: {\mathcal{F}} \rightarrow {\mathcal{L}}^{1}({\mathcal{G}})$ be the uniform schema (without loss of generality, we omit the second argument place). We claim K(F) is a density satisfying rigidity. It is ${\mathcal{G}}$-measurable. By conservativeness and the definition of a schema (see Proposition 3) we have $P(F \cap G)=[E^{P}(p)](F \cap G) = {\int \limits }_{G} K^{P}(F) dp$ which is the law of total probability. Thus it is a density. To see it satisfies rigidity note that also,

$$ [E^{P}(q)](F \cap G) = {\int}_{G} K^{P}(F) dq \qquad \forall q \in {\varTheta}(\mathcal{G}), $$

and so, for all $q \in {\varTheta }({\mathcal{G}})$,

$$ \frac{d[E^{P}(q)](F \cap .)}{dq(.)}=K^{P}(F) $$

almost surely with respect to q, as desired. □

Proof

(Proposition 6) The proof is a slight variant of [24, Lemma 2 and Theorem 3]. It suffices to show that, for any almost regular conditional distribution $P_{{\mathcal{G}}}$ on ${\mathcal{G}}$,

$$ P_{\mathcal{G}} (\cdot,\omega)=P(\cdot) \qquad \text{for }P\text{-almost every }\omega \in {\varOmega}. $$

This suffices because, fixing such an ω and letting G_ω be the atom containing ω (see footnote ??), it follows that P_G(G_ω, ω) = P(G_ω) = 0 ≠ 1 (here we use the fact that G_ω has countably many elements, and hence is P-zero). Since $G_{\omega } \in {\mathcal{G}}$ (since this ${\mathcal{G}}$ is atomic; see footnote ??), this is a violation of propriety at ω. To show this conclusion holds, first we note, by the Kolmogorov zero–one law, that P(G) equals 0 or 1 for every $G \in {\mathcal{G}}$. Next we note that ${\mathcal{F}}$ is countably generated. For each generator F_i, we define

$$ H_{i}^{+} = \{\omega: P_{\mathcal{G}}(F_{i},\omega) > P(F_{i})\}, \qquad H_{i}^{-} = \{\omega: P_{\mathcal{G}}(F_{i},\omega) < P(F_{i})\}, $$

$$ H_{i} = \{\omega: P_{\mathcal{G}}(F_{i},\omega) = P(F_{i})\}. $$

Since $P_{{\mathcal{G}}}(G_{i},\cdot )$ is ${\mathcal{G}}$-measurable and P(G_i) is constant, these three sets belong to ${\mathcal{G}}$ and hence have probability either 0 or 1. Now suppose toward contradiction that $H_{i}^{+}$ has probability 1. By the generalized law of total probability, we then obtain

$$ P(F_{i} \cap H_{i}^{+}) = {\int}_{H_{i}^{+}} P_{\mathcal{G}}(F_{i},\omega) dP(\omega) > {\int}_{H_{i}^{+}} P(F_{i}) dP(\omega) = P(F_{i})P(H_{i}^{+})=P(F_{i} \cap H_{i}^{+}), $$

which is a contradiction. The same reasoning applies to $H^{-}_{i}$. It then follows that H_i has probability one, for any i. Now let S denote the probability-one set on which $P_{{\mathcal{G}}}$ is regular, and define D ≡ (∩_iH_i) ∩ S. First note that P(D) = 1. Next note that, by definition:

$$ \omega \in D \Longrightarrow P_{\mathcal{G}}(F_{i},\omega)=P(F_{i}) \ \text{for all }i\text{, and } P_{\mathcal{G}}(\cdot,\omega) \ \text{is a countably additive measure.} $$

Since the F_i generate ${\mathcal{F}}$, if two probability measures agree on all the F_i then they must agree on all of ${\mathcal{F}}$. And so $\omega \in D \Longrightarrow P_{{\mathcal{G}}}(\cdot ,\omega )=P(\cdot )$. Since P(D) = 1, this is enough to establish the conclusion. □

Proof

(Proposition 7) We will show that, in Example 7, there exists a proper conditional distribution $P_{{\mathcal{G}}}$ such that $P_{{\mathcal{G}}}(.,\omega )$ is a finitely additive probability distribution for any ω ∈Ω. Then, by the same reasoning as the proof of Theorem 2 (direction ⇐), it follows that a complete uniform extension* on ${\mathcal{G}}$ exists—note, crucially, that the demonstration of rigidity relies only on finite additivity, not on countable additivity. First, in Example 7 define $X_{n}:{\varOmega }\rightarrow \{0,1\}$ that projects each ω to its n-th coordinate. Intuitively, X_n records the outcome of n-th toss. Let ${\mathcal{F}}_{n}=\sigma \{X_{1},\dots ,X_{n-1}\}$ and ${\mathcal{G}}_{n}=\sigma \{X_{m}: m\geq n\}$. Then ${\mathcal{G}}=\bigcap _{n}{\mathcal{G}}_{n}$ and ${\mathcal{F}}={\mathcal{F}}_{n}\vee {\mathcal{G}}_{n}$. (Equivalently, ${\mathcal{F}}={\mathcal{F}}_{n}\otimes {\mathcal{G}}_{n}$ where ${\mathcal{F}}_{n}$ and ${\mathcal{G}}_{n}$ are viewed as sigma-algebras on ${\Omega }_{n}={0,1}^{n}$ and Ω respectively. Note that Ω = Ω_n ×Ω for all n. In what follows we use these two notations interchangeably.) By Lemma 1 below, there exists a conditional distribution of P on ${\mathcal{G}}_{n}$ for each n. Next, a definition: given a measure space $({\varOmega },{\mathcal{F}})$ and ${\mathcal{G}}\subset {\mathcal{F}}$, we say ${\mathcal{G}}$ is tame if there exists a proper conditional distribution of P given ${\mathcal{G}}$ which is finitely additive (for any ω ∈Ω) for any countably additive probability distribution P on $({\varOmega },{\mathcal{F}})$. By Lemma 2 below (a result due to Dubins and Heath), it then follows from the above statement that there exists a proper conditional distribution of P given the tail-sigma-field ${\mathcal{G}}$ which is finitely additive (for any ω ∈Ω) as claimed. □

Lemma 1

For each n, there is a conditional distribution $P_{{\mathcal{G}}_{n}}$of P on ${\mathcal{G}}_{n}$such that $P_{{\mathcal{G}}_{n}}(\cdot , \omega )$is finitely additive for all ω ∈Ω.²

Lemma 2 (Dubins and Heath [6, Proposition 1])

The intersection of a decreasing sequence of sigma fields, each of which is tame, is tame.

Proof

(Lemma 1) Let $P_{n}=P|_{{\mathcal{F}}_{n}}$. First, we claim that, for each n, there exists a regular conditional distribution of P_n given ${\mathcal{G}}_{n}$. Consider $Q_{n}: {\mathcal{F}}_{n}\times {\varOmega }\rightarrow [0,1]$ such that Q_n(F, ω) = P(F) for any ω ∈Ω. We verify that Q_n so defined is an additive conditional distribution of $X_{1},\dots ,X_{n-1}$ given {X_m : m ≥ n}. Both measurability and additivity are trivial. So it remains to verify that Q_n satisfies the generalized law of total probability in the sense that $P(F\cap G)={\int \limits }_{B}Q_{n}(F,\omega )dP$ for any $F\in {\mathcal{F}}_{n}$ and $G\in {\mathcal{G}}_{n}$. Let $F\in {\mathcal{F}}_{n}$ and $G\in {\mathcal{G}}_{n}$. Then ${\int \limits }_{B}Q_{n}(F,\omega )dP=P(F)P(G)=P(F\cap G)$, where the last equality follows from the fact that ${\mathcal{F}}_{n}$ is independent of ${\mathcal{G}}_{n}$. Now define $P_{{\mathcal{G}}_{n}}:{\mathcal{F}}\times {\varOmega }\rightarrow [0,1]$ as the product measure $P_{{\mathcal{G}}_{n}}(.,\omega )=Q_{n}(.,\omega )\otimes \delta _{\omega }(.)$, where we recall that ${\mathcal{F}}={\mathcal{F}}_{n}\otimes {\mathcal{G}}_{n}$. By Theorem 6 (Appendix A), $P_{{\mathcal{G}}_{n}}$ is a prcd of P on ${\mathcal{G}}_{n}$. □

Proof

(Proposition 8) By Corollary 1 and 2, such a Jeffrey conditionalization exists just in case there is a proper, regular conditional distribution $P_{{\mathcal{G}}}$, where ${\mathcal{G}}=\sigma Y$. First, we claim there is a regular conditional distribution of Z given Y. Consider $Q_{Z}: \sigma Z\times {\varOmega }\rightarrow [0,1]$ given by

$$ Q_{Z}(A, \omega)=P(A)\quad\forall A\in\mathcal{F}_{Z}, \omega\in{\varOmega}. $$

(1)

It is straightforward to verify Q_Z is a regular conditional distribution: Regularity and measurability are trivial; since Y, Z are independent, for any $A\in {\mathcal{F}}_{Z}$ and $B\in {\mathcal{F}}_{Y}$, $P(A\cap B)=P(A)P(B)={\int \limits }_{B}Q_{Z}(A,\omega )dP$. So Q_Z also satisfies the generalized law of total probability. Now, let ${\mathcal{F}}_{1}={\mathcal{F}}_{Z}$, ${\mathcal{F}}_{2}={\mathcal{F}}_{Y}$, Q₁ = Q_Z and Q₂ be the trivial conditional distribution that maps each (A, ω) to δ_ω(A). By Theorem 6 (Appendix A), $P_{{\mathcal{G}}}$ defined as $P_{{\mathcal{G}}}(.,\omega )=Q_{1}(.,\omega )\otimes Q_{2}(.,\omega )$ is an everywhere prcd of P given σY. □

Proof

(Proposition 9) Let $X:{\varOmega }\rightarrow \{H,T\}$ be the other projection map and P_X = P|_σX. As above, the strategy is to construct a regular conditional distribution of P_X given σY and extend it to a prcd of P on σY. Define $Q_{X}:\sigma X\times {\varOmega }\rightarrow [0,1]$ as

$$ Q_{X}(A,\omega)=\lim_{\epsilon\rightarrow 0}\frac{P((X\in A)\cap |Y-Y(\omega)|\leq\epsilon)}{P(|Y-Y(\omega)|\leq\epsilon)}. $$

(2)

We check that Q_X is a conditional distribution of P_X given σY. First, we observe that Q_X({H, T}, ω) = 1, Q_X(∅, ω) = 0, Q_X({H}, ω) = Y (ω) and Q_X({T}, ω) = 1 − Y (ω) for any ω ∈Ω. We verify that Q_X so defined is in fact a conditional distribution.

• (measurability) By a well-known result in measure theory (e.g. [3, Theorem II.4.4]), a real-valued function $f:{\varOmega }\rightarrow \mathbb {R}$ is measurable with respect to ${\mathcal{G}}=\sigma Y$ iff there is a real-valued function g on $\mathbb {R}$ such that f = g ∘ Y. So it suffices to show that for any A ∈ σX, the map Q(A,.) is a function of Y. Consider $f_{n}(\omega )=\frac {P((X\in A)\cap |Y-Y(\omega )|\leq 1/n)}{P(|Y-Y(\omega )|\leq 1/n)}$ and $g_{n}(x)=\frac {P((X\in A)\cap |Y-x|\leq 1/n)}{P(|Y-x|\leq 1/n)}$. Then f_n(ω) = g_n ∘ Y (ω) and $Q_{X}(A,.)=\lim _{n}f_{n}$. Since each f_n is ${\mathcal{G}}$-measurable and measurability is closed under limit, it follows that Q_X(A,.) is ${\mathcal{G}}$-measurable.

• (regularity) This follows trivially from the above observation about values of Q_X(., ω).

• (generalized law of total probability) Let A ∈ σX and B ∈ σY. Clearly if A = {H, T}, then

  $${\int}_{B}Q_{X}(A,\omega)dP=P(B)=P((X\in A)\cap B).$$

  Similarly, if A = ∅, then ${\int \limits }_{B}Q_{X}(A,\omega )dP=0=P((X\in A)\cap B)$. So it remains to check the cases of A = {H} and A = {T}. Let A = {H} and μ = PY^− 1 be the uniform distribution on [0, 1]. Then

  $$ {\int}_{B} Q_{X}(H,\omega)dP={\int}_{B} Y(\omega)dP={\int}_{Y(B)} t \mu(dt)=P(H\cap B). $$

  (3)

  The case of X = {T} follows analogously.

Now note that ${\mathcal{F}}=\sigma X\otimes \sigma Y$. Let Q₁ = Q_X and $Q_{2}:\sigma Y\times {\varOmega }\rightarrow [0,1]$ defined as Q₂(., ω) = δ_ω. By Theorem 6 in Appendix A, $P_{{\mathcal{G}}}(.,\omega )=Q_{1}(.,\omega )\otimes Q_{2}(.,\omega )$ is a prcd of P given σY. By Corollary 2, if $P_{{\mathcal{G}}}$ is a prcd of P given σY, then

$$ [E^{P}(q)](F)={\int}_{{\varOmega}} P_{\mathcal{G}}(F,\omega)dq(\omega) $$

(4)

defines a complete uniform extension of σY (given P). Thus there is a complete uniform extension in Example 6. Moreover, note that $P_{{\mathcal{G}}}(H,\omega )=Q_{1}(H,\omega )=Y(\omega )$. So

$$ [E^{P}({\delta_{b}^{Y}})](H)={\int}_{{\varOmega}} P_{\mathcal{G}}(H,\omega){\delta_{b}^{Y}}(d\omega)={\int}_{{\varOmega}} Y(\omega){\delta_{b}^{Y}}(d\omega)=b, $$

(5)

as desired. □

1.2 C.2 Theorems

Proof

(Theorem 2) (⇐) We first show that the assignment sends $q \in {\varDelta }({\mathcal{G}})$ to Q which are probability measures, i.e. belong to ${\varDelta }({\mathcal{F}})$. This follows essentially from the fact that $P_{{\mathcal{G}}}$ is regular. More specifically:

• (non-negativity) $\forall F \in {\mathcal{F}}$, $Q(F)= {\int \limits }_{{\varOmega }} P_{{\mathcal{G}}}(F, \omega ) dq(\omega ) \geq 0$ since $P_{{\mathcal{G}}}(F,\omega ) \geq 0$ for all $F \in {\mathcal{F}}$;

• (normalizability) $Q({\varOmega }) = {\int \limits }_{{\varOmega }} P_{{\mathcal{G}}}({\varOmega }, \omega ) dq(\omega ) = {\int \limits }_{{\varOmega }} 1 dq(\omega ) = q({\varOmega }) = 1$;

• (countable additivity) Let {F_i}_i∈I be a countable collection of disjoint sets in ${\mathcal{F}}$ (I may be finite). Then,

  $$ \begin{array}{@{}rcl@{}} Q(\cup_{i} F_{i})&=&{\int}_{{\varOmega}} P_{\mathcal{G}}(\cup_{i} F_{i}, \omega) dq(\omega) = {\int}_{{\varOmega}} \sum\limits_{i} P_{\mathcal{G}}(F_{i},\omega) dq(\omega) \\ &=& \sum\limits_{i} {\int}_{{\varOmega}} P_{\mathcal{G}}(F_{i},\omega) dq(\omega)= \sum\limits_{i} Q(F_{i}), \end{array} $$

  (6)

  where we use the Monotone Convergence Theorem.

Next we show $Q|_{{\mathcal{G}}} = q$. This follows from propriety; for all $G \in {\mathcal{G}}$,

$$ Q(G)= {\int}_{{\varOmega}} P_{\mathcal{G}}(G,\omega) dq(\omega) = {\int}_{{\varOmega}} 1_{G}(\omega) dq(\omega) = q(G). $$

Finally, for rigidity, fix $F \in {\mathcal{F}}$ and note that $P_{{\mathcal{G}}}(F, \cdot ): {\varOmega } \rightarrow \mathbb {R}$ is a density $\frac {dP(F \cap .)}{dp(.)} : {\varOmega } \rightarrow \mathbb {R}$ since the generalized law of total probability is precisely the integral condition for a Radon-Nikodym derivative, where μ = p and ν = P(F ∩ .). So it suffices to show,

$$ Q(F \cap G) = {\int}_{G} P_{\mathcal{G}}(F, \omega) dq(\omega) $$

for all $G \in {\mathcal{G}}$. By regularity $P_{{\mathcal{G}}}(F, \omega )=P_{{\mathcal{G}}}(F \cap G, \omega )+P_{{\mathcal{G}}}(F \cap G^{c}, \omega )$ and so it suffices to show

$$ Q(F \cap G) = {\int}_{G} P_{\mathcal{G}}(F\cap G, \omega) dq(\omega)+{\int}_{G} P_{\mathcal{G}}(F \cap G^{c}, \omega) dq(\omega). $$

By regularity and propriety, $P_{{\mathcal{G}}}(F \cap G, \omega ) \leq P_{{\mathcal{G}}}(G,\omega ) =1_{G}(\omega )$ and $P_{{\mathcal{G}}}(F \cap G^{c}, \omega ) \leq P_{{\mathcal{G}}}(G^{c},\omega ) =1_{G^{c}}(\omega )$. So if ω∉G then $P_{{\mathcal{G}}}(F\cap G,\omega ) = 0$ and if ω ∈ G then $P_{{\mathcal{G}}}(F \cap G^{c}, \omega )=0$. Thus the above equation is equivalent to

$$ Q(F \cap G)= {\int}_{{\varOmega}} P_{\mathcal{G}}(F \cap G, \omega) dq(\omega) +0. $$

By assumption the right-hand side equals Q(F ∩ G) and so we are done.

(⇒) Consider the map $P_{{\mathcal{G}}}$ defined in the statement of the theorem. By definition $P_{{\mathcal{G}}}(F, \cdot )$ is ${\mathcal{G}}$-measurable and satisfies the generalized law of total probability since $\frac {dP(F \cap .)}{dp(.)}$ is ${\mathcal{G}}$-measurable and satisfies the integral condition for a Radon-Nikodym derivative, where μ = p and ν = P(F ∩ .). It remains to show propriety and regularity.

• (propriety) Fix $G \in {\mathcal{G}}$. Let $H_{G}^{-} = \{\omega \in {\varOmega }: P_{{\mathcal{G}}}(G,\omega ) < 1_{G}(\omega )\}$. Note that $H_{G}^{-} \in {\mathcal{G}}$. Suppose toward contradiction that $H_{G}^{-} \neq \emptyset$. Choose $q \in {\varDelta }({\mathcal{G}})$ such that $q(H_{G}^{-})=1$, for example a Dirac measure concentrated at a point in $H_{G}^{-}$. By rigidity,

  $$ \begin{array}{@{}rcl@{}} Q(G)&=&{\int}_{{\varOmega}} P_{\mathcal{G}}(G, \omega) dq(\omega)={\int}_{H_{G}^{-}} P_{\mathcal{G}}(G, \omega) dq(\omega)\\ &&< {\int}_{{\varOmega}} 1_{G}(\omega) dq(\omega) = q(G), \end{array} $$

  which violates the faithfulness requirement $Q|_{{\mathcal{G}}} = q$, a contradiction. Similar reasoning with $H_{G}^{+} = \{\omega \in {\varOmega }: P_{{\mathcal{G}}}(G,\omega ) > 1_{G}(\omega )\}$ implies $P_{{\mathcal{G}}}(G,\omega )=1_{G}(\omega )$ for all ω ∈Ω, as desired.

• (regularity) Fix a countable collection $\mathcal {C}=\{F_{i}\}_{i \in I}$ disjoint and in ${\mathcal{F}}$. Let $A_{\mathcal {C}}^{-} = \{\omega \in {\varOmega }: P_{{\mathcal{G}}}(\cup _{i} F_{i}, \omega ) < {\sum }_{i} P_{{\mathcal{G}}}(F_{i},\omega )\}$ which belongs to ${\mathcal{G}}$ [1, Theorem 13.4]. Using the same reasoning as above: Pick $q \in {\varDelta }({\mathcal{G}})$ such that $q(A_{\mathcal {C}}^{-})=1$. We have, using the Monotone Convergence Theorem,

  $$ \begin{array}{@{}rcl@{}} Q(\cup_{i} F_{i}) &=& {\int}_{{\varOmega}} P_{\mathcal{G}}(\cup_{i} F_{i}, \omega) dq(\omega)={\int}_{A_{\mathcal{C}}^{-}} P_{\mathcal{G}}(\cup_{i} F_{i}, \omega) dq(\omega)\\ &&< {\int}_{{\varOmega}} \sum\limits_{i} P_{\mathcal{G}}(F_{i},\omega) dq(\omega) = \sum\limits_{i} {\int}_{{\varOmega}} P_{\mathcal{G}}(F_{i},\omega) dq(\omega) = \sum\limits_{i} Q(F_{i}), \end{array} $$

  which contradicts the assumption $Q \in {\varDelta }({\mathcal{F}})$. Repeat with $A_{\mathcal {C}}^{+}$ to show $\{\omega \in {\varOmega }: P_{{\mathcal{G}}}(\cup _{i} F_{i}, \omega ) = {\sum }_{i} P_{{\mathcal{G}}}(F_{i},\omega )\}={\varOmega }$ as desired.

And we are done. □

Proof

(Theorem 3) By Radon-Nikodym there exists a conditional distribution $P_{{\mathcal{G}}}: {\mathcal{F}} \times {\varOmega } \rightarrow \mathbb {R}$. Let $E^{P}: {\varDelta }^{+}({\mathcal{G}}) \rightarrow {\varDelta }({\mathcal{F}})$ be the map defined by

$$ E^{P}(F) = {\int}_{{\varOmega}} P_{\mathcal{G}}(F,\omega) dq(\omega) \qquad \forall q \in {\varDelta}^{+}(\mathcal{G}). $$

We claim that E^P is a uniform minimal extension. Note it is uniform and minimal by construction. It is a standard result [1, Theorem 33.2] that, fixing some countable disjoint sequence {F_i}_i∈I, for P-almost every ω ∈Ω we have $0 \leq P_{{\mathcal{G}}}(F_{i},\omega ) \leq 1$ and $P_{{\mathcal{G}}}(\cup _{i} F_{i}, \omega ) = {\sum }_{i} P_{{\mathcal{G}}}(F_{i}, \omega )$. So,

$$ [E^{P}(q)](\cup_{i} F_{i}) = {\int}_{{\varOmega}} {\sum}_{i} P_{\mathcal{G}}(F_{i},\omega) dq(\omega) ={\sum}_{i} [E^{P}(q)](F_{i}), $$

where the first equality follows from the fact q is absolutely continuous with respect to P, and the second follows from the same reasoning as Eq. 6. Thus $E^{P}(q) \in {\varDelta }({\mathcal{F}})$. For faithfulness we use the well-known result that, for any $G \in {\mathcal{G}}$, its conditional expectation with respect to ${\mathcal{G}}$ is almost surely equal to its indicator, i.e. we have $P_{{\mathcal{G}}}(G,\omega )=1_{G}(\omega )$ for P-almost every ω ∈Ω (this property is sometimes called conditional determinism [3, 144]). Since q is absolutely continuous with respect to p,

$$ [E^{P}(q)](G)={\int}_{{\varOmega}} 1_{G}(\omega) dq(\omega) = q(G). $$

For conservativeness note,

$$ [E^{P}(p)](F)={\int}_{{\varOmega}} P_{\mathcal{G}}(F,\omega) dp = P(F \cap {\varOmega})=P(F), $$

using the generalized law of total probability.

Finally, we show uniqueness. Suppose $\bar {E}^{P}: {\varDelta }^{+}({\mathcal{G}})) \rightarrow {\varDelta }({\mathcal{F}})$ is another uniform extension. Thanks to conservativeness the associated schema K^P must be such that $K^{P}(\cdot , q)(\cdot ): {\mathcal{F}} \times {\varOmega } \rightarrow \mathbb {R}$ is a conditional distribution of P on ${\mathcal{G}}$. K^P(F, q) is unique P-almost surely and does not vary as q varies except perhaps on a P-null set. Since every $q \in {\varDelta }^{+}({\mathcal{G}})$ is absolutely continuous with respect to P, it follows K^P(F, q) can be treated as $P_{{\mathcal{G}}}(F,\cdot )$ from above and hence $E^{P}(q)=\bar {E}^{P}(q)$ for all $q \in {\varDelta }^{+}({\mathcal{G}})$. □

Proof

(Theorem 4) (⇐) Let S be the probability-one set on which regularity and propriety are both satisfied. The proof proceeds much the same way as Theorem 2, except we must concentrate the integrals over S. Since S may not belong to ${\mathcal{G}}$, the adjustment is not completely trivial. To see how this is done consider non-negativity. We fix any $F \in {\mathcal{F}}$ and any $q \in {\varDelta }^{S}({\mathcal{G}})$. The trick is to define

$$ N_{F}= \{\omega \in {\varOmega}: P_{\mathcal{G}}(F,\omega) < 0\}, $$

which belongs to ${\mathcal{G}}$ (since $P_{{\mathcal{G}}}(F,\cdot )$ is ${\mathcal{G}}$-measurable) and falls inside the complement of S. Hence, by definition of ${\varDelta }^{S}({\mathcal{G}})$, q(N_F) = 0 and so (since q is a probability measure) $q({N_{F}^{c}})=1$. Now we can freely swap out the domain of integration Ω (in the integrals in the proof of Theorem 2) with the complement of N_F, allowing us to show: $Q(G) = {\int \limits }_{{\varOmega }} P_{{\mathcal{G}}}(F,\omega ) dq(\omega ) = {\int \limits }_{{N_{F}^{c}}} P_{{\mathcal{G}}}(F,\omega ) dq(\omega ) \geq 0$. For the other properties we repeat the same strategy. For normalizabilty we fix any $q \in {\varDelta }^{S}({\mathcal{G}})$ and consider

$$ I_{{\varOmega}} = \{\omega \in {\varOmega}: P_{\mathcal{G}}({\varOmega}, \omega) \neq 1\}, $$

which belongs to ${\mathcal{G}}$ and falls in the complement of S, hence $q(I^{c}_{{\varOmega }})=1$. For countable additivity, we fix any $q \in {\varDelta }^{S}({\mathcal{G}})$ and any countable collection $\mathcal {C}=\{F_{i}\}_{i \in I}$ and consider

$$ A_{\mathcal{C}}=\{\omega \in {\varOmega}: {\sum}_{i} P_{\mathcal{G}}(F_{i},\omega) \ \text{does not converge, or does not equal } P_{\mathcal{G}}(\cup_{i} F_{i},\omega)\}, $$

which belongs to ${\mathcal{G}}$ [1, Theorem 13.4] and falls in the complement of S, hence $q(A^{c}_{\mathcal {C}})=1$. To show faithfulness we fix any $q \in {\varDelta }^{S}({\mathcal{G}})$ and any $G \in {\mathcal{G}}$ and consider

$$ H_{G} = \{\omega \in {\varOmega}: P_{\mathcal{G}}(G,\omega) \neq 1_{G}(\omega) \}, $$

which belongs to ${\mathcal{G}}$ and falls in the complement of S, hence $q({H^{c}_{G}})=1$. For conservativeness note that for all $F \in {\mathcal{F}}$, $[E^{P}(q)](F) ={\int \limits }_{{\varOmega }} P_{{\mathcal{G}}}(F,\omega ) dp(\omega ) = P(F \cap {\varOmega }) = P(F)$ since $P_{{\mathcal{G}}}$ satisfies the generalized law of total probability. Finally, it remains to show $P_{{\mathcal{G}}}$ is a uniform schema. Uniformity is obvious, but we need to show that $P_{{\mathcal{G}}}$ satisfies the role of a schema (recall Proposition 3), i.e., letting Q ≡ [E^P(q)],

$$ Q(F \cap G)={\int}_{G} P_{\mathcal{G}}(F,\omega) dq(\omega) \qquad \forall F \in \mathcal{F}, \forall G \in \mathcal{G}, \forall q \in {\varDelta}^{S}(\mathcal{G}). $$

We can use the same argument as for Theorem 2, except as above we must concentrate on S. To do this we fix any $F\in {\mathcal{F}}$ and $G\in {\mathcal{G}}$. We then define the union of the above sets N, A, H as applied to all the logical combinations of this F and G, and (in the case of A) all the disjoint collections (e.g. {F ∩ G, F ∩ G^c}) that can be made from these combinations. This finite union belongs to ${\mathcal{G}}$, and it falls inside S^c, so q assigns 1 to its complement, the conjunction of all the proper and regular behavior. We then repeat the same reasoning as in the proof of Theorem 2, freely swapping out Ω with this set in the integration.

(⇒) Consider the map $P_{{\mathcal{G}}}$ as defined in the statement of the theorem. By definition $P_{{\mathcal{G}}}(F,\cdot )$ is ${\mathcal{G}}$-measurable and satisfies the generalized law of total probability since by conservativeness and the definition of a schema (recall Proposition 3),

$$ P(F \cap G) = [E^{P}(p)](F\cap G)={\int}_{G} P_{\mathcal{G}}(F,\omega) dp(\omega). $$

It remains to show almost propriety and almost regularity. We follow the same steps as in the proof of Theorem 2, except rather than showing that violations of regularity or propriety can only happen in the emptyset (i.e. can’t happen), we show they can only happen in S^c. For example, for almost propriety, we suppose toward contradiction that $H_{G}^{-}$ is not in the complement of S, so $S \cap H_{G}^{-}$ is non-empty. We then choose q to be a Dirac measure concentrated at a point ω^∗ in $S \cap H_{G}^{-}$, and derive a contradiction in the same way. (Crucially, note that, for such a q, if G ⊂ S^c, then ω^∗∉G and so q(G) = 0 as required. Also note that $\omega ^{*} \in H_{G}^{-}$, and so $q(H_{G}^{-})=1$.) For almost regularity we repeat the same reasoning but with $A_{\mathcal {C}}^{-}$. □

Proof

(Theorem 6) We check that Q satisfies the two desired properties:

• (regularity) This is trivial since Q(., ω) is a probability measure by construction.

• (measurability) Let $\mathcal {R}$ denote the collection of measurable rectangles. Note that $\mathcal {R}$ forms a π-system, i.e. it is nonempty and closed under conjunction. Let $\mathcal {A}$ denote the collection

  $$\{A\in\mathcal{F}: Q(A,.)\ \text{is}\ \mathcal{G}\text{ - measurable}\}.$$

  We note that $\mathcal {A}$ is a λ-system, i.e. it contains Ω and is closed under complementation as well as countable disjoint union.³ To prove measurability, it suffices to show that ${\mathcal{F}}\subset \mathcal {A}$. Since ${\mathcal{F}}=\sigma \mathcal {R}$, by the monotone class theorem, ${\mathcal{F}}\subset \mathcal {A}$ if $\mathcal {R}\subset \mathcal {A}$. Let $A\cap B\in \mathcal {R}$. Then Q(A ∩ B,.) = Q₁(A,.)Q₂(B,.). Since both Q₁(A,.) and Q₂(B,.) are ${\mathcal{F}}_{2}$-measurable, their product is ${\mathcal{G}}$-measurable. Thus $\mathcal {R}\subset \mathcal {A}$.

Next, suppose ${\mathcal{G}}={\mathcal{F}}_{2}$. We show that Q is a prcd. That is, Q satisfies in addition the requirement of propriety and the generalized law of total probability.

• (generalized law of total probability) We adopt the same strategy as above: let

  $$ \mathcal{C}=\{A\in\mathcal{F}: P(C\cap B)={\int}_{B}Q(C,\omega)dP\ \text{for any }B\in\mathcal{G}\}. $$

  Then $\mathcal {C}$ is a λ-system.⁴ So it remains to check that $\mathcal {R}\subset \mathcal {D}$. This is trivial: let $A=A_{1}\cap A_{2}\in \mathcal {R}$, and $B\in {\mathcal{G}}$, then ${\int \limits }_{B}Q(A,\omega )dP={\int \limits }_{B}Q_{1}(A_{1},\omega )Q_{2}(A_{2},\omega )dP={\int \limits }_{B\cap A_{2}}Q_{1}(A_{1},\omega )dP=P(A_{1}\cap (B\cap A_{2}) )=P((A_{1}\cap A_{2})\cap B)=P(A\cap B)$.

• (propriety) Let $B\in {\mathcal{F}}_{2}$. Then Q(Ω₁ ∩ B, ω) = Q₁(Ω₁, ω)Q₂(B, ω) = δ_ω(B) = 1_B(ω), as desired.

□

Proof

(Theorem 7) Let $E^{P}: {\varTheta }({\mathcal{G}}) \rightarrow {\varDelta }({\mathcal{F}})$ be a uniform extension with ${\mathcal{G}} = \sigma Y$ and $K^{P}: {\mathcal{F}} \rightarrow {\mathcal{L}}^{1}({\mathcal{G}})$ the associated schema (without loss of generality we will omit the second argument place). By conservativeness, K^P(F) is a Radon-Nikodym derivative of P(F ∩ .) with respect to p(.). It suffices to show there is a function $g: {\mathcal{F}} \times \mathbb {R}^{n} \rightarrow \mathbb {R}$ such that,

$$ K^{P}(F)(\omega) = \left\{\begin{array}{ll} \frac{f_{Y}(F, Y(\omega))}{f_{Y}(Y(\omega))} &\text{if } \omega \in S\\ g(F,Y(\omega)) & \text{if } \omega \in S^{c} \end{array}\right. $$

for all $F \in {\mathcal{F}}$. We first observe that since K^P(F) is ${\mathcal{G}}$-measurable, if $Y(\omega ) = Y(\omega ^{\prime })$ then $K^{P}(F)(\omega )=K^{P}(F)(\omega ^{\prime })$. For each $y \in \mathbb {R}^{n}$, let ω_y be a representative element in Y^− 1(y). Define $f_{Y}: {\mathcal{F}} \times \mathbb {R}^{n} \rightarrow \mathbb {R}$ by

$$ f_{Y}(F,y)=K^{P}(F)(\omega_{y}) \cdot f_{Y}(y). $$

We claim that f_Y so defined is in fact a joint pdf for ${\mathcal{F}}$ and Y. Let $C \subset \mathbb {R}^{n}$ Borel. Then,

$$ \begin{array}{@{}rcl@{}} {\int}_{C} f_{Y}(F, y) dy &=& {\int}_{C} K^{P}(F)(\omega_{y}) \cdot f_{Y}(y) dy\\ &=& {\int}_{C} K^{P}(F)(\omega_{y}) dp_{Y}(y)\\ &=& {\int}_{Y \in C} K^{P}(F)(\omega) dP(\omega)\\ &=& P(F \cap (Y \in C)), \end{array} $$

as desired, where we use the fact f_Y(y)dy = dp_Y(y). We then define

$$ g(F, y) = K^{P}(F)(\omega_{y}) \cdot 1_{\{f_{Y}(y)=0\}}. $$

It is easy to check that

$$ K^{P}(F)(\omega) = \left\{\begin{array}{ll} \frac{K^{P}(F)(\omega_{Y})\cdot f_{Y}(y)}{f_{Y}(y)} = \frac{f_{Y}(F, y)}{f_{Y}(y)} & \text{if }\omega \in S\\ \frac{K^{P}(F)(\omega_{y})\cdot 1_{\{f_{Y}(y)=0\}}}{1_{\{f_{Y}(y)=0\}}}=g(F,y) &\text{if } \omega \in S^{c} \end{array}\right. $$

as desired.

For the second claim, note by the Lebesgue differentiation theorem there is a set $T \subset \mathbb {R}^{n}$ with full measure under p_Y such that for all b ∈ T, we have

$$ \lim_{\epsilon \rightarrow 0} \frac{1}{|B_{\epsilon} (b)|} {\int}_{B_{\epsilon}(b)} f_{Y}(F, y) dy = f_{Y}(F, b), $$

and

$$ \lim_{\epsilon \rightarrow 0} \frac{1}{|B_{\epsilon} (b)|} {\int}_{B_{\epsilon}(b)} f_{Y}(y) dy = f_{Y}(b). $$

Since the intersection of two measure-one sets is still measure 1, we have P(S ∩ Y^− 1(T)) = 1 and for every $b \in \mathbb {R}^{n}$ such that Y^− 1(b) ⊂ S ∩ Y^− 1(T) we have,

$$ \begin{array}{@{}rcl@{}} \frac{f_{Y}(F,b)}{f_{Y}(b)} &=& \frac{ \lim_{\epsilon \rightarrow 0} \frac{1}{|B_{\epsilon}(b)|} {\int}_{B_{\epsilon}(b)} f_{Y}(F,y) dy}{\lim_{\epsilon \rightarrow 0} \frac{1}{|B_{\epsilon}(b)|} {\int}_{B_{\epsilon}(b)} f_{Y}(y) dy}\\ &=& \lim_{\epsilon \rightarrow 0} \frac{{\int}_{B_{\epsilon}(b)}f_{Y}(F,y) dy}{{\int}_{B_{\epsilon}(b)} f_{Y}(y) dy}\\ &=& \lim_{\epsilon \rightarrow 0} \frac{P(F \cap Y \in B_{\epsilon}(b))}{P(Y \in B_{\epsilon}(b))}, \end{array} $$

as claimed, where we use the fact that the denominator limit is non-zero. □