On linear optimization over Wasserstein balls

Abstract

Wasserstein balls, which contain all probability measures within a pre-specified Wasserstein distance to a reference measure, have recently enjoyed wide popularity in the distributionally robust optimization and machine learning communities to formulate and solve data-driven optimization problems with rigorous statistical guarantees. In this technical note we prove that the Wasserstein ball is weakly compact under mild conditions, and we offer necessary and sufficient conditions for the existence of optimal solutions. We also characterize the sparsity of solutions if the Wasserstein ball is centred at a discrete reference measure. In comparison with the existing literature, which has proved similar results under different conditions, our proofs are self-contained and shorter, yet mathematically rigorous, and our necessary and sufficient conditions for the existence of optimal solutions are easily verifiable in practice.

Appendices

Appendix A: Auxiliary measure-theoretic results

We review some well-known facts from measure theory that we use to prove our results. We first recall a connection between the notions of tightness and weak sequential compactness of collections of probability measures.

Definition 1

A collection ${\mathcal {S}} \subseteq {\mathcal {P}} (X)$ of probability measures is tight if for any $\epsilon >0$, there exists a compact subset $B \subseteq X$ such that $\mu (X \setminus B) \le \epsilon$ for all $\mu \in {\mathcal {S}}$.

Definition 2

A sequence $\{\mu ^k\}_k \subseteq {\mathcal {P}} (X)$ of probability measures converges weakly to $\mu ^\infty \in {\mathcal {P}} (X)$ if for any bounded and continuous function g on X, we have

$$\begin{aligned} \lim _{k\longrightarrow \infty } \int _X g(x) \,\mathrm {d}\mu ^k \;\; = \;\; \int _X g(x) \,\mathrm {d}\mu ^\infty . \end{aligned}$$

Definition 3

A collection ${\mathcal {S}} \subseteq {\mathcal {P}} (X)$ of probability measures is weakly sequentially compact if every sequence in ${\mathcal {S}}$ has a subsequence that converges weakly to an element of ${\mathcal {S}}$.

The concepts of tightness and weak sequential compactness are connected by Prokho-rov’s Theorem, see for example [3, Theorem 5.1].

Theorem 5

(Prokhorov’s Theorem) A collection ${\mathcal {S}} \subseteq {\mathcal {P}} (X)$ of probability measures is tight if and only if the closure of ${\mathcal {S}}$ is weakly sequentially compact in ${\mathcal {P}}(X)$.

Note that the space ${\mathcal {P}}(X)$ is metrizable, sequential compactness and compactness of subsets of ${\mathcal {P}}(X)$ are equivalent to each other.

The following lemma, which is excerpted from the Portmanteau Theorem (see for example [4, Problem 29.1(c)]), provides a useful characterization of weak convergence.

Lemma 3

A sequence $\{\mu ^k \}_k \subseteq {\mathcal {P}} (X)$ of probability measures converges weakly to $\mu ^\infty \in {\mathcal {P}} (X)$ if and only if for any upper bounded and upper semi-continuous function g on X, we have

$$\begin{aligned} \limsup _{k\longrightarrow \infty } \int _X g(x) \,\mathrm {d}\mu ^k (x) \;\; \le \;\; \int _X g(x) \,\mathrm {d}\mu ^\infty (x). \end{aligned}$$

Appendix B: Basic feasible solutions in infinite-dimensional linear programming

It is well-known that if a finite-dimensional linear program with m equality constraints has an optimal solution, then there must be an optimal basic feasible solution with at most m non-zero entries. An infinite-dimensional analogue of this fact is proved in [21, Corollary 5 and Proposition 6(v)]. To state this result, let Z be a topological space, let ${\mathcal {M}}_+ (Z)$ be the set of non-negative finite Borel measures supported on Z, and let $\psi , \phi _1,\dots ,\phi _m : Z \rightarrow {\mathbb {R}}$ be Borel functions as well as $v \in {\mathbb {R}}^m$. Consider now the optimization problem

$$\begin{aligned} \begin{array}{l@{\quad }l@{\quad }l} \displaystyle \mathop {\text {maximize}}_{\gamma } &{} \displaystyle \int _Z \psi (z) \, \mathrm {d}\gamma (z) \\ \displaystyle \text {subject to} &{} \gamma \in {\mathcal {M}}_+ (Z) \\ &{} \displaystyle \int _Z \phi _i (z) \, \mathrm {d}\gamma (z)= v_i &{} \displaystyle \forall i =1,\dots ,m, \end{array} \end{aligned}$$

(11)

and denote by ${\mathcal {F}}$ the feasible region of (11) and by $\mathrm {ext}({\mathcal {F}})$ the set of extreme points of ${\mathcal {F}}$.

Proposition 1

Suppose that for all $\gamma \in {\mathcal {F}}$, at least one of the integrals $\int _X [\psi (z)]_+ \, \mathrm {d} \gamma (z)$ and $\int _X [-\psi (z)]_+ \, \mathrm {d} \gamma (z)$ is finite and that $\int _Z |\phi _i| (z) \, \mathrm {d}\gamma (z) <\infty$ for all $i = 1,\dots ,m$. If

$$\begin{aligned} \sup \left\{ \int _Z \psi (z) \, \mathrm {d}\gamma (z): \gamma \in {\mathcal {F}} \right\} \;\; = \;\; \sup \left\{ \int _Z \psi (z) \, \mathrm {d}\gamma (z): \gamma \in \mathrm {ext}({\mathcal {F}}) \right\} , \end{aligned}$$

(12)

then it holds that

$$\begin{aligned} \sup \left\{ \int _Z \psi (z) \, \mathrm {d}\gamma (z) : \gamma \in {\mathcal {F}} \right\} \;\; = \;\; \sup \left\{ \int _Z \psi (z) \, \mathrm {d}\gamma : \gamma \in {\mathcal {F}}\cap {\mathcal {D}}_m (Z) \right\} , \end{aligned}$$

where ${\mathcal {D}}_m (Z)$ is the set of non-negative discrete measures supported on at most m points in Z. Furthermore, if ${\mathcal {F}}\subseteq {\mathcal {P}}(Z)$ and Z is Hausdorff, then the condition (12) is satisfied.

We note that the conclusion of Proposition 1 cannot readily be drawn from the Richter-Rogosinski theorem [22, Theorem 7.32]. Indeed, in our context the Richter-Rogosinski theorem would only ensure the existence of a non-negative discrete measure $\gamma ^\star$ that is supported on at most $m + 1$ (instead of m) points since $\gamma ^\star$ would have to satisfy $m + 1$ moment conditions: the m moment constraints of problem (11) as well as the additional constraint that $\gamma ^\star$ attains the optimal objective value of problem (11).