Sparse Multi-Reference Alignment: Phase Retrieval, Uniform Uncertainty Principles and the Beltway Problem

Abstract

Motivated by cutting-edge applications like cryo-electron microscopy (cryo-EM), the Multi-Reference Alignment (MRA) model entails the learning of an unknown signal from repeated measurements of its images under the latent action of a group of isometries and additive noise of magnitude $\sigma$. Despite significant interest, a clear picture for understanding rates of estimation in this model has emerged only recently, particularly in the high-noise regime $\sigma \gg 1$ that is highly relevant in applications. Recent investigations have revealed a remarkable asymptotic sample complexity of order $\sigma ^6$ for certain signals whose Fourier transforms have full support, in stark contrast to the traditional $\sigma ^2$ that arise in regular models. Often prohibitively large in practice, these results have prompted the investigation of variations around the MRA model where better sample complexity may be achieved. In this paper, we show that sparse signals exhibit an intermediate $\sigma ^4$ sample complexity even in the classical MRA model. Further, we characterize the dependence of the estimation rate on the support size s as $O_p(1)$ and $O_p(s^{3.5})$ in the dilute and moderate regimes of sparsity respectively. Our techniques have implications for the problem of crystallographic phase retrieval, indicating a certain local uniqueness for the recovery of sparse signals from their power spectrum. Our results explore and exploit connections of the MRA estimation problem with two classical topics in applied mathematics: the beltway problem from combinatorial optimization, and uniform uncertainty principles from harmonic analysis. Our techniques include a certain enhanced form of the probabilistic method, which might be of general interest in its own right.

Appendices

Appendix

Appendix: Additional Notations

Definition 37

Let $\{X_n\}_{n \ge 1}$ be a sequence of non-negative random variables, and $\{a_n\}_{n \ge 1}$ is a sequence of positive numbers (deterministic or random). Then:

• By the statement $X_n=O_p(a_n)$ we mean that, for every $\varepsilon >0$, there exists $0<C(\varepsilon )<\infty$ such that

  $$\begin{aligned} \liminf _{n \rightarrow \infty }\mathbb {P}\left[ X_n/a_n \le C(\varepsilon ) \right] \ge 1-\varepsilon . \end{aligned}$$

• By the statement $X_n=\Omega _p(a_n)$ we mean that, for every $\varepsilon >0$, there exists $0<c(\varepsilon )<\infty$ such that

  $$\begin{aligned} \liminf _{n \rightarrow \infty }\mathbb {P}\left[ X_n/a_n \ge c(\varepsilon ) \right] \ge 1-\varepsilon . \end{aligned}$$

• By the statement $X_n=\Theta _p(a_n)$ we mean that for every $\varepsilon >0$, there exist $0<c(\varepsilon )<C(\varepsilon )<\infty$ such that

  $$\begin{aligned} \liminf _{n \rightarrow \infty }\mathbb {P}\left[ c(\varepsilon ) \le X_n/a_n \le C(\varepsilon ) \right] \ge 1-\varepsilon . \end{aligned}$$

Further, $\Vert \cdot \Vert _F$ will denote the Frobenius norm of a matrix, and the expectation ${\mathbb {E}}_G$ will be taken with respect to G chosen uniformly from the group of isometries ${\mathcal {G}}$.

For any positive integer m, by the symbol [m] we denote the set $\{1,\ldots ,m\}$.

For two sequences of positive numbers $(a_k)_{k>0}$ and $(b_k)_{k>0}$, we write $a_k \ll b_k$ when we have $b_k/a_k \rightarrow \infty$ as $k \rightarrow \infty$.

A sequence of events $\{E_m\}_{m \ge 1}$, defined with respect to probability measures $\mathbb {P}_m$, is said to occur with high probability if $\mathbb {P}_m[E_m] \rightarrow 1$ as $m \rightarrow \infty$.

For any $\theta = (\theta _1,\ldots ,\theta _L) \in {\mathbb {R}}^L$, we denote ${\overline{\theta }}=\frac{1}{L}\sum _{i=1}^L \theta _i$.

Appendix: Bernoulli–Gaussian Distributions

We define the notion of the Bernoulli–Gaussian distribution, and the symmetric version thereof. For that, we first define the notion of a Gaussian distribution indexed by a subset of ${\mathbb {Z}}_L$.

Definition 38

(Subset-indexed Gaussian distributions) Let $A \subset {\mathbb {Z}}_L$, $\mu :{\mathbb {Z}}_L \rightarrow {\mathbb {R}}$ a function supported on A and $\Sigma$ be a positive definite $|A|\times |A|$ matrix. Then the Gaussian distribution indexed by A with mean $\mu$ and covariance $\Sigma$, denoted $N_A(0,\Sigma )$, is the random vector $(\eta _k)_{k \in {\mathbb {Z}}_L}$, with $\eta _k=0$ for $k \in A^\complement$, and $(\eta _k)_{k \in A}$ is the |A|-dimensional Gaussian random vector with mean $\mu$ and covariance $\Sigma$.

This allows us to define the Bernoulli–Gaussian distribution, a key property of which is that the support is chosen at random according to a Bernoulli sampling scheme.

Definition 39

(Bernoulli–Gaussian distribution) Let $s \in [L]$ and $\Xi \subset {\mathbb {Z}}_L$ be a random subset obtained by selecting each member of ${\mathbb {Z}}_L$ independently with probability s/L. The Bernoulli–Gaussian distribution on $Z_L$ with variance $\zeta ^2$ and sparsity s is then defined as the Gaussian distribution indexed by $\Xi$ with mean ${\mathbf {0}}$ and covariance $\zeta ^2 I$; in other words the random variable $N_\Xi ({\mathbf {0}},\zeta ^2 I)$, with the Gaussian entries being statistically independent of the support $\Xi$.

Next, we introduce the concept of a standard symmetric Gaussian random variable indexed by a subset of ${\mathbb {Z}}_L$. To introduce the notion of a symmetric signal, we first recall the notion of the standard parametrization of ${\mathbb {Z}}_L$ (1.7).

We are now ready to define

Definition 40

(Symmetric subset-indexed Gaussian distributions) Let ${\mathbb {Z}}_L$ be in the standard enumeration (1.7), let $A \subset {\mathbb {Z}}_L$ be symmetric, i.e. $A = -A$ and let $\rho >0$. Let $A_+:=\{0,\ldots ,\lfloor (L-1)/2 \rfloor \} \cap A$, and let $(X_k)_{k \in {\mathbb {Z}}_L}$ denote the random variable $N_{A_+}(0,\zeta ^2 I)$. Then the symmetric Gaussian distribution indexed by A with mean 0 and variance $\zeta ^2$, denoted $N_A^{\mathrm {symm}}(0,\zeta ^2 I)$, is the random vector $(\eta _k)_{k \in {\mathbb {Z}}_L}$ with $\eta _k=X_{|k|}$.

Finally, all of the above taken together allows us to define

Definition 41

(Symmetric Bernoulli–Gaussian distribution) Let ${\mathbb {Z}}_L$ be in the standard enumeration (1.7). Let $\Xi _0 \subset {\mathbb {Z}}_L^+=\{0,\ldots ,\lfloor (L-1)/2\rfloor \}$ be a random subset obtained by selecting each member of ${\mathbb {Z}}_L^+$ independently with probability s/L, and consider the symmetric subset $\Xi :=\Xi _0 \cup (-\Xi _0)$. Then the symmetric Bernoulli–Gaussian distribution with mean zero, variance $\zeta ^2$ and sparsity parameter s is the distribution $N^{\mathrm {symm}}_\Xi (0,\zeta ^2 I)$, with the Gaussian entries being statistically independent of the support $\Xi$.

Heuristically, the symmetric Bernoulli–Gaussian distribution is obtained by taking a Bernoulli–Gaussian random variable on the positive part of ${\mathbb {Z}}_L$ and extending it to all of ${\mathbb {Z}}_L$ by making it symmetric about the origin.

Appendix: Generic Sparse Signals

We introduce the notions of signal support sets that are typically s-sparse and $\Gamma$-cosine generic.

Definition 42

Let $\alpha ,\beta >0$ be fixed numbers and $s \in [L]$ be a parameter that possibly depends on L. A probability distribution over subsets $\Xi \subset {\mathbb {Z}}_L$ is said to be typically s-sparse with sparsity constants $(\alpha ,\beta )$ if we have $\alpha \cdot s \le |\Xi | \le \beta \cdot s$ with probability $1-o_L(1)$.

To introduce the concept of cosine-genericity of a set, we first define the cosine functional of a set $\Xi \subset {\mathbb {Z}}_L$ for an element $a \in {\mathbb {Z}}_L$.

For $\Xi \subset {\mathbb {Z}}_L$ and $a \in {\mathbb {Z}}_L$, define

$$\begin{aligned} {\mathcal {V}}(\Xi ,a)=\mathbbm {1}_{\{0 \in \Xi \}} + 2 \sum _{k \in \Xi \setminus \{0\}} \cos ^2(2\pi a k/L), \end{aligned}$$

(C.1)

where $\mathbbm {1}_A$ denotes the indicator function of the event A.

Then we are ready to introduce

Definition 43

Let $\Gamma >0$ be a parameter, possibly depending on L. A probability distribution over subsets $\Xi \subset {\mathbb {Z}}_L$ is said to be $\Gamma$-cosine generic if, with probability $1-o_L(1)$, we have $\min _{a \in {\mathbb {Z}}_L} {\mathcal {V}}(\Xi ,a) \ge \Gamma (1-o_L(1))$.

Equivalently, we say that the random variable $\Xi$ is cosine generic with parameter $\Gamma$. Cosine genericity of a (random) set is a condition that aims to ensure that, with high probability, the set under consideration is sufficiently generic, in the sense that there are no specialized algebraic or arithmetic relations satisfied by the elements of the set which would make $\min _{a \in {\mathbb {Z}}_L} {\mathcal {V}}(\Xi ,a)$ small.

Putting all of the above together, we may introduce the generic s-sparse symmetric signals.

Definition 44

Let $s \in [L]$ be a parameter, possibly depending on L, and $\alpha ,\beta ,\zeta ,\tau >0$ be fixed. We call a random signal $\theta :{\mathbb {Z}}_L \rightarrow {\mathbb {R}}$ to be a generic s-sparse symmetric signal with dispersion $\zeta ^2$, sparsity constants $\alpha ,\beta$ and index $\tau$ if the following hold:

• The support $\Xi$ of $\theta$ is typically s-sparse with sparsity constants $(\alpha ,\beta )$and $s^\tau$-cosine generic.

• $\theta \sim N_{\Xi }^{\mathrm {symm}}(0,\zeta ^2 I)$, with the non-zero entries of $\theta$ being statistically independent of $\Xi$.

Appendix: On the Size of Collision Free Sets

In this section, we provide detailed arguments for the assertions that the size of a collision-free subset $A \subset {\mathbb {Z}}_L$ is maximally $O(L^{1/2})$ and typically $O(L^{1/3})$.

To this end, we let $1 \le k \le L$, and we consider a subset $B \subset {\mathbb {Z}}_L$ of size $|B|=k$. If B is collision-free, then B entails $k(k-1)$ distinct differences between its points; we call this set of differences D. For $x \in {\mathbb {Z}}_L \setminus B$, we want to understand size restrictions on |B| that enable $B \cup \{x\}$ to be a collision-free set. If $B \cup \{x\}$ has to be collision-free, we note that for any fixed $u \in B$, the difference $x-u$ needs to be $\notin D$. This rules out $k(k-1)$ choices for x. Thus, such a point x can be found only if $k(k-1) < L - k$, which gives us an upper bound of $k=O(L^{1/2})$, as desired.

We note in passing that the probability of a randomly selected x in the above setting to yield a collision-free subset $B \cup \{x\}$ is bounded above by $(L-k-k(k-1))/L$, for any set B.

Now we examine the largest value of m for which a random subset drawn of size m drawn from ${\mathbb {Z}}_L$ collision free with positive probability. For concreteness, we consider m samples without replacement from ${\mathbb {Z}}_L$.

For $1\le k \le m$, we denote by ${\mathfrak {S}}_k$ the set of first k random samples without replacement. Then we may write

$$\begin{aligned}&\mathbb {P}[{\mathfrak {S}}_m ~\text {is collision-free}] \\&\quad = \mathbb {P}[{\mathfrak {S}}_m ~\text {is collision-free} ~| ~{\mathfrak {S}}_{m-1} ~\text {is collision-free}] \cdot \mathbb {P}[{\mathfrak {S}}_{m-1} ~\text {is collision-free}] \\&\quad = \prod _{k=1}^{m-1} \mathbb {P}[{\mathfrak {S}}_{k+1} ~\text {is collision-free} ~| ~{\mathfrak {S}}_{k} ~\text {is collision-free}] \\&\quad = \prod _{k=1}^{m-1} \mathbb {P}_{x \sim \text {Unif}({\mathbb {Z}}_L \setminus {\mathfrak {S}}_{k})} \big [{\mathfrak {S}}_{k} \cup \{x\} ~\text {is collision-free} ~| ~{\mathfrak {S}}_{k} ~\text {is collision-free}\big ] \\&\quad \le \prod _{k=1}^{m-1} \frac{L-k-k(k-1)}{L} \quad \text {[using the analysis for the set}\ B\ \text {above]} \\&\quad = \prod _{k=1}^{m-1} \left( 1 - \frac{k^2}{L} \right) ~\le \prod _{k=1}^{m-1} \exp (-\frac{k^2}{L}) ~\le \exp (-c m^3/L). \end{aligned}$$

Thus, if $m^3/L \rightarrow \infty , \mathbb {P}[{\mathfrak {S}}_k ~\text {is collision-free}] \rightarrow 0$. Therefore, for a random subset of size m to be collision-free with positive probability, we must have $m=O(L^{1/3})$, and to have the same property with high probability, we must have $m=o(L^{1/3})$.