Analysis of Decimation on Finite Frames with Sigma-Delta Quantization

Abstract

In analog-to-digital conversion, signal decimation has been proven to greatly improve the efficiency of data storage while maintaining high accuracy. When one couples signal decimation with the $\Sigma \Delta$ quantization scheme, the reconstruction error decays exponentially with respect to the bit-rate. In this study, similar results are proved for finite unitarily generated frames. We introduce a process called alternative decimation on finite frames that is compatible with first- and second-order $\Sigma \Delta$ quantization. In both cases, alternative decimation results in exponential error decay with respect to the bit usage.

Appendices

Appendix A: Limitation of Alternative Decimation: Third-Order Decimation

The non-commutativity between ${\bar{\Delta }}_\rho$ and $\Delta ^{-1}$ results in incomplete difference scaling when applying $D_\rho S_\rho ^r$ on $\Delta ^r$, creating substantial error terms. This phenomenon already occurs for $r=3$.

Proposition A.1

Given $m,\rho \in {{\mathbb {N}}}$ with $\rho \mid m$, the third-order decimation satisfies $D_\rho S_\rho ^3\Delta ^3=\frac{1}{\rho ^3}(\Delta ^{(\eta )})^3 D_\rho +O(\rho ^{-2})$. In particular, $D_\rho S_\rho ^3$ only yields quadratic error decay with respect to the oversampling ratio $\rho$.

First, by noting that $\Delta ^{-1}{\bar{\Delta }}_\rho \Delta ={\mathcal {E}}$ as in Lemma 7.4, one has

$$\begin{aligned} \begin{aligned}&D_\rho S_\rho ^3\Delta ^3\\&\quad =\frac{1}{\rho ^3}D_\rho {\bar{\Delta }}_\rho \Delta ^{-1}{\bar{\Delta }}_\rho \Delta ^{-1}{\bar{\Delta }}_\rho \Delta ^2\\&\quad =\frac{1}{\rho ^3}D_\rho {\bar{\Delta }}_\rho (\Delta ^{-1}{\bar{\Delta }}_\rho \Delta )\Delta ^{-2}{\bar{\Delta }}_\rho \Delta ^2\\&\quad =\frac{1}{\rho ^3}D_\rho {\bar{\Delta }}_\rho ({\bar{\Delta }}_\rho +{\mathcal {E}})\Delta ^{-1}({\bar{\Delta }}_\rho +{\mathcal {E}})\Delta \\&\quad =\frac{1}{\rho ^3}D_\rho {\bar{\Delta }}_\rho ({\bar{\Delta }}_\rho +{\mathcal {E}})({\bar{\Delta }}_\rho +{\mathcal {E}}+\Delta ^{-1}{\mathcal {E}}\Delta )\\&\quad =\frac{1}{\rho ^3}D_\rho \bigg ({\bar{\Delta }}_\rho ^3+{\bar{\Delta }}_\rho ^2{\mathcal {E}}+{\bar{\Delta }}_\rho ^2(\Delta ^{-1}{\mathcal {E}}\Delta )+{\bar{\Delta }}_\rho {\mathcal {E}}{\bar{\Delta }}_\rho +{\bar{\Delta }}_\rho {\mathcal {E}}^2+{\bar{\Delta }}_\rho {\mathcal {E}}(\Delta ^{-1}{\mathcal {E}}\Delta )\bigg ). \end{aligned} \end{aligned}$$

(15)

We shall calculate all terms one by one.

Lemma A.2

We have the following equalities:

• (1)
  $$\begin{aligned} (D_\rho {\bar{\Delta }}_\rho ^2{\mathcal {E}})_{l,s}=\delta (s-(m-\rho ))\bigg (\delta (l-1)-\delta (l-2)\bigg ), \end{aligned}$$
• (2)
  $$\begin{aligned} (D_\rho {\bar{\Delta }}_\rho ^2(\Delta ^{-1}{\mathcal {E}}\Delta ))_{l,s}=\left\{ \begin{array}{lll} -\rho &{}\quad \text {if}&{}\quad (l,s)=(1,m-\rho -1),\\ \rho &{}\quad \text {if}&{}\quad (l,s)=(1,m-\rho ),\\ 0&{}\quad \text {otherwise},\end{array}\right. \end{aligned}$$
• (3)
  $$\begin{aligned} (D_\rho {\bar{\Delta }}_\rho {\mathcal {E}}{\bar{\Delta }}_\rho )_{l,s}=\delta (l-1)\bigg (\delta (s-(m-\rho ))-\delta (s-(m-2\rho ))\bigg ), \end{aligned}$$
• (4)
  $$\begin{aligned} (D_\rho {\bar{\Delta }}_\rho {\mathcal {E}}^2)_{l,s}=\delta (l-1)\delta (s-(m-\rho )), \end{aligned}$$
• (5)
  $$\begin{aligned}&(D_\rho {\bar{\Delta }}_\rho {\mathcal {E}}(\Delta ^{-1}{\mathcal {E}}\Delta ))_{l,s}\\&\quad =(m-\rho )\delta (l-1)\bigg (\delta (s-(m-\rho ))-\delta (s-(m-\rho -1)\bigg ), \end{aligned}$$

where given $n\in {{\mathbb {N}}}$, $[n]:=\{1,\dots , n\}$. In particular, $D_\rho \big ({\bar{\Delta }}_\rho ^2(\Delta ^{-1}{\mathcal {E}}\Delta )+{\bar{\Delta }}_\rho {\mathcal {E}}(\Delta ^{-1}{\mathcal {E}}\Delta )\big )=O(m)$, and $D_\rho ({\bar{\Delta }}_\rho ^2{\mathcal {E}}+{\bar{\Delta }}_\rho {\mathcal {E}}{\bar{\Delta }}_\rho +{\bar{\Delta }}_\rho {\mathcal {E}}^2)=O(1)$.

Proof

We will first compute each term without the effect of $D_\rho$ since $D_\rho$ is the sub-sampling matrix retaining only the $t\rho$th rows for $t\in [\eta ]$.

• (1), (3) First, note that $({\bar{\Delta }}_\rho {\mathcal {E}})_{l,s}=\delta (l-\rho )\delta (s+\rho )$, so

  $$\begin{aligned} ({\bar{\Delta }}_\rho ^2{\mathcal {E}})_{l,s}=\delta (s+\rho )({\bar{\Delta }}_\rho )_{l,\rho }=\delta (s-(m-\rho ))(\delta (l-\rho )-\delta (l-2\rho )). \end{aligned}$$

  Similarly,

  $$\begin{aligned}&({\bar{\Delta }}_\rho {\mathcal {E}}{\bar{\Delta }}_\rho )_{l,s}\\&\quad =\delta (l-\rho )({\bar{\Delta }}_\rho )_{m-\rho ,s}=\delta (l-\rho )(\delta (s-(m-\rho ))-\delta (s-(m-2\rho ))). \end{aligned}$$

• (5) Now, to compute $\Delta ^{-1}{\mathcal {E}}\Delta$, we see that, for $s\ne m$,

  $$\begin{aligned} (\Delta ^{-1}{\mathcal {E}}\Delta )_{l,s}=\sum _{j=1}^l({\mathcal {E}}_{j,s}-{\mathcal {E}}_{j,s+1})=l(\delta (m-\rho -s)-\delta (m-\rho -(s+1))), \end{aligned}$$

  and $(\Delta ^{-1}{\mathcal {E}}\Delta )_{l,m}=0$. In particular,

  $$\begin{aligned} \Delta ^{-1}{\mathcal {E}}\Delta =\begin{pmatrix} 0&{}\quad \ldots &{}\quad 0&{}\quad -\,1&{}\quad 1&{}\quad 0&{}\quad \ldots &{}\quad 0\\ \vdots &{}\quad &{}\quad \vdots &{}\quad -\,2&{}\quad 2&{}\quad \vdots &{}\quad &{}\quad \vdots \\ \vdots &{}\quad &{}\quad \vdots &{}\quad \vdots &{}\quad \vdots &{}\quad \vdots &{}\quad &{}\quad \vdots \\ 0&{}\quad \ldots &{}\quad 0&{}\quad -\,m&{}\quad m&{}\quad 0&{}\quad \ldots &{}\quad 0 \end{pmatrix}, \end{aligned}$$

  where the nonzero columns occur at the $(m-\rho -1)$ and $(m-\rho )$th positions.

  For ${\bar{\Delta }}_\rho {\mathcal {E}}(\Delta ^{-1}{\mathcal {E}}\Delta )$,

  $$\begin{aligned} \begin{aligned} ({\bar{\Delta }}_\rho {\mathcal {E}}(\Delta ^{-1}{\mathcal {E}}\Delta ))_{l,s}&=\delta (l-\rho )(\Delta ^{-1}{\mathcal {E}}\Delta )_{m-\rho ,s}\\&=\delta (l-\rho )(m-\rho )(\delta (s-(m-\rho ))-\delta (s-(m-\rho -1))). \end{aligned} \end{aligned}$$

• (4) Note that ${\bar{\Delta }}_\rho {\mathcal {E}}^2={\bar{\Delta }}_\rho {\mathcal {E}}$. The result then follows from the calculation on the first term.

• (2) Finally, as $\Delta ^{-1}{\mathcal {E}}\Delta$ only has nonzero entries on the $(m-\rho -1)$ and $(m-\rho )$th columns, and the two columns differ by a sign, it suffices to calculate the $(m-\rho )$th column of ${\bar{\Delta }}_\rho ^2(\Delta ^{-1}{\mathcal {E}}\Delta )$.

  $$\begin{aligned} \begin{aligned} ({\bar{\Delta }}_\rho (\Delta ^{-1}{\mathcal {E}}\Delta ))_{l,m-\rho }&=\sum _{j=1}^mj({\bar{\Delta }}_\rho )_{l,j}\\&=\left\{ \begin{array}{lll} l-(l-\rho )=\rho &{}\quad \text {if}&{}\quad l>\rho ,\\ l-(l-\rho +m)=-(m-\rho )&{}\quad \text {if}&{}\quad l<\rho ,\\ l=\rho &{}\quad \text {if}&{}\quad l=\rho . \end{array}\right. \end{aligned} \end{aligned}$$

  Then,

  $$\begin{aligned} \begin{aligned} ({\bar{\Delta }}_\rho ^2(\Delta ^{-1}{\mathcal {E}}\Delta ))_{l,m-\rho }&=\sum _{j=1}^m({\bar{\Delta }}_\rho )_{l,j}({\bar{\Delta }}_\rho (\Delta ^{-1}{\mathcal {E}}\Delta ))_{j,m-\rho }\\&=\left\{ \begin{array}{lll} -m&{}\quad \text {if}&{}\quad l\in [2\rho -1]\backslash \{\rho \},\\ \rho &{}\quad \text {if}&{}\quad l=\rho ,\\ 0&{}\quad \text {otherwise}. \end{array}\right. \end{aligned} \end{aligned}$$

$\square$

Proof of Proposition A.1

From (15) and Lemma A.2, we see that

$$\begin{aligned} D_\rho S_\rho ^3\Delta ^3=\frac{1}{\rho ^3}D_\rho {\bar{\Delta }}_\rho ^3+\frac{\eta }{\rho ^2}{\mathcal {E}}_1+\frac{1}{\rho ^3}{\mathcal {E}}_2=\frac{1}{\rho ^3}D_\rho {\bar{\Delta }}_\rho ^3+O(\rho ^{-2}), \end{aligned}$$

where

$$\begin{aligned} ({\mathcal {E}}_1)_{l,s}= & {} \frac{1}{m}\bigg (D_\rho ({\bar{\Delta }}_\rho ^2(\Delta ^{-1}{\mathcal {E}}\Delta )+{\bar{\Delta }}_\rho {\mathcal {E}}(\Delta ^{-1}{\mathcal {E}}\Delta ))\bigg )_{l,s}\\= & {} \left\{ \begin{array}{lll} -\,1&{}\quad \text {if}&{}\quad (l,s)=(1,m-\rho -1),\\ 1&{}\quad \text {if}&{}\quad (l,s)=(1,m-\rho ),\\ 0&{}\quad \text {otherwise,} \end{array}\right. \end{aligned}$$

and

$$\begin{aligned} ({\mathcal {E}}_2)_{l,s}= & {} \bigg (D_\rho ({\bar{\Delta }}_\rho ^2{\mathcal {E}}+{\bar{\Delta }}_\rho {\mathcal {E}}{\bar{\Delta }}_\rho +{\bar{\Delta }}_\rho {\mathcal {E}}^2)\bigg )_{l,s}\\= & {} \left\{ \begin{array}{lll} -1&{}\quad \text {if} &{}\quad (l,s)=(2,m-\rho )\,\text { or }\, (1,m-2\rho ),\\ 3&{}\quad \text {if} &{}\quad (l,s)=(1,m-\rho ),\\ 0&{}\quad \text {otherwise}. \end{array}\right. \end{aligned}$$

$\square$

Even in higher-order cases, alternative decimation still only yields quadratic error decay with respect to the oversampling ratio, as can be seen in Fig. 2d, e.

Alternative decimation is limited by this incomplete cancellation, but canonical decimation has even worse error decay. Contrary to the quadratic decay for alternative decimation, canonical decimation only has linear decay for high-order $\Sigma \Delta$ quantization. The same thing applies to plain $\Sigma \Delta$ quantization, as can be seen in Fig. 2b.

Appendix B: Numerical Experiments

Here, we present numerical evidence that the alternative decimation on frames has linear and quadratic error decay rate for the first and the second order, respectively. Moreover, it is shown that the canonical decimation, as described in Remark 3.2, is not suitable for our purpose when $r\ge 2$.

Recall that given $m,r,\rho$, one can define the canonical decimation operator $D_\rho {\tilde{S}}_\rho ^r\in {{\mathbb {R}}}^{\eta \times m}$, where ${\tilde{S}}_\rho \in {{\mathbb {R}}}^{m\times m}$ is a circulant matrix.

1.1 B.1. Setting

In our experiment, we look at three different quantization schemes: alternative decimation, canonical decimation, and plain $\Sigma \Delta$. Given observed data $y\in {{\mathbb {C}}}^m$ from a frame $E\in {{\mathbb {C}}}^{m\times k}$ and $r\in {{\mathbb {N}}}$, one can determine the quantized samples $q\in {{\mathbb {C}}}^m$ by

$$\begin{aligned} y-q=\Delta ^r u \end{aligned}$$

for some bounded u. The three schemes differ in the choice of dual frames:

• Alternative decimation: ${\tilde{x}}=(D_\rho S_\rho ^r E)^\dagger D_\rho S_\rho ^r q=F_a q$.

• Canonical decimation: ${\tilde{x}}=(D_\rho {\tilde{S}}_\rho ^r E)^\dagger D_\rho {\tilde{S}}_\rho ^r q=F_c q$.

• Plain $\Sigma \Delta$: ${\tilde{x}}=E^\dagger q=F_p q$.

For each experiment, we use the mid-rise quantizer ${\mathscr {A}}$ and fix $k=55, \delta =0.5, L=100$, and $\eta =65$. For each $\rho$, we set $m=\rho \eta$ and pick 10 randomly generated vectors $\{x^j\}_{j=1}^{10}\subset {{\mathbb {C}}}^k$. $\Sigma \Delta$ quantization on each signal gives $\{q^j\}_{j=1}^{10}\subset {{\mathbb {C}}}^m$. The maximum reconstruction error over the 10 experiments is recorded, namely

$$\begin{aligned} {\mathscr {E}}_{i}=\max _{1\le j\le 10}\Vert x^j-F_i q^j\Vert _2,\quad i\in \{a,c,p\}. \end{aligned}$$

The frame in our experiment is

$$\begin{aligned} (E^{m,k})_{l,j}=(E)_{l,j}=\frac{1}{\sqrt{k}}(\exp (-2\pi \imath (l+1)(j+1)/m))_{l,j}. \end{aligned}$$

First, we shall compare alternative decimation with plain $\Sigma \Delta$ quantization from Fig. 2. For $r=1$, alternative decimation performs worse than plain $\Sigma \Delta$ quantization, as plain $\Sigma \Delta$ quantization benefits from the smoothness of the frame elements, having decay rate $O((\frac{m}{k})^{-5/4})$ proved in [3]. However, for $r\ge 2$, alternative decimation supersedes plain $\Sigma \Delta$ quantization as the better scheme. This can be explained by the boundary effect in finite-dimensional spaces that results in incomplete cancellation for backward difference matrices. We are interested in the case $r=1$ or 2. As we can see, the theoretical error bound does not have a tight constant, although the decay rate is consistent with our experimental result.

Fig. 2

The log-log plot for reconstruction error against the decimation ratio $\rho$ for different quantization schemes. In the case $r=1$, alternative decimation coincides with canonical decimation. For $r\ge 2$, alternative decimation has better error decay rate than both canonical decimation and plain $\Sigma \Delta$ quantization

1.2 B.2. Necessity of Alternative Decimation

The main difference between the alternative decimation operator $D_\rho S_\rho ^r$ and the canonical one $D_\rho {\tilde{S}}_\rho ^r$ lies in the scaling effect on difference structures. We have ${\tilde{S}}_\rho ^r=(S_\rho +L)^r$ with $\rho L$ having unit entries on the first $\rho -1$ rows and 0 everywhere else.

In Fig. 2, we can see the performance drop-off when switching from alternative decimation to canonical decimation for $r\ge 2$. We can see that canonical decimation incurs much worse reconstruction error than the alternative one, while generally having worse decay rate. For demonstration, we show explicitly the difference between alternative and canonical decimation schemes for $r=2$:

$$\begin{aligned} \begin{aligned} {\tilde{S}}_\rho ^2\Delta ^2&=(S_\rho +L)^2\Delta ^2\\&=S_\rho ^2\Delta ^2+(LS_\rho +S_\rho L+L^2)\Delta ^2\\&=S_\rho ^2\Delta ^2+L(S_\rho +L^2)\Delta ^2+S_\rho L\Delta ^2. \end{aligned} \end{aligned}$$

Since $D_\rho L=0$, we are left with $D_\rho S_\rho L\Delta ^2$. Now,

$$\begin{aligned} (L\Delta ^2)_{l,j}=\left\{ \begin{array}{ll} \frac{-1}{\rho }&{}\quad \text {if}\quad 1\le l\le \rho -1,\, j=m-1,\\ \frac{1}{\rho }&{}\quad \text {if}\quad 1\le l\le \rho -1,\, j=m,\\ 0&{}\quad \text {otherwise}. \end{array}\right. \end{aligned}$$

Then, we see that

$$\begin{aligned} (DS_\rho L\Delta ^2)_{l,j}=\left\{ \begin{array}{ll} \frac{-(\rho -1)}{\rho ^2}&{}\quad \text {if}\quad l=1,\, j=m-1,\\ \frac{\rho -1}{\rho ^2}&{}\quad \text {if}\quad l=1,\, j=m,\\ 0&{}\quad \text {otherwise}. \end{array}\right. \end{aligned}$$

We see that $D_\rho {\tilde{S}}_\rho ^2\Delta ^2=O(\rho ^{-1})$, hence the linear decay for $r=2$.