Motivated from the fact that universal source coding on countably infinite alphabets ( $\infty $ -alphabets) is not feasible, this work introduces the notion of “almost lossless source coding”. Analog to the weak variable-length source coding problem studied by Han (IEEE Trans. Inf. Theory, vol. 46, no. 4, pp. 1217–1226, Jul. 2000), almost lossless source coding aims at relaxing the lossless block-wise assumption to allow an average per-letter distortion that vanishes asymptotically as the block-length tends to infinity. In this setup, we show on one hand that Shannon entropy characterizes the minimum achievable rate (similarly to the case of finite alphabet sources) while on the other that almost lossless universal source coding becomes feasible for the family of finite-entropy stationary memoryless sources with $\infty $ -alphabets. Furthermore, we study a stronger notion of almost lossless universality that demands uniform convergence of the average per-letter distortion to zero, where we establish a necessary and sufficient condition for the so-called family of “envelope distributions” to achieve it. Remarkably, this condition is the same necessary and sufficient condition needed for the existence of a strongly minimax (lossless) universal source code for the family of envelope distributions. Finally, we show that an almost lossless coding scheme offers faster rate of convergence for the (minimax) redundancy compared to the well-known information radius developed for the lossless case at the expense of tolerating a non-zero distortion that vanishes to zero as the block-length grows. This shows that even when lossless universality is feasible, an almost lossless scheme can offer different regimes on the rates of convergence of the (worst case) redundancy versus the (worst case) distortion.

中文翻译：

---

可数无限字母表上的通用弱变长源编码

受可数无限字母表 ($\infty $ -alphabets) 上的通用源编码不可行这一事实的启发，这项工作引入了“几乎无损源编码”的概念。类似于 Han 研究的弱可变长度源编码问题（IEEE Trans. Inf. Theory, vol. 46, no. 4, pp. 1217–1226, Jul. 2000），几乎无损源编码旨在放松无损块-wise 假设允许平均每个字母的失真随着块长度趋于无穷大而逐渐消失。在这个设置中，我们一方面表明香农熵表征了最小可达到的速率（类似于有限字母源的情况），而另一方面，几乎无损的通用源编码对于有限熵固定无记忆源家族变得可行$\infty $ -字母。此外，我们研究了一个更强的几乎无损普遍性的概念，它要求每个字母的平均失真均匀收敛到零，在那里我们为所谓的“包络分布”族建立一个必要和充分条件来实现它。值得注意的是，此条件与存在包络分布族的强极小极大（无损）通用源代码所需的必要和充分条件相同。最后，我们表明，与为无损情况开发的众所周知的信息半径相比，几乎无损的编码方案为（minimax）冗余提供了更快的收敛速度，但代价是容忍非零失真消失为零，因为块长度增加。这表明即使无损普遍性是可行的，