Modern information theory is largely developed in connection with random elements residing in large, complex, and discrete data spaces, or alphabets. Lacking natural metrization and hence moments, the associated probability and statistics theory must rely on information measures in the form of various entropies, for example, Shannon’s entropy, mutual information and Kullback–Leibler divergence, which are functions of an entropic basis in the form of a sequence of entropic moments of varying order. The entropicmoments collectively characterize the underlying probability distribution on the alphabet, and hence provide an opportunity to develop statistical procedures for their estimation. As such statistical development becomes an increasingly important line of research in modern data science, the relationship between the underlying distribution and the asymptotic behavior of the entropic moments, as the order increases, becomes a technical issue of fundamental importance. This paper offers a general methodology to capture the relationship between the rates of divergence of the entropic moments and the types of underlying distributions, for a special class of distributions. As an application of the established results, it is demonstrated that the asymptotic normality of the remarkable Turing’s formula for missing probabilities holds under distributions with much thinner tails than those previously known.

中文翻译：

---

可数字母的熵矩和吸引域

现代信息论在很大程度上与驻留在大型，复杂和离散数据空间或字母中的随机元素有关。缺乏自然的度量化和矩，相关的概率和统计理论必须依靠各种熵形式的信息度量，例如，香农的熵，互信息和Kullback-Leibler散度，它们是熵形式的熵基础的函数一系列不同阶的熵矩。熵矩共同表征了字母上潜在的概率分布，因此为开发用于估计它们的统计程序提供了机会。随着这种统计发展成为现代数据科学中越来越重要的研究领域，随着阶次的增加，基本分布与熵矩的渐近行为之间的关系成为具有根本重要性的技术问题。对于特殊的分布类别，本文提供了一种通用的方法来捕获熵矩的发散率与基础分布类型之间的关系。作为已确定结果的一种应用，证明了显着的图灵公式对于丢失概率的渐近正态性在具有比以前已知的尾部细得多的分布的分布下成立。对于特殊的分布类别，本文提供了一种通用的方法来捕获熵矩的发散率与基础分布类型之间的关系。作为已确定结果的一种应用，证明了显着的图灵公式对于丢失概率的渐近正态性在具有比以前已知的尾部细得多的分布的分布下成立。对于特殊的分布类别，本文提供了一种通用的方法来捕获熵矩的发散率与基础分布类型之间的关系。作为已确定结果的一种应用，证明了显着的图灵公式对于丢失概率的渐近正态性在具有比以前已知的尾部细得多的分布的分布下成立。