A novel functional estimator for Rényi's α-entropy and its multivariate extension was recently proposed in terms of the normalized eigenspectrum of a Hermitian matrix of the projected data in a reproducing kernel Hilbert space (RKHS). However, the utility and possible applications of these new estimators are rather new and mostly unknown to practitioners. In this brief, we first show that this estimator enables straightforward measurement of information flow in realistic convolutional neural networks (CNNs) without any approximation. Then, we introduce the partial information decomposition (PID) framework and develop three quantities to analyze the synergy and redundancy in convolutional layer representations. Our results validate two fundamental data processing inequalities and reveal more inner properties concerning CNN training.

中文翻译：

---

用信息论理解卷积神经网络：初步探索。

最近，针对再现核希尔伯特空间（RKHS）中投影数据的埃尔米特矩阵的归一化特征谱，提出了一种针对Rényiα熵及其多元扩展的新型函数估计器。但是，这些新估算器的实用性和可能的​​应用是相当新的，并且对于从业人员几乎是未知的。在本摘要中，我们首先证明了该估计器能够在不进行任何近似的情况下直接测量实际卷积神经网络（CNN）中的信息流。然后，我们介绍了部分信息分解（PID）框架，并开发了三个量来分析卷积层表示中的协同作用和冗余。我们的结果验证了两个基本的数据处理不等式，并揭示了与CNN训练有关的更多内部属性。