We consider increasingly complex models of matrix denoising and dictionary learning in the Bayes-optimal setting, in the challenging regime where the matrices to infer have a rank growing linearly with the system size. This is in contrast with most existing literature concerned with the low-rank (i.e., constant-rank) regime. We first consider a class of rotationally invariant matrix denoising problems whose mutual information and minimum mean-square error are computable using standard techniques from random matrix theory. Next, we analyze the more challenging models of dictionary learning. To do so we introduce a novel combination of the replica method from statistical mechanics together with random matrix theory, coined spectral replica method. It allows us to conjecture variational formulas for the mutual information between hidden representations and the noisy data as well as for the overlaps quantifying the optimal reconstruction error. The proposed methods reduce the number of degrees of freedom from $\Theta(N^2)$ (matrix entries) to $\Theta(N)$ (eigenvalues or singular values), and yield Coulomb gas representations of the mutual information which are reminiscent of matrix models in physics. The main ingredients are the use of HarishChandra-Itzykson-Zuber spherical integrals combined with a new replica symmetric decoupling ansatz at the level of the probability distributions of eigenvalues (or singular values) of certain overlap matrices.

中文翻译：

---

字典学习的统计限制：随机矩阵理论和谱复制方法

我们考虑在贝叶斯最优设置中越来越复杂的矩阵去噪和字典学习模型，在具有挑战性的情况下，要推断的矩阵的秩随系统大小线性增长。这与有关低秩（即恒定秩）制度的大多数现有文献形成对比。我们首先考虑一类旋转不变矩阵去噪问题，其互信息和最小均方误差可以使用随机矩阵理论的标准技术进行计算。接下来，我们分析更具挑战性的字典学习模型。为此，我们引入了统计力学中复制方法与随机矩阵理论的新组合，即创造谱复制方法。它允许我们推测隐藏表示和噪声数据之间的互信息以及量化最佳重构误差的重叠的变分公式。所提出的方法将自由度数从 $\Theta(N^2)$（矩阵项）减少到 $\Theta(N)$（特征值或奇异值），并产生互信息的库仑气体表示让人想起物理学中的矩阵模型。主要成分是使用 HarishChandra-Itzykson-Zuber 球积分，并在某些重叠矩阵的特征值（或奇异值）的概率分布水平上结合了新的副本对称解耦 ansatz。所提出的方法将自由度数从 $\Theta(N^2)$（矩阵项）减少到 $\Theta(N)$（特征值或奇异值），并产生互信息的库仑气体表示让人想起物理学中的矩阵模型。主要成分是使用 HarishChandra-Itzykson-Zuber 球积分，并在某些重叠矩阵的特征值（或奇异值）的概率分布水平上结合了新的副本对称解耦 ansatz。所提出的方法将自由度数从 $\Theta(N^2)$（矩阵项）减少到 $\Theta(N)$（特征值或奇异值），并产生互信息的库仑气体表示让人想起物理学中的矩阵模型。主要成分是使用 HarishChandra-Itzykson-Zuber 球积分，并在某些重叠矩阵的特征值（或奇异值）的概率分布水平上结合了新的副本对称解耦 ansatz。