Random backpropagation (RBP) is a variant of the backpropagation algorithm for training neural networks, where the transpose of the forward matrices are replaced by fixed random matrices in the calculation of the weight updates. It is remarkable both because of its effectiveness, in spite of using random matrices to communicate error information, and because it completely removes the taxing requirement of maintaining symmetric weights in a physical neural system. To better understand random backpropagation, we first connect it to the notions of local learning and learning channels. Through this connection, we derive several alternatives to RBP, including skipped RBP (SRPB), adaptive RBP (ARBP), sparse RBP, and their combinations (e.g. ASRBP) and analyze their computational complexity. We then study their behavior through simulations using the MNIST and CIFAR-10 bechnmark datasets. These simulations show that most of these variants work robustly, almost as well as backpropagation, and that multiplication by the derivatives of the activation functions is important. As a follow-up, we study also the low-end of the number of bits required to communicate error information over the learning channel. We then provide partial intuitive explanations for some of the remarkable properties of RBP and its variations. Finally, we prove several mathematical results, including the convergence to fixed points of linear chains of arbitrary length, the convergence to fixed points of linear autoencoders with decorrelated data, the long-term existence of solutions for linear systems with a single hidden layer and convergence in special cases, and the convergence to fixed points of non-linear chains, when the derivative of the activation functions is included.

中文翻译：

---

在机器中学习：随机反向传播和深度学习通道

随机反向传播 (RBP) 是用于训练神经网络的反向传播算法的一种变体，其中在权重更新的计算中将前向矩阵的转置替换为固定随机矩阵。尽管使用随机矩阵来传达错误信息，但由于其有效性，并且因为它完全消除了在物理神经系统中保持对称权重的繁重要求，因此它非常出色。为了更好地理解随机反向传播，我们首先将其与本地学习和学习渠道的概念联系起来。通过这种联系，我们推导出了 RBP 的几种替代方案，包括跳过的 RBP (SRPB)、自适应 RBP (ARBP)、稀疏 RBP 及其组合（例如 ASRBP），并分析了它们的计算复杂性。然后，我们通过使用 MNIST 和 CIFAR-10 基准数据集的模拟来研究它们的行为。这些模拟表明，这些变体中的大多数都可以稳健地工作，几乎与反向传播一样，并且与激活函数的导数相乘很重要。作为后续工作，我们还研究了通过学习通道传达错误信息所需的比特数的低端。然后，我们对 RBP 的一些显着特性及其变化提供了部分直观的解释。最后，我们证明了几个数学结果，包括任意长度线性链向不动点的收敛性、具有去相关数据的线性自编码器向不动点的收敛性、具有单个隐藏层的线性系统解的长期存在性和收敛性在特殊情况下，