Predictive coding (PC) networks are a biologically interesting class of neural networks. Their layered hierarchy mimics the reciprocal connectivity pattern observed in the mammalian cortex, and they can be trained using local learning rules that approximate backpropagation (Bogacz, 2017). However, despite having feedback connections that enable information to flow down the network hierarchy, discriminative PC networks are not typically generative. Clamping the output class and running the network to equilibrium yields an input sample that usually does not resemble the training input. This letter studies this phenomenon and proposes a simple solution that promotes the generation of input samples that resemble the training inputs. Simple decay, a technique already in wide use in neural networks, pushes the PC network toward a unique minimum two-norm solution, and that unique solution provably (for linear networks) matches the training inputs. The method also vastly improves the samples generated for nonlinear networks, as we demonstrate on MNIST.

中文翻译：

---

一个既具有判别性又具有生成性的预测编码网络

预测编码 (PC) 网络是一类具有生物学意义的神经网络。它们的分层层次结构模仿了在哺乳动物皮层中观察到的相互连接模式，并且可以使用近似反向传播的局部学习规则来训练它们（Bogacz，2017）。然而，尽管具有使信息能够沿网络层次结构向下流动的反馈连接，判别性 PC 网络通常不是生成性的。限制输出类并将网络运行到平衡会产生一个输入样本，该样本通常与训练输入不同。这封信研究了这种现象并提出了一个简单的解决方案，以促进生成类似于训练输入的输入样本。简单衰减，一种已经在神经网络中广泛使用的技术，将 PC 网络推向唯一的最小二范数解决方案，并且该唯一解决方案可证明（对于线性网络）与训练输入匹配。正如我们在 MNIST 上展示的那样，该方法还极大地改进了为非线性网络生成的样本。