We study the learning dynamics and the representations emerging in recurrent neural networks (RNNs) trained to integrate one or multiple temporal signals. Combining analytical and numerical investigations, we characterize the conditions under which an RNN with $n$ neurons learns to integrate $D(\ll n)$ scalar signals of arbitrary duration. We show, for linear, ReLU, and sigmoidal neurons, that the internal state lives close to a $D$-dimensional manifold, whose shape is related to the activation function. Each neuron therefore carries, to various degrees, information about the value of all integrals. We discuss the deep analogy between our results and the concept of mixed selectivity forged by computational neuroscientists to interpret cortical recordings.

我们研究了经过训练以整合一个或多个时间信号的循环神经网络 (RNN) 中出现的学习动态和表示。结合分析和数值研究，我们描述了 RNN 与$n$ 神经元学习整合 $D(\ll n)$任意持续时间的标量信号。我们表明，对于线性、ReLU 和 sigmoidal 神经元，内部状态接近于$D$维流形，其形状与激活函数有关。因此，每个神经元都在不同程度上携带有关所有积分值的信息。我们讨论了我们的结果与计算神经科学家为解释皮层记录而伪造的混合选择性概念之间的深刻类比。