In recent years, the efficacy of using artificial recurrent neural networks to model cortical dynamics has been a topic of interest. Gated recurrent units (GRUs) are specialized memory elements for building these recurrent neural networks. Despite their incredible success in natural language, speech, video processing, and extracting dynamics underlying neural data, little is understood about the specific dynamics representable in a GRU network, and how these dynamics play a part in performance and generalization. As a result, it is both difficult to know a priori how successful a GRU network will perform on a given task, and also their capacity to mimic the underlying behavior of their biological counterparts. Using a continuous time analysis, we gain intuition on the inner workings of GRU networks. We restrict our presentation to low dimensions, allowing for a comprehensive visualization. We found a surprisingly rich repertoire of dynamical features that includes stable limit cycles (nonlinear oscillations), multi-stable dynamics with various topologies, and homoclinic bifurcations. At the same time GRU networks are limited in their inability to produce continuous attractors, which are hypothesized to exist in biological neural networks. We contextualize the usefulness of different kinds of observed dynamics and support our claims experimentally.

中文翻译：

---

通过连续时间动力系统的视角观察门控循环单元

近年来，使用人工循环神经网络模拟皮质动力学的功效一直是人们感兴趣的话题。门控循环单元 (GRU) 是用于构建这些循环神经网络的专用存储元件。尽管它们在自然语言、语音、视频处理和提取神经数据背后的动态方面取得了令人难以置信的成功，但人们对 GRU 网络中可表示的特定动态以及这些动态如何在性能和泛化中发挥作用却知之甚少。因此，我们很难预先知道 GRU 网络在给定任务上的执行效果如何，也很难知道它们模仿其生物对应物的潜在行为的能力。通过连续时间分析，我们可以直观地了解 GRU 网络的内部运作。我们将演示限制在低维度，以实现全面的可视化。我们发现了令人惊讶的丰富动力学特征，包括稳定极限环（非线性振荡）、具有各种拓扑的多稳态动力学以及同宿分岔。同时，GRU 网络因其无法产生连续吸引子而受到限制，而连续吸引子被假设存在于生物神经网络中。我们将不同类型的观察到的动态的有用性结合起来，并通过实验支持我们的主张。