An open problem in the cognitive dimensions of navigation concerns how previous exploratory experience is reorganized in order to allow the creation of novel efficient navigation trajectories. This behavior is revealed in the "traveling salesrat problem" (TSP) when rats discover the shortest path linking baited food wells after a few exploratory traversals. We have recently published a model of navigation sequence learning, where sharp wave ripple replay of hippocampal place cells transmit "snippets" of the recent trajectories that the animal has explored to the prefrontal cortex (PFC) (Cazin et al. in PLoS Comput Biol 15:e1006624, 2019). PFC is modeled as a recurrent reservoir network that is able to assemble these snippets into the efficient sequence (trajectory of spatial locations coded by place cell activation). The model of hippocampal replay generates a distribution of snippets as a function of their proximity to a reward, thus implementing a form of spatial credit assignment that solves the TSP task. The integrative PFC reservoir reconstructs the efficient TSP sequence based on exposure to this distribution of snippets that favors paths that are most proximal to rewards. While this demonstrates the theoretical feasibility of the PFC-HIPP interaction, the integration of such a dynamic system into a real-time sensory-motor system remains a challenge. In the current research, we test the hypothesis that the PFC reservoir model can operate in a real-time sensory-motor loop. Thus, the main goal of the paper is to validate the model in simulated and real robot scenarios. Place cell activation encoding the current position of the simulated and physical rat robot feeds the PFC reservoir which generates the successor place cell activation that represents the next step in the reproduced sequence in the readout. This is input to the robot, which advances to the coded location and then generates de novo the current place cell activation. This allows demonstration of the crucial role of embodiment. If the spatial code readout from PFC is played back directly into PFC, error can accumulate, and the system can diverge from desired trajectories. This required a spatial filter to decode the PFC code to a location and then recode a new place cell code for that location. In the robot, the place cell vector output of PFC is used to physically displace the robot and then generate a new place cell coded input to the PFC, replacing part of the software recoding procedure that was required otherwise. We demonstrate how this integrated sensory-motor system can learn simple navigation sequences and then, importantly, how it can synthesize novel efficient sequences based on prior experience, as previously demonstrated (Cazin et al. 2019). This contributes to the understanding of hippocampal replay in novel navigation sequence formation and the important role of embodiment.

中文翻译：

---

在模拟和物理大鼠机器人中实时进行海马位置细胞回放和前额叶序列学习的感觉运动整合，以实现新颖的路径优化。

导航的认知维度中的一个悬而未决的问题涉及如何重新组织先前的探索经验，以允许创建新颖有效的导航轨迹。当大鼠经过几次探索性遍历后，发现了连接诱饵食物井的最短路径时，这种行为在“旅行销售问题”（TSP）中得以体现。我们最近发布了一种导航序列学习模型，在该模型中，海马位置细胞的尖锐的波涟漪重播将动物探索的最近轨迹的“片段”传递到前额叶皮层（PFC）（Cazin等人，在PLoS Comput Biol 15 ：e1006624，2019）。PFC被建模为一个循环水库网络，该网络能够将这些代码段组装成有效的序列（通过位置单元激活编码的空间位置的轨迹）。海马重播模型根据其与奖励的接近程度生成摘要的分布，从而实现了一种解决TSP任务的空间信用分配形式。集成的PFC储存库基于对这些片段分布的暴露来重构有效的TSP序列，这些片段倾向于最接近奖励的路径。尽管这证明了PFC-HIPP相互作用的理论可行性，但将这种动态系统集成到实时感官运动系统中仍然是一个挑战。在当前的研究中，我们测试了PFC储层模型可以在实时感觉运动回路中运行的假设。因此，本文的主要目标是在模拟和真实机器人场景中验证模型。编码模拟和物理大鼠机器人当前位置的位置细胞激活将输入PFC容器，该PFC容器将生成后续的位置细胞激活，该后续位置细胞激活表示读出中所复制序列中的下一步。这输入到机器人，机器人前进到编码位置，然后从头生成当前的位置单元激活。这允许展示实施例的关键作用。如果从PFC中读取的空间代码直接回放到PFC中，则错误可能会累积，并且系统可能会偏离所需的轨迹。这需要一个空间滤波器，才能将PFC代码解码到某个位置，然后为该位置重新编码一个新的位置单元代码。在机器人中，PFC的位置单元矢量输出用于对机器人进行物理位移，然后为PFC生成新的位置单元编码输入，替换否则需要的部分软件记录过程。我们证明了这种集成的感觉运动系统如何能够学习简单的导航序列，然后重要的是，如先前所示，它如何可以基于先前的经验来合成新颖的有效序列（Cazin等人2019）。这有助于理解海马重放在新的导航序列形成中的作用以及实施方式的重要作用。