Our real-time actions in everyday life reflect a range of spatiotemporal dynamic brain activity patterns, the consequence of neuronal computation with spikes in the brain. Most existing models with spiking neurons aim at solving static pattern recognition tasks such as image classification. Compared with static features, spatiotemporal patterns are more complex due to their dynamics in both space and time domains. Spatiotemporal pattern recognition based on learning algorithms with spiking neurons therefore remains challenging. We propose an end-to-end recurrent spiking neural network model trained with an algorithm based on spike latency and temporal difference backpropagation. Our model is a cascaded network with three layers of spiking neurons where the input and output layers are the encoder and decoder, respectively. In the hidden layer, the recurrently connected neurons with transmission delays carry out high-dimensional computation to incorporate the spatiotemporal dynamics of the inputs. The test results based on the data sets of spiking activities of the retinal neurons show that the proposed framework can recognize dynamic spatiotemporal patterns much better than using spike counts. Moreover, for 3D trajectories of a human action data set, the proposed framework achieves a test accuracy of 83.6% on average. Rapid recognition is achieved through the learning methodology–based on spike latency and the decoding process using the first spike of the output neurons. Taken together, these results highlight a new model to extract information from activity patterns of neural computation in the brain and provide a novel approach for spike-based neuromorphic computing.

我们在日常生活中的实时行为反映了一系列时空动态大脑活动模式，这是神经元计算与大脑尖峰的结果。大多数具有尖峰神经元的现有模型旨在解决静态模式识别任务，例如图像分类。与静态特征相比，时空模式由于其在空间和时间域中的动态而更加复杂。因此，基于具有尖峰神经元的学习算法的时空模式识别仍然具有挑战性。我们提出了一种端到端的循环尖峰神经网络模型，该模型使用基于尖峰延迟和时间差异反向传播的算法进行训练。我们的模型是一个具有三层尖峰神经元的级联网络，其中输入和输出层分别是编码器和解码器。在隐藏层，具有传输延迟的循环连接神经元进行高维计算以结合输入的时空动态。基于视网膜神经元尖峰活动数据集的测试结果表明，所提出的框架可以比使用尖峰计数更好地识别动态时空模式。此外，对于人类行为数据集的 3D 轨迹，所提出的框架平均达到了 83.6% 的测试准确率。快速识别是通过学习方法实现的——基于尖峰延迟和使用输出神经元的第一个尖峰的解码过程。综上所述，这些结果突出了一种从大脑神经计算活动模式中提取信息的新模型，并为基于尖峰的神经形态计算提供了一种新方法。