The aggregate-label learning paradigm tackles the long-standing temporary credit assignment (TCA) problem in neuroscience and machine learning, enabling spiking neural networks to learn multimodal sensory clues with delayed feedback signals. However, the existing aggregate-label learning algorithms only work for single spiking neurons, and with low learning efficiency, which limit their real-world applicability. To address these limitations, in this article, we first propose an efficient threshold-driven plasticity algorithm for spiking neurons, namely ETDP. It enables spiking neurons to generate the desired number of spikes that match the magnitude of delayed feedback signals and to learn useful multimodal sensory clues embedded within spontaneous spiking activities. Furthermore, we extend the ETDP algorithm to support multi-layer spiking neural networks (SNNs), which significantly improves the applicability of aggregate-label learning algorithms. We also validate the multi-layer ETDP learning algorithm in a multimodal computation framework for audio-visual pattern recognition. Experimental results on both synthetic and realistic datasets show significant improvements in the learning efficiency and model capacity over the existing aggregate-label learning algorithms. It, therefore, provides many opportunities for solving real-world multimodal pattern recognition tasks with spiking neural networks.

中文翻译：

---

一种用于多模态信息处理的有效阈值驱动聚合标签学习算法

聚合标签学习范式解决了神经科学和机器学习中长期存在的临时信用分配 (TCA) 问题，使尖峰神经网络能够学习具有延迟反馈信号的多模态感官线索。然而，现有的聚合标签学习算法仅适用于单个尖峰神经元，学习效率低，限制了它们的实际适用性。为了解决这些限制，在本文中，我们首先提出了一种有效的阈值驱动可塑性算法，用于尖峰神经元，即 ETDP。它使尖峰神经元能够产生所需数量的尖峰，与延迟反馈信号的幅度相匹配，并学习嵌入自发尖峰活动中的有用的多模态感觉线索。此外，我们扩展了 ETDP 算法以支持多层尖峰神经网络 (SNN)，这显着提高了聚合标签学习算法的适用性。我们还在用于视听模式识别的多模态计算框架中验证了多层 ETDP 学习算法。在合成和现实数据集上的实验结果表明，与现有的聚合标签学习算法相比，学习效率和模型容量有了显着提高。因此，它为使用尖峰神经网络解决现实世界的多模态模式识别任务提供了许多机会。我们还在用于视听模式识别的多模态计算框架中验证了多层 ETDP 学习算法。在合成和现实数据集上的实验结果表明，与现有的聚合标签学习算法相比，学习效率和模型容量有了显着提高。因此，它为使用尖峰神经网络解决现实世界的多模态模式识别任务提供了许多机会。我们还在用于视听模式识别的多模态计算框架中验证了多层 ETDP 学习算法。在合成和现实数据集上的实验结果表明，与现有的聚合标签学习算法相比，学习效率和模型容量有了显着提高。因此，它为使用尖峰神经网络解决现实世界的多模态模式识别任务提供了许多机会。