Spiking Neural Networks (SNNs) bear the potential of efficient unsupervised and continual learning capabilities because of their biological plausibility, but their complexity still poses a serious research challenge to enable their energy-efficient design for resource-constrained scenarios (like embedded systems, IoT-Edge, etc.). We propose SpikeDyn, a comprehensive framework for energy-efficient SNNs with continual and unsupervised learning capabilities in dynamic environments, for both the training and inference phases. It is achieved through the following multiple diverse mechanisms: 1) reduction of neuronal operations, by replacing the inhibitory neurons with direct lateral inhibitions; 2) a memory- and energy-constrained SNN model search algorithm that employs analytical models to estimate the memory footprint and energy consumption of different candidate SNN models and selects a Pareto-optimal SNN model; and 3) a lightweight continual and unsupervised learning algorithm that employs adaptive learning rates, adaptive membrane threshold potential, weight decay, and reduction of spurious updates. Our experimental results show that, for a network with 400 excitatory neurons, our SpikeDyn reduces the energy consumption on average by 51% for training and by 37% for inference, as compared to the state-of-the-art. Due to the improved learning algorithm, SpikeDyn provides on avg. 21% accuracy improvement over the state-of-the-art, for classifying the most recently learned task, and by 8% on average for the previously learned tasks.

中文翻译：

---

SpikeDyn：动态环境中具有连续和无监督学习能力的节能尖峰神经网络框架

尖刺神经网络（SNN）具有生物学上的合理性，因此具有有效的无监督和持续学习能力的潜力，但是其复杂性仍然构成严峻的研究挑战，以使其能够针对资源受限的场景（例如嵌入式系统，IoT-边缘等）。我们建议使用SpikeDyn，这是一个针对节能型SNN的全面框架，它在训练和推理阶段都具有动态环境中的连续和无监督学习能力。它是通过以下多种多样的机制实现的：1）通过用直接的侧向抑制代替抑制性神经元来减少神经元操作；2）内存和能量受限的SNN模型搜索算法，该算法采用分析模型来估计不同候选SNN模型的内存占用量和能耗，并选择帕累托最优SNN模型；和3）轻量级的连续无监督学习算法，该算法采用自适应学习速率，自适应膜阈值电位，权重衰减和虚假更新的减少。我们的实验结果表明，与现有技术相比，对于具有400个兴奋性神经元的网络，我们的SpikeDyn平均减少了51％的训练能量消耗和37％的推理能量消耗。由于改进了学习算法，SpikeDyn提供了平均值。对于最新学习的任务进行分类，与最新技术相比，准确性提高了21％，对于先前学习的任务，则平均提高了8％。