Spiking neural networks (SNNs), as the brain-inspired neural networks, encode information in spatio-temporal dynamics. They have the potential to serve as low-power alternatives to artificial neural networks (ANNs) due to their sparse and event-driven nature. However, existing SNN-based models for pixel-level semantic segmentation tasks suffer from poor performance and high memory overhead, failing to fully exploit the computational effectiveness and efficiency of SNNs. To address these challenges, we propose the multi-scale and full spike segmentation network (MFS-Seg), which is based on the deep direct trained SNN and represents the first attempt to train a deep SNN with surrogate gradients for semantic segmentation. Specifically, we design an efficient fully-spike residual block (EFS-Res) to alleviate representation issues caused by spiking noise on different channels. EFS-Res utilizes depthwise separable convolution to improve the distributions of spiking feature maps. The visualization shows that our model can effectively extract the edge features of segmented objects. Furthermore, it can significantly reduce the memory overhead and energy consumption of the network. In addition, we theoretically analyze and prove that EFS-Res can avoid the degradation problem based on block dynamical isometry theory. Experimental results on the Camvid dataset, the DDD17 dataset, and the DSEC-Semantic dataset show that our model achieves comparable performance to the mainstream UNet network with up to 31 fewer parameters, while significantly reducing power consumption by over 13. Overall, our MFS-Seg model demonstrates promising results in terms of performance, memory efficiency, and energy consumption, showcasing the potential of deep SNNs for semantic segmentation tasks. Our code is available in .

中文翻译：

---

用于语义分割的多尺度全尖峰模式

尖峰神经网络（SNN）作为受大脑启发的神经网络，以时空动态方式编码信息。由于其稀疏和事件驱动的性质，它们有潜力成为人工神经网络（ANN）的低功耗替代品。然而，现有的基于 SNN 的像素级语义分割任务模型性能较差且内存开销较高，无法充分利用 SNN 的计算有效性和效率。为了应对这些挑战，我们提出了多尺度全尖峰分割网络（MFS-Seg），它基于深度直接训练的 SNN，代表了使用代理梯度训练深度 SNN 进行语义分割的首次尝试。具体来说，我们设计了一个高效的全尖峰残差块（EFS-Res）来缓解不同通道上尖峰噪声引起的表示问题。 EFS-Res 利用深度可分离卷积来改善尖峰特征图的分布。可视化表明我们的模型可以有效地提取分割对象的边缘特征。此外，它可以显着降低网络的内存开销和能耗。此外，我们还基于块动态等距理论从理论上分析并证明了EFS-Res可以避免退化问题。在 Camvid 数据集、DDD17 数据集和 DSEC-Semantic 数据集上的实验结果表明，我们的模型在参数减少多达 31 个的情况下实现了与主流 UNet 网络相当的性能，同时显着降低了 13 以上的功耗。总体而言，我们的 MFS- Seg 模型在性能、内存效率和能耗方面展示了有希望的结果，展示了深度 SNN 在语义分割任务中的潜力。我们的代码可在 .