Brains process information in spiking neural networks. Their intricate connections shape the diverse functions these networks perform. Yet how network connectivity relates to function is poorly understood, and the functional capabilities of models of spiking networks are still rudimentary. The lack of both theoretical insight and practical algorithms to find the necessary connectivity poses a major impediment to both studying information processing in the brain and building efficient neuromorphic hardware systems. The training algorithms that solve this problem for artificial neural networks typically rely on gradient descent. But doing so in spiking networks has remained challenging due to the nondifferentiable nonlinearity of spikes. To avoid this issue, one can employ surrogate gradients to discover the required connectivity. However, the choice of a surrogate is not unique, raising the question of how its implementation influences the effectiveness of the method. Here, we use numerical simulations to systematically study how essential design parameters of surrogate gradients affect learning performance on a range of classification problems. We show that surrogate gradient learning is robust to different shapes of underlying surrogate derivatives, but the choice of the derivative's scale can substantially affect learning performance. When we combine surrogate gradients with suitable activity regularization techniques, spiking networks perform robust information processing at the sparse activity limit. Our study provides a systematic account of the remarkable robustness of surrogate gradient learning and serves as a practical guide to model functional spiking neural networks.

大脑在尖峰神经网络中处理信息。它们错综复杂的联系塑造了这些网络执行的各种功能。然而，人们对网络连接与功能的关系知之甚少，并且尖峰网络模型的功能仍处于初级阶段。缺乏寻找必要连接的理论洞察力和实用算法，这对研究大脑中的信息处理和构建高效的神经形态硬件系统构成了主要障碍。为人工神经网络解决这个问题的训练算法通常依赖于梯度下降。但由于尖峰的不可微的非线性，在尖峰网络中这样做仍然具有挑战性。为了避免这个问题，可以使用替代梯度来发现所需的连接。然而，替代物的选择并不是唯一的，这就提出了其实施如何影响方法有效性的问题。在这里，我们使用数值模拟来系统地研究替代梯度的基本设计参数如何影响一系列分类问题的学习性能。我们表明代理梯度学习对不同形状的潜在代理导数具有鲁棒性，但导数尺度的选择会显着影响学习性能。当我们将替代梯度与合适的活动正则化技术相结合时，尖峰网络在稀疏活动限制下执行稳健的信息处理。