With more and more event-based neuromorphic hardware systems being developed at universities and in industry, there is a growing need for assessing their performance with domain specific measures. In this work, we use the methodology of converting pre-trained non-spiking to spiking neural networks to evaluate the performance loss and measure the energy-per-inference for three neuromorphic hardware systems (BrainScaleS, Spikey, SpiNNaker) and common simulation frameworks for CPU (NEST) and CPU/GPU (GeNN). For analog hardware we further apply a re-training technique known as hardware-in-the-loop training to cope with device mismatch. This analysis is performed for five different networks, including three networks that have been found by an automated optimization with a neural architecture search framework. We demonstrate that the conversion loss is usually below one percent for digital implementations, and moderately higher for analog systems with the benefit of much lower energy-per-inference costs.

中文翻译：

---

在神经形态硬件上对深度尖峰神经网络进行基准测试

随着越来越多的基于事件的神经形态硬件系统在大学和工业中开发，越来越需要使用特定领域的措施来评估它们的性能。在这项工作中，我们使用将预训练的非尖峰神经网络转换为尖峰神经网络的方法来评估性能损失并测量三个神经形态硬件系统（BrainScaleS、Spikey、SpiNNaker）和通用模拟框架的每次推理能量CPU (NEST) 和 CPU/GPU (GeNN)。对于模拟硬件，我们进一步应用称为硬件在环训练的再训练技术来应对设备不匹配。该分析是针对五个不同的网络执行的，其中包括通过神经架构搜索框架的自动优化找到的三个网络。