Randomly switching neurons ON/OFF while training and inference process is an interesting characteristic of biological neural networks, that potentially results in inherent adaptability and creativity expressed by human mind. Dropouts inspire from this random switching behaviour and in the artificial neural network they are used as a regularization techniques to reduce the impact of over-fitting during the training. The energy-efficient digital implementations of convolutional neural networks (CNN) have been on the rise for edge computing IoT applications. Pruning larger networks and optimization for performance accuracy has been the main direction of work in this field. As opposed to this approach, we propose to build a near-sensor analogue CNN with high-density memristor crossbar arrays. Since several active elements such as amplifiers are used in analogue designs, energy efficiency becomes a main challenge. To address this, we extend the idea of using dropouts in training to also the inference stage. The CNN implementations require a subsampling layer, which is implemented as a mean pooling layer in the design to ensure lower energy consumption. Along with the dropouts, we also investigate the effect of non-idealities of memristor and that of the network.

中文翻译：

---

模拟神经忆阻卷积 dropout 网络

在训练和推理过程中随机打开/关闭神经元是生物神经网络的一个有趣特征，这可能会导致人类思维表达的内在适应性和创造力。Dropouts 从这种随机切换行为中得到启发，在人工神经网络中，它们被用作正则化技术，以减少训练过程中过度拟合的影响。卷积神经网络 (CNN) 的节能数字实现在边缘计算物联网应用中呈上升趋势。修剪更大的网络和优化性能精度一直是该领域的主要工作方向。与这种方法相反，我们建议构建一个具有高密度忆阻器交叉阵列的近传感器模拟 CNN。由于模拟设计中使用了放大器等多种有源元件，因此能效成为主要挑战。为了解决这个问题，我们将在训练中使用 dropout 的想法扩展到推理阶段。CNN 实现需要一个子采样层，该层在设计中作为平均池化层实现，以确保较低的能耗。除了辍学，我们还研究了忆阻器和网络的非理想性的影响。