当前位置: X-MOL 学术arXiv.cs.ET › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Gradient Descent in Materio
arXiv - CS - Emerging Technologies Pub Date : 2021-05-15 , DOI: arxiv-2105.11233
Marcus N. Boon, Hans-Christian Ruiz Euler, Tao Chen, Bram van de Ven, Unai Alegre Ibarra, Peter A. Bobbert, Wilfred G. van der Wiel

Deep learning, a multi-layered neural network approach inspired by the brain, has revolutionized machine learning. One of its key enablers has been backpropagation, an algorithm that computes the gradient of a loss function with respect to the weights in the neural network model, in combination with its use in gradient descent. However, the implementation of deep learning in digital computers is intrinsically wasteful, with energy consumption becoming prohibitively high for many applications. This has stimulated the development of specialized hardware, ranging from neuromorphic CMOS integrated circuits and integrated photonic tensor cores to unconventional, material-based computing systems. The learning process in these material systems, taking place, e.g., by artificial evolution or surrogate neural network modelling, is still a complicated and time-consuming process. Here, we demonstrate an efficient and accurate homodyne gradient extraction method for performing gradient descent on the loss function directly in the material system. We demonstrate the method in our recently developed dopant network processing units, where we readily realize all Boolean gates. This shows that gradient descent can in principle be fully implemented in materio using simple electronics, opening up the way to autonomously learning material systems.

中文翻译:

Materio的梯​​度下降

深度学习是一种受大脑启发的多层神经网络方法,它彻底改变了机器学习。它的主要推动力之一是反向传播,它是一种计算损失函数相对于神经网络模型中权重的梯度的算法,并结合其在梯度下降中的使用。但是,在数字计算机中实施深度学习本质上是浪费的,对于许多应用而言,能耗变得过高。这刺激了专用硬件的开发,从神经形态的CMOS集成电路和集成的光子张量内核到非常规的基于材料的计算系统,一应俱全。这些材料系统中的学习过程例如通过人工进化或替代神经网络建模而发生,仍然是一个复杂且耗时的过程。在这里,我们展示了一种有效且准确的零差梯度提取方法,可直接在材料系统中对损失函数进行梯度下降。我们在最近开发的掺杂剂网络处理单元中演示了该方法,在那里我们可以轻松实现所有布尔门。这表明,从原理上讲,可以使用简单的电子设备完全实现梯度下降,从而为自主学习材料系统开辟了道路。
更新日期:2021-05-25
down
wechat
bug