Ever-growing deep-learning technologies are making revolutionary changes for modern life. However, conventional computing architectures are designed to process sequential and digital programs but are burdened with performing massive parallel and adaptive deep-learning applications. Photonic integrated circuits provide an efficient approach to mitigate bandwidth limitations and the power-wall brought on by its electronic counterparts, showing great potential in ultrafast and energy-free high-performance computation. Here, we propose an optical computing architecture enabled by on-chip diffraction to implement convolutional acceleration, termed “optical convolution unit” (OCU). We demonstrate that any real-valued convolution kernels can be exploited by the OCU with a prominent computational throughput boosting via the concept of structral reparameterization. With the OCU as the fundamental unit, we build an optical convolutional neural network (oCNN) to implement two popular deep learning tasks: classification and regression. For classification, Fashion Modified National Institute of Standards and Technology (Fashion-MNIST) and Canadian Institute for Advanced Research (CIFAR-4) data sets are tested with accuracies of 91.63% and 86.25%, respectively. For regression, we build an optical denoising convolutional neural network to handle Gaussian noise in gray-scale images with noise level σ=10$\sigma =10$, 15, and 20, resulting in clean images with an average peak signal-to-noise ratio (PSNR) of 31.70, 29.39, and 27.72 dB, respectively. The proposed OCU presents remarkable performance of low energy consumption and high information density due to its fully passive nature and compact footprint, providing a parallel while lightweight solution for future compute-in-memory architecture to handle high dimensional tensors in deep learning.

中文翻译：

---

通过片上光学衍射张量处理进行复杂的深度学习

不断发展的深度学习技术正在为现代生活带来革命性的变化。然而，传统的计算架构旨在处理顺序和数字程序，但却背负着执行大规模并行和自适应深度学习应用程序的负担。光子集成电路提供了一种有效的方法来减轻带宽限制和电子同行带来的功率墙，显示出超快和无能量高性能计算的巨大潜力。在这里，我们提出了一种通过片上衍射实现卷积加速的光学计算架构，称为“光学卷积单元”（OCU）。我们证明了 OCU 可以利用任何实值卷积核，并通过结构重新参数化的概念显着提高计算吞吐量。以 OCU 为基本单元，我们构建了一个光学卷积神经网络 (oCNN) 来实现两个流行的深度学习任务：分类和回归。对于分类，Fashion Modified National Institute of Standards and Technology (Fashion-MNIST) 和加拿大高级研究所 (CIFAR-4) 数据集经过测试，准确率分别为 91.63% 和 86.25%。对于回归，我们构建了一个光学去噪卷积神经网络来处理具有噪声水平的灰度图像中的高斯噪声 我们构建了一个光学卷积神经网络 (oCNN) 来实现两个流行的深度学习任务：分类和回归。对于分类，Fashion Modified National Institute of Standards and Technology (Fashion-MNIST) 和加拿大高级研究所 (CIFAR-4) 数据集经过测试，准确率分别为 91.63% 和 86.25%。对于回归，我们构建了一个光学去噪卷积神经网络来处理具有噪声水平的灰度图像中的高斯噪声 我们构建了一个光学卷积神经网络 (oCNN) 来实现两个流行的深度学习任务：分类和回归。对于分类，Fashion Modified National Institute of Standards and Technology (Fashion-MNIST) 和加拿大高级研究所 (CIFAR-4) 数据集经过测试，准确率分别为 91.63% 和 86.25%。对于回归，我们构建了一个光学去噪卷积神经网络来处理具有噪声水平的灰度图像中的高斯噪声σ = 10、15$\sigma =10$和 20，生成清晰的图像，平均峰值信噪比 (PSNR) 分别为 31.70、29.39 和 27.72 dB。由于其完全被动的性质和紧凑的占用空间，所提出的 OCU 呈现出低能耗和高信息密度的卓越性能，为未来的内存计算架构提供并行而轻量级的解决方案，以处理深度学习中的高维张量。