Abstract The technologically-relevant task of feature extraction from data performed in deep-learning systems is routinely accomplished as repeated fast Fourier transforms (FFT) electronically in prevalent domain-specific architectures such as in graphics processing units (GPU). However, electronics systems are limited with respect to power dissipation and delay, due to wire-charging challenges related to interconnect capacitance. Here we present a silicon photonics-based architecture for convolutional neural networks that harnesses the phase property of light to perform FFTs efficiently by executing the convolution as a multiplication in the Fourier-domain. The algorithmic executing time is determined by the time-of-flight of the signal through this photonic reconfigurable passive FFT ‘filter’ circuit and is on the order of 10’s of picosecond short. A sensitivity analysis shows that this optical processor must be thermally phase stabilized corresponding to a few degrees. Furthermore, we find that for a small sample number, the obtainable number of convolutions per {time, power, and chip area) outperforms GPUs by about two orders of magnitude. Lastly, we show that, conceptually, the optical FFT and convolution-processing performance is indeed directly linked to optoelectronic device-level, and improvements in plasmonics, metamaterials or nanophotonics are fueling next generation densely interconnected intelligent photonic circuits with relevance for edge-computing 5G networks by processing tensor operations optically.

中文翻译：

---

用于光子张量运算的集成光子 FFT，以实现高效和高速的神经网络

摘要 从深度学习系统中执行的数据中提取特征的技术相关任务通常通过电子重复快速傅立叶变换 (FFT) 在流行的特定领域架构中完成，例如在图形处理单元 (GPU) 中。然而，由于与互连电容相关的线充电挑战，电子系统在功耗和延迟方面受到限制。在这里，我们提出了一种基于硅光子学的卷积神经网络架构，该架构利用光的相位特性，通过将卷积作为傅立叶域中的乘法执行来有效地执行 FFT。算法执行时间由通过该光子可重构无源 FFT“滤波器”电路的信号的飞行时间确定，短至 10 皮秒。灵敏度分析表明，该光学处理器必须在热相位上稳定几度。此外，我们发现对于小样本数量，每（时间、功率和芯片面积）可获得的卷积数比 GPU 高出大约两个数量级。最后，我们表明，从概念上讲，光学 FFT 和卷积处理性能确实与光电器件级和等离子体的改进直接相关，