The recognition of human actions in video streams is a challenging task in computer vision, with cardinal applications in brain–computer interfaces and surveillance, for example. Recently, deep learning has produced remarkable results, but it can be hard to use in practice, as its training requires large datasets and special-purpose and energy-consuming hardware. In this work, we propose a photonic hardware approach. Our experimental set-up comprises off-the-shelf components and implements an easy-to-train recurrent neural network with 16,384 nodes, scalable to hundreds of thousands of nodes. The system, based on the reservoir computing paradigm, is trained to recognize six human actions from the KTH video database using either raw frames as inputs or a set of features extracted with the histograms of an oriented gradients algorithm. We report a classification accuracy of 91.3%, comparable to state-of-the-art digital implementations, while promising a higher processing speed in comparison to the existing hardware approaches. Because of the massively parallel processing capabilities offered by photonic architectures, we anticipate that this work will pave the way towards simply reconfigurable and energy-efficient solutions for real-time video processing.

识别视频流中的人类动作是计算机视觉中一项具有挑战性的任务，例如在脑机接口和监视中的主要应用。最近，深度学习取得了显著成果，但由于训练需要大量数据集以及专用且耗能的硬件，因此很难在实践中使用。在这项工作中，我们提出了一种光子硬件方法。我们的实验设置包括现成的组件，并实现了一个易于训练的递归神经网络，该网络具有16,384个节点，可扩展到成千上万个节点。该系统基于油藏计算范例，经过训练可以使用原始帧作为输入或使用定向梯度算法的直方图提取的一组特征从KTH视频数据库中识别六个人的动作。我们报告的分类精度为91.3％，可与最新的数字实现相媲美，同时与现有的硬件方法相比，有望实现更高的处理速度。由于光子架构提供了大规模的并行处理功能，因此我们预计这项工作将为实时视频处理的简单可重新配置和高能效解决方案铺平道路。