当前位置: X-MOL 学术ACM J. Emerg. Technol. Comput. Syst. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Accelerating Deep Neuroevolution on Distributed FPGAs for Reinforcement Learning Problems
ACM Journal on Emerging Technologies in Computing Systems ( IF 2.2 ) Pub Date : 2021-04-05 , DOI: 10.1145/3425500
Alexis Asseman 1 , Nicolas Antoine 1 , Ahmet S. Ozcan 1
Affiliation  

Reinforcement learning, augmented by the representational power of deep neural networks, has shown promising results on high-dimensional problems, such as game playing and robotic control. However, the sequential nature of these problems poses a fundamental challenge for computational efficiency. Recently, alternative approaches such as evolutionary strategies and deep neuroevolution demonstrated competitive results with faster training time on distributed CPU cores. Here we report record training times (running at about 1 million frames per second) for Atari 2600 games using deep neuroevolution implemented on distributed FPGAs. Combined hardware implementation of the game console, image preprocessing and the neural network in an optimized pipeline, multiplied with the system level parallelism enabled the acceleration. These results are the first application demonstration on the IBM Neural Computer, which is a custom designed system that consists of 432 Xilinx FPGAs interconnected in a 3D mesh network topology. In addition to high performance, experiments also showed improvement in accuracy for all games compared to the CPU implementation of the same algorithm.

中文翻译:

加速分布式 FPGA 的深度神经进化以解决强化学习问题

由深度神经网络的表征能力增强的强化学习在游戏和机器人控制等高维问题上显示出可喜的结果。然而,这些问题的顺序性对计算效率提出了根本性挑战。最近,进化策略和深度神经进化等替代方法在分布式 CPU 内核上以更快的训练时间展示了具有竞争力的结果。在这里,我们报告了使用在分布式 FPGA 上实现的深度神经进化的 Atari 2600 游戏的创纪录训练时间(以每秒约 100 万帧的速度运行)。在优化的管道中结合游戏控制台的硬件实现、图像预处理和神经网络,再加上系统级并行性,实现了加速。这些结果是 IBM 神经计算机上的第一个应用演示,这是一个定制设计的系统,由 432 个 Xilinx FPGA 组成,以 3D 网状网络拓扑互连。除了高性能之外,实验还表明,与相同算法的 CPU 实现相比,所有游戏的准确性都有所提高。
更新日期:2021-04-05
down
wechat
bug