Due to the movement expressiveness and privacy assurance of human skeleton data, 3D skeleton-based action inference is becoming popular in healthcare applications. These scenarios call for more advanced performance in application-specific algorithms and efficient hardware support. Warnings on health emergencies sensitive to response speed require low latency output and action early detection capabilities. Medical monitoring that works in an always-on edge platform needs the system processor to have extreme energy efficiency. Therefore, in this paper, we propose the MC-LSTM, a functional and versatile 3D skeleton-based action detection system, for the above demands. Our system achieves state-of-the-art accuracy on trimmed and untrimmed cases of general-purpose and medical-specific datasets with early-detection features. Further, the MC-LSTM accelerator supports parallel inference on up to 64 input channels. The implementation on Xilinx ZCU104 reaches a throughput of 18 658 Frames-Per-Second (FPS) and an inference latency of 3.5 ms with the batch size of 64. Accordingly, the power consumption is 3.6 W for the whole FPGA+ARM system, which is 37.8x and 10.4x more energy-efficient than the high-end Titan X GPU and i7-9700 CPU, respectively. Meanwhile, our accelerator also keeps a 4 $\sim$ 5x energy efficiency advantage against the low-power high-performance Firefly-RK3399 board carrying an ARM Cortex-A72+A53 CPU. We further synthesize an 8-bit quantized version on the same hardware, providing a 48.8% increase in energy efficiency under the same throughput.

中文翻译：

---

MC-LSTM：用于智能医疗保健应用的实时 3D 人体动作检测系统

由于人体骨骼数据的运动表现力和隐私保证，基于 3D 骨骼的动作推理在医疗保健应用中越来越流行。这些场景要求在特定于应用程序的算法和高效的硬件支持方面具有更高级的性能。对响应速度敏感的突发卫生事件警告需要低延迟输出和行动早期检测能力。在永远在线的边缘平台中工作的医疗监测需要系统处理器具有极高的能效。因此，在本文中，我们针对上述需求提出了 MC-LSTM，这是一种功能性且多功能的基于 3D 骨架的动作检测系统。我们的系统在具有早期检测功能的通用和医学特定数据集的修剪和未修剪案例上实现了最先进的准确性。更远，MC-LSTM 加速器支持多达 64 个输入通道的并行推理。在 Xilinx ZCU104 上的实现达到了 18 658 帧每秒 (FPS) 的吞吐量和 3.5 ms 的推理延迟，批大小为 64。因此，整个 FPGA+ARM 系统的功耗为 3.6 W，这分别比高端 Titan X GPU 和 i7-9700 CPU 的能效高 37.8 倍和 10.4 倍。同时，我们的加速器也保持了 4 $\sim$ 与搭载 ARM Cortex-A72+A53 CPU 的低功耗高性能 Firefly-RK3399 板相比，能效提高了 5 倍。我们在相同的硬件上进一步合成了一个 8 位量化版本，在相同的吞吐量下提供了 48.8% 的能效提升。