Chinese sign language (CSL) is one of the most widely used sign language systems in the world. As such, the automatic recognition and generation of CSL is a key technology enabling bidirectional communication between deaf and hearing people. Most previous studies have focused solely on sign language recognition (SLR), which only addresses communication in a single direction. As such, there is a need for sign language generation (SLG) to enable communication in the other direction (i.e., from hearing people to deaf people). To achieve a smoother exchange of ideas between these two groups, we propose a skeleton-based CSL recognition and generation framework based on a recurrent neural network (RNN), to support bidirectional CSL communication. This process can also be extended to other sequence-to-sequence information interactions. The core of the proposed framework is a two-level probability generative model. Compared with previous techniques, this approach offers a more flexible approximate posterior distribution, which can produce skeletal sequences of varying styles that are recognizable to humans. In addition, the proposed generation method compensated for a lack of training data. A series of experiments in bidirectional communication were conducted on the large 500 CSL dataset. The proposed algorithm achieved high recognition accuracy for both real and synthetic data, with a reduced runtime. Furthermore, the generated data improved the performance of the discriminator. These results suggest the proposed bidirectional communication framework and generation algorithm to be an effective new approach to CSL recognition.

中文翻译：

---

基于骨架的中文手语识别和生成，用于聋人和听力人之间的双向通信。

中文手语（CSL）是世界上使用最广泛的手语系统之一。因此，CSL的自动识别和生成是一项关键技术，可实现聋人和听力者之间的双向通信。以前的大多数研究都只集中在手语识别（SLR）上，后者仅解决单一方向上的交流问题。因此，需要手语生成（SLG）以实现另一个方向的通信（即，从听觉到聋哑人）。为了实现这两个小组之间更顺畅的思想交流，我们提出了一种基于骨架的CSL识别和生成框架，该框架基于递归神经网络（RNN），以支持双向CSL通信。该过程还可以扩展到其他序列间信息交互。所提出框架的核心是两级概率生成模型。与以前的技术相比，此方法提供了更灵活的近似后验分布，可以生成人类可以识别的各种样式的骨骼序列。另外，提出的生成方法弥补了训练数据的不足。在大型500 CSL数据集上进行了一系列双向通信实验。所提算法对真实数据和合成数据均具有较高的识别精度，并减少了运行时间。此外，生成的数据提高了鉴别器的性能。这些结果表明，所提出的双向通信框架和生成算法是一种有效的CSL识别新方法。