Devices that facilitate nonverbal communication typically require high computational loads or have rigid and bulky form factors that are unsuitable for use on the face or on other curvilinear body surfaces. Here, we report the design and pilot testing of an integrated system for decoding facial strains and for predicting facial kinematics. The system consists of mass-manufacturable, conformable piezoelectric thin films for strain mapping; multiphysics modelling for analysing the nonlinear mechanical interactions between the conformable device and the epidermis; and three-dimensional digital image correlation for reconstructing soft-tissue surfaces under dynamic deformations as well as for informing device design and placement. In healthy individuals and in patients with amyotrophic lateral sclerosis, we show that the piezoelectric thin films, coupled with algorithms for the real-time detection and classification of distinct skin-deformation signatures, enable the reliable decoding of facial movements. The integrated system could be adapted for use in clinical settings as a nonverbal communication technology or for use in the monitoring of neuromuscular conditions.

促进非语言交流的设备通常需要高计算负载或具有不适合在面部或其他曲线身体表面上使用的刚性和庞大的外形因素。在这里，我们报告了用于解码面部应变和预测面部运动学的集成系统的设计和试点测试。该系统由用于应变映射的可大规模制造、适形的压电薄膜组成；用于分析适形装置和表皮之间的非线性机械相互作用的多物理场建模；和三维数字图像相关性，用于在动态变形下重建软组织表面以及为设备设计和放置提供信息。在健康个体和肌萎缩侧索硬化患者中，我们发现压电薄膜，再加上用于实时检测和分类不同皮肤变形特征的算法，可以对面部运动进行可靠解码。该集成系统可适用于临床环境中作为非语言交流技术或用于监测神经肌肉状况。