We introduce a novel in-home hand rehabilitation system for monitoring hand motions and assessing grip forces of stroke patients. The overall system is composed of a sensing device and a computer vision system. The sensing device is a lightweight cylindrical object for easy grip and manipulation, which is covered by a passive sensing layer called “Smart Skin.” The Smart Skin is fabricated using soft silicone elastomer, which contains embedded microchannels partially filled with colored fluid. When the Smart Skin is compressed by grip forces, the colored fluid rises and fills in the top surface display area. Then, the computer vision system captures the image of the display area through a red–green–blue camera, detects the length change of the liquid through image processing, and eventually maps the liquid length to the calibrated force for estimating the gripping force. The passive sensing mechanism of the proposed Smart Skin device works in conjunction with a single camera setup, making the system simple and easy to use, while also requiring minimum maintenance effort. Our system, on one hand, aims to support home-based rehabilitation therapy with minimal or no supervision by recording the training process and the force data, which can be automatically conveyed to physical therapists. In contrast, the therapists can also remotely instruct the patients with their training prescriptions through online videos. This study first describes the design, fabrication, and calibration of the Smart Skin, and the algorithm for image processing, and then presents experimental results from the integrated system. The Smart Skin prototype shows a relatively linear relationship between the applied force and the length change of the liquid in the range of 0–35 N. The computer vision system shows the estimation error <4% and a relatively high stability in estimation under different hand motions.

中文翻译：

---

智能皮肤：用于家庭手部康复的基于视觉的软压力传感系统

我们介绍了一种新型的家用手部康复系统，用于监测手部运动和评估中风患者的握力。整个系统由传感装置和计算机视觉系统组成。传感装置是一个轻量的圆柱形物体，便于抓握和操作，它被称为“智能皮肤”的无源传感层覆盖。Smart Skin 使用柔软的有机硅弹性体制造，其中包含部分填充有色流体的嵌入式微通道。当 Smart Skin 被握力压缩时，有色液体上升并填充顶部表面显示区域。然后，计算机视觉系统通过红绿蓝相机捕捉显示区域的图像，通过图像处理检测液体的长度变化，并最终将液体长度映射到校准力以估计夹持力。提议的 Smart Skin 设备的被动感应机制与单个摄像头设置结合使用，使系统简单易用，同时还需要最少的维护工作。一方面，我们的系统旨在通过记录训练过程和力量数据来支持以最少或没有监督的家庭为基础的康复治疗，这些数据可以自动传送给物理治疗师。相比之下，治疗师还可以通过在线视频远程指导患者的训练处方。本研究首先描述了智能皮肤的设计、制造和校准，以及图像处理算法，然后介绍了集成系统的实验结果。