Healthy people use a keyboard and mouse as standard input devices for controlling a computer. However, these input devices are usually not suitable for people with severe physical disabilities. This study aims to design and implement a suitable and reliable human–computer interactive (HCI) interface for disabled users by integrating both eye tracking technology and lip motion recognition. Eye movements control the cursor position on the computer screen. An eye gaze followed by a lip motion of the mouth opening served as a mouse click action. Seven lip motion features were extracted to discriminate mouth openings and mouth closures with the cumulative sum control chart algorithm. A novel smoothing technique called the threshold-based Savitzky–Golay smoothing filter was proposed to stabilize the cursor movement due to the inherent jittery motions of eyes and to reduce eye tracking latencies. In this study, a fixation experiment with nine dots was carried out to evaluate the efficacy of eye gaze data smoothing. A Chinese text entry experiment based on the on-screen keyboard with four keypad sizes was designed to evaluate the influence of the keypad size on the Chinese text entry rate. The results of the fixation experiment indicated that the threshold-based Savitzky–Golay smoothing filter with a threshold value of two standard deviations, a polynomial order of 3, and a window length of 61 can significantly improve the stability of eye cursor movements by 44.86% on average. The averaged Chinese text entry rate achieved 4.41 wpm with the dynamical enlargeable keypads. The current results encouraged us to utilize the proposed HCI interface for disabled users in the future.

健康的人使用键盘和鼠标作为控制计算机的标准输入设备。但是，这些输入设备通常不适合严重身体残疾的人。这项研究旨在通过整合眼动追踪技术和唇部动作识别技术，为残疾用户设计和实现合适且可靠的人机交互（HCI）接口。眼睛的移动控制着光标在计算机屏幕上的位置。眼睛凝视，然后嘴唇张开，这是鼠标的点击动作。提取了七个嘴唇运动特征，以使用累积和控制图算法来区分嘴巴张开和嘴巴张开。提出了一种新的平滑技术，称为基于阈值的Savitzky-Golay平滑滤波器，以稳定由于眼睛固有的抖动运动而引起的光标移动并减少眼睛跟踪延迟。在这项研究中，进行了九个点的固定实验，以评估眼睛注视数据平滑的功效。设计了基于具有四个键盘尺寸的屏幕键盘的中文输入实验，以评估键盘尺寸对中文输入率的影响。固定实验的结果表明，基于阈值的Savitzky-Golay平滑滤波器的阈值为两个标准差，多项式阶数为3，窗口长度为61，可以显着提高眼睛光标移动的稳定性44.86％一般。中文平均输入率达到4。带有动态可扩展键盘的41 wpm。当前的结果鼓励我们将来将拟议的HCI接口用于残障用户。