Abstract Recent studies in computer vision show that, due to inhalation and exhalation, pixel variations can be captured by RGB-thermal videos to estimate the breathing rate (BR). In the last few years, considerable progress has been made to BR contactless monitoring, however, still many issues remain open. The performances of those methods significantly degrade under challenging conditions, specifically when subjects’ spontaneous movements, facial expressions and illumination variations are involved. In this paper, we propose a dynamic time warping-based optimization framework to automatically and accurately select regions that are most useful for robust BR estimation irrespective of these interfering factors. A signal quality index called Respiratory Signals Quality Index based on dynamic time warping (RSQI_dtw) is empirically developed. In order to address the undetected problems of facial areas due to large movements, two different methods based on face tracking or motion detection by RGB-thermal images are used to acquire the respiratory signals, and BRs are measured by the re-concatenated signal. In our framework, a time-domain processing procedure is further proposed, which contains de-trend by recursive least squares (RLS) algorithm, normalization and de-noise by band-pass filter for extracting mainstream signals. The results of validation experiments conducted with 36 subjects suggest that the proposed approach outperforms state-of-the-art BR estimation methods significantly under real-world conditions. Our study can be utilized to assist medical staff in diagnosis and treatment by remote and accurate respiratory rate detection and reduce close contact between medical staff and patients to a certain extent.

中文翻译：

---

在现实条件下协同使用 RGB 和热成像进行远程呼吸率测量

摘要 最近的计算机视觉研究表明，由于吸气和呼气，RGB 热视频可以捕获像素变化以估计呼吸率 (BR)。在过去几年中，BR 非接触式监测取得了相当大的进展，但仍有许多问题悬而未决。这些方法的性能在具有挑战性的条件下会显着降低，特别是当涉及受试者的自发运动、面部表情和光照变化时。在本文中，我们提出了一种基于动态时间扭曲的优化框架，以自动准确地选择对稳健 BR 估计最有用的区域，而不管这些干扰因素。一种称为基于动态时间扭曲的呼吸信号质量指数 (RSQI_dtw) 的信号质量指数是根据经验开发的。为了解决大运动导致面部区域未检测到的问题，采用基于面部跟踪或RGB热图像运动检测两种不同的方法来获取呼吸信号，并通过重新连接的信号测量BR。在我们的框架中，进一步提出了一种时域处理程序，其中包含通过递归最小二乘 (RLS) 算法去趋势、通过带通滤波器进行归一化和去噪以提取主流信号。对 36 名受试者进行的验证实验结果表明，所提出的方法在实际条件下明显优于最先进的 BR 估计方法。