Photoplethysmography imaging (PPGI) for non-contact monitoring of preterm infants in the neonatal intensive care unit (NICU) is a promising technology, as it could reduce medical adhesive-related skin injuries and associated complications. For practical implementations of PPGI, a region of interest has to be detected automatically in real time. As the neonates’ body proportions differ significantly from adults, existing approaches may not be used in a straightforward way, and color-based skin detection requires RGB data, thus prohibiting the use of less-intrusive near-infrared (NIR) acquisition. In this paper, we present a deep learning-based method for segmentation of neonatal video data. We augmented an existing encoder-decoder semantic segmentation method with a modified version of the ResNet-50 encoder. This reduced the computational time by a factor of 7.5, so that 30 frames per second can be processed at 960 × 576 pixels. The method was developed and optimized on publicly available databases with segmentation data from adults. For evaluation, a comprehensive dataset consisting of RGB and NIR video recordings from 29 neonates with various skin tones recorded in two NICUs in Germany and India was used. From all recordings, 643 frames were manually segmented. After pre-training the model on the public adult data, parts of the neonatal data were used for additional learning and left-out neonates are used for cross-validated evaluation. On the RGB data, the head is segmented well (82% intersection over union, 88% accuracy), and performance is comparable with those achieved on large, public, non-neonatal datasets. On the other hand, performance on the NIR data was inferior. By employing data augmentation to generate additional virtual NIR data for training, results could be improved and the head could be segmented with 62% intersection over union and 65% accuracy. The method is in theory capable of performing segmentation in real time and thus it may provide a useful tool for future PPGI applications.

用于新生儿重症监护病房 (NICU) 中早产儿非接触式监测的光电体积描记成像 (PPGI) 是一项很有前途的技术，因为它可以减少与医疗粘合剂相关的皮肤损伤和相关并发症。对于 PPGI 的实际实现，必须实时自动检测感兴趣区域。由于新生儿的身体比例与成人显着不同，现有的方法可能无法直接使用，并且基于颜色的皮肤检测需要 RGB 数据，因此禁止使用侵入性较小的近红外 (NIR) 采集。在本文中，我们提出了一种基于深度学习的新生儿视频数据分割方法。我们使用 ResNet-50 编码器的修改版本增强了现有的编码器-解码器语义分割方法。这将计算时间减少了 7.5 倍，因此可以以每秒 30 帧的速度处理 960 × 576 像素。该方法是根据成人的细分数据在公开数据库上开发和优化的。为了进行评估，使用了一个综合数据集，该数据集由在德国和印度的两个新生儿重症监护室记录的 29 名具有不同肤色的新生儿的 RGB 和 NIR 视频记​​录组成。从所有记录中，手动分割了 643 帧。在利用公共成人数据对模型进行预训练后，部分新生儿数据用于额外学习，而剩下的新生儿用于交叉验证评估。在 RGB 数据上，头部分割得很好（交集比并集为 82%，准确度为 88%），并且性能与在大型公共非新生儿数据集上实现的性能相当。另一方面，近红外数据的性能较差。 通过采用数据增强来生成用于训练的额外虚拟 NIR 数据，可以改进结果，并且可以以 62% 的交集比并集和 65% 的准确度对头部进行分割。该方法理论上能够实时执行分割，因此它可以为未来的 PPGI 应用提供有用的工具。