This paper proposes an ultrasound video interpretation algorithm that enables novel classes or instances to be added over time, without significantly compromising prediction abilities on prior representations. The motivating application is diagnostic fetal echocardiography analysis. Currently in clinical practice, recording full diagnostic fetal echocardiography is not common. Diagnostic videos are typically available in varying length and summarize a number of diagnostic sub-tasks of varying difficulty. Although large clinical datasets may be available at onset to build ultrasound image-based models for automatic image analysis, data may also become available over extended time to assist in algorithm refinement. To address this scenario, we propose to use an incremental learning approach to build a hierarchical network model that allows for a parallel inclusion of previously unseen anatomical classes without requiring prior data distributions. Super classes are obtained by coarse classification followed by fine classification to allow the model to self-organize anatomical structures in a sequence of categories through a modular architecture. We show that this approach can be adapted with new variable data distributions without significantly affecting previously learned representations. Two extreme situations of new data addition are considered; (1) when new class data is available over time with volume and distribution similar to prior available classes, and (2) when imbalanced datasets arrive over future time to be learned in a few-shot setting. In either case, availability of data from prior classes is not assumed. Evolution of the learning process is validated using incremental accuracies of fine classification over novel classes and compared to results from an end-to-end transfer learning-derived model fine-tuned on a clinical dataset annotated by experienced sonographers. The modularization of subsequent learning reduces the depreciation in future accuracies over old tasks from 6.75% to 1.10% using balanced increments. The depreciation is reduced from 6.95% to 1.89% with imbalanced data distributions in future increments, while retaining competitive classification accuracies in new additions of fine classes with parameter operations in the same order of magnitude in all stages in both cases.

中文翻译：

---

胎儿超声心动图视频中解剖结构的阶层式增量学习。

本文提出了一种超声视频解释算法，该算法能够随着时间的推移添加新颖的类或实例，而不会显着损害先前表示的预测能力。激励性的应用是诊断性胎儿超声心动图分析。当前在临床实践中，记录完整的胎儿超声心动图诊断并不常见。诊断视频通常以不同的长度提供，并且总结了许多难度不同的诊断子任务。尽管开始时可以使用大型临床数据集来构建基于超声图像的模型以进行自动图像分析，但是数据也可能会在较长的时间内变得可用，以协助算法优化。为了解决这种情况，我们建议使用增量学习方法来构建分层网络模型，该模型允许并行包含以前未见过的解剖学类，而无需事先进行数据分发。超级类是通过粗分类然后再进行细分类来获得的，以允许模型通过模块化体系结构按类别顺序自组织解剖结构。我们表明，该方法可以适应新的可变数据分布，而不会显着影响以前学习的表示形式。考虑了两种添加新数据的极端情况：（1）当新的类数据随时间推移可用并且其数量和分布与以前的可用类相似时，（2）当不平衡的数据集在将来的时间内到达并且需要几次拍摄才能学习。在任一情况下，不假定来自先前类别的数据的可用性。学习过程的演进通过使用新颖类的精细分类的增量准确性进行验证，并与端到端迁移学习派生模型的结果进行比较，该模型在经验丰富的超声医师注释的临床数据集上进行了微调。后续学习的模块化使用平衡增量将旧任务的未来准确性的折旧率从6.75％降低到1.10％。折旧率从6.95％降低到1.89％，这是因为在未来增量中数据分布不平衡，而在这两种情况下，在所有阶段中，新增加的细类的参数操作在相同数量级时都保持了竞争性分类精度。学习过程的演进通过使用新颖类的精细分类的增量准确性进行验证，并与端到端迁移学习派生模型的结果进行比较，该模型在经验丰富的超声医师注释的临床数据集上进行了微调。后续学习的模块化使用平衡增量将旧任务的未来准确性的折旧率从6.75％降低到1.10％。折旧率从6.95％降低到1.89％，这是因为在未来增量中数据分布不平衡，而在这两种情况下，在所有阶段中，新增加的细类的参数操作在相同数量级时都保持了竞争性的分类精度。学习过程的演进通过使用新颖类的精细分类的增量准确性进行验证，并与端到端迁移学习派生模型的结果进行比较，该模型在经验丰富的超声医师注释的临床数据集上进行了微调。后续学习的模块化使用平衡增量将旧任务的未来准确性的折旧率从6.75％降低到1.10％。折旧率从6.95％降低到1.89％，这是因为在未来增量中数据分布不平衡，而在这两种情况下，在所有阶段中，新增加的细类的参数操作在相同数量级时都保持了竞争性的分类精度。