We introduce a region template abstraction and framework for the efficient storage, management and processing of common data types in analysis of large datasets of high resolution images on clusters of hybrid computing nodes. The region template abstraction provides a generic container template for common data structures, such as points, arrays, regions, and object sets, within a spatial and temporal bounding box. It allows for different data management strategies and I/O implementations, while providing a homogeneous, unified interface to applications for data storage and retrieval. A region template application is represented as a hierarchical dataflow in which each computing stage may be represented as another dataflow of finer-grain tasks. The execution of the application is coordinated by a runtime system that implements optimizations for hybrid machines, including performance-aware scheduling for maximizing the utilization of computing devices and techniques to reduce the impact of data transfers between CPUs and GPUs. An experimental evaluation on a state-of-the-art hybrid cluster using a microscopy imaging application shows that the abstraction adds negligible overhead (about 3%) and achieves good scalability and high data transfer rates. Optimizations in a high speed disk based storage implementation of the abstraction to support asynchronous data transfers and computation result in an application performance gain of about 1.13×. Finally, a processing rate of 11,730 4K×4K tiles per minute was achieved for the microscopy imaging application on a cluster with 100 nodes (300 GPUs and 1,200 CPU cores). This computation rate enables studies with very large datasets.

中文翻译：

---

区域模板：高通量图像分析的数据表示和管理。


我们引入了区域模板抽象和框架，用于在混合计算节点集群上分析高分辨率图像的大型数据集时有效存储、管理和处理常见数据类型。区域模板抽象为空间和时间边界框中的常见数据结构（例如点、数组、区域和对象集）提供通用容器模板。它允许不同的数据管理策略和 I/O 实现，同时为数据存储和检索应用程序提供同质、统一的接口。区域模板应用程序被表示为分层数据流，其中每个计算阶段可以被表示为更细粒度任务的另一个数据流。应用程序的执行由运行时系统协调，该运行时系统实现混合机器的优化，包括用于最大化计算设备利用率的性能感知调度以及减少CPU和GPU之间数据传输影响的技术。使用显微镜成像应用程序对最先进的混合集群进行的实验评估表明，抽象增加的开销可以忽略不计（约 3%），并实现了良好的可扩展性和高数据传输率。对基于高速磁盘的抽象存储实现进行优化以支持异步数据传输和计算，从而使应用程序性能提高约 1.13 倍。最终，在具有 100 个节点（300 个 GPU 和 1,200 个 CPU 核心）的集群上，显微镜成像应用的处理速率达到了每分钟 11,730 个 4K×4K 切片。这种计算速率使得能够使用非常大的数据集进行研究。