The movement of large-scale (tens of Terabytes and larger) data sets between high performance computing (HPC) facilities is an important and increasingly critical capability. A growing number of scientific collaborations rely on HPC facilities for tasks which either require large-scale data sets as input or produce large-scale data sets as output. In order to enable the transfer of these data sets as needed by the scientific community, HPC facilities must design and deploy the appropriate data transfer capabilities to allow users to do data placement at scale. This paper describes the Petascale DTN Project, an effort undertaken by four HPC facilities, which succeeded in achieving routine data transfer rates of over 1PB/week between the facilities. We describe the design and configuration of the Data Transfer Node (DTN) clusters used for large-scale data transfers at these facilities, the software tools used, and the performance tuning that enabled this capability.

中文翻译：

---

Petascale DTN 项目：HPC 设施的高性能数据传输

在高性能计算 (HPC) 设施之间移动大规模（数十 TB 或更大）数据集是一项重要且日益关键的能力。越来越多的科学合作依赖于 HPC 设施来完成需要大规模数据集作为输入或产生大规模数据集作为输出的任务。为了能够根据科学界的需要传输这些数据集，HPC 设施必须设计和部署适当的数据传输功能，以允许用户进行大规模数据放置。本文介绍了 Petascale DTN 项目，该项目由四个 HPC 设施承担，成功实现了设施之间超过 1PB/周的常规数据传输率。