当前位置: X-MOL 学术Int. J. High Perform. Comput. Appl. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Demystifying asynchronous I/O Interference in HPC applications
The International Journal of High Performance Computing Applications ( IF 3.1 ) Pub Date : 2021-05-13 , DOI: 10.1177/10943420211016511
Shu-Mei Tseng 1 , Bogdan Nicolae 2 , Franck Cappello 2 , Aparna Chandramowlishwaran 1
Affiliation  

With increasing complexity of HPC workflows, data management services need to perform expensive I/O operations asynchronously in the background, aiming to overlap the I/O with the application runtime. However, this may cause interference due to competition for resources: CPU, memory/network bandwidth. The advent of multi-core architectures has exacerbated this problem, as many I/O operations are issued concurrently, thereby competing not only with the application but also among themselves. Furthermore, the interference patterns can dynamically change as a response to variations in application behavior and I/O subsystems (e.g. multiple users sharing a parallel file system). Without a thorough understanding, I/O operations may perform suboptimally, potentially even worse than in the blocking case. To fill this gap, this paper investigates the causes and consequences of interference due to asynchronous I/O on HPC systems. Specifically, we focus on multi-core CPUs and memory bandwidth, isolating the interference due to each resource. Then, we perform an in-depth study to explain the interplay and contention in a variety of resource sharing scenarios such as varying priority and number of background I/O threads and different I/O strategies: sendfile, read/write, mmap/write underlining trade-offs. The insights from this study are important both to enable guided optimizations of existing background I/O, as well as to open new opportunities to design advanced asynchronous I/O strategies.



中文翻译:

揭开HPC应用程序中异步I / O干扰的神秘面纱

随着HPC工作流程复杂性的增加,数据管理服务需要在后台异步执行昂贵的I / O操作,以使I / O与应用程序运行时重叠。但是,这可能会由于争用资源而引起干扰:CPU,内存/网络带宽。多核体系结构的出现加剧了这个问题,因为许多I / O操作是同时发出的,因此不仅与应用程序竞争,而且还相互竞争。此外,作为对应用程序行为和I / O子系统(例如,多个用户共享并行文件系统)变化的响应,干扰模式可以动态更改。如果没有透彻的了解,I / O操作可能无法达到最佳效果,甚至可能比阻塞情况下更差。为了填补这一空白,本文研究了HPC系统上异步I / O引起的干扰的原因和后果。具体来说,我们专注于多核CPU和内存带宽,以隔离每种资源造成的干扰。然后,我们进行深入研究,以解释各种资源共享方案中的相互作用和争用,例如,不同的优先级和后台I / O线程数以及不同的I / O策略:sendfile,read / write,mmap / write强调权衡。这项研究的见解对于实现对现有背景I / O的指导性优化以及为设计高级异步I / O策略提供新的机会都非常重要。隔离每种资源造成的干扰。然后,我们进行深入研究,以解释各种资源共享方案中的相互作用和争用,例如,不同的优先级和后台I / O线程数以及不同的I / O策略:sendfile,read / write,mmap / write强调权衡。这项研究的见解对于实现对现有背景I / O的指导性优化以及为设计高级异步I / O策略提供新的机会都非常重要。隔离每种资源造成的干扰。然后,我们进行深入研究,以解释各种资源共享方案中的相互作用和争用,例如,不同的优先级和后台I / O线程数以及不同的I / O策略:sendfile,read / write,mmap / write强调权衡。这项研究的见解对于实现对现有背景I / O的指导性优化以及为设计高级异步I / O策略提供新的机会都非常重要。

更新日期:2021-05-13
down
wechat
bug