Heterogeneous memory systems that comprise memory nodes with disparate architectural characteristics (e.g., DRAM and high-bandwidth memory (HBM)) have surfaced as a promising solution in a variety of computing domains ranging from embedded to high-performance computing. Since deep learning (DL) is one of the most widely-used workloads in various computing domains, it is crucial to explore efficient memory management techniques for DL applications that execute on heterogeneous memory systems. Despite extensive prior works on system software and architectural support for efficient DL, it still remains unexplored to investigate heterogeneity-aware memory management techniques for high-performance DL on heterogeneous memory systems. To bridge this gap, we analyze the characteristics of representative DL workloads on a real heterogeneous memory system. Guided by the characterization results, we propose HALO, hotness- and lifetime-aware data placement and migration for high-performance DL on heterogeneous memory systems. Through quantitative evaluation, we demonstrate the effectiveness of HALO in that it significantly outperforms various memory management policies (e.g., 28.2 percent higher performance than the HBM-Preferred policy) supported by the underlying system software and hardware, achieves the performance comparable to the ideal case with infinite HBM, incurs small performance overheads, and delivers high performance across a wide range of application working-set sizes.

中文翻译：

---

用于异构内存系统上高性能深度学习的热度和生命周期感知数据放置和迁移

包含具有不同架构特征的存储器节点（例如，DRAM 和高带宽存储器 (HBM)）的异构存储器系统已成为从嵌入式计算到高性能计算的各种计算领域中的有前途的解决方案。由于深度学习 (DL) 是各种计算领域中使用最广泛的工作负载之一，因此为在异构内存系统上执行的 DL 应用程序探索有效的内存管理技术至关重要。尽管在系统软件和对高效 DL 的架构支持方面有大量的先前工作，但在异构内存系统上研究用于高性能 DL 的异构感知内存管理技术仍有待探索。为了弥补这一差距，我们分析了真实异构内存系统上代表性 DL 工作负载的特征。在表征结果的指导下，我们为异构内存系统上的高性能 DL 提出了 HALO、热度和生命周期感知数据放置和迁移。通过定量评估，我们证明了 HALO 的有效性，它显着优于底层系统软硬件支持的各种内存管理策略（例如，性能比 HBM-Preferred 策略高 28.2%），达到与理想情况相当的性能具有无限 HBM，产生很小的性能开销，并在广泛的应用程序工作集大小范围内提供高性能。