Distributed applications, such as database queries and distributed training, consist of both compute and network tasks. DAG-based abstraction primarily targets compute tasks and has no explicit network-level scheduling. In contrast, Coflow abstraction collectively schedules network flows among compute tasks but lacks the end-to-end view of the application DAG. Because of the dependencies and interactions between these two types of tasks, it is sub-optimal to only consider one of them. We argue that co-scheduling of both compute and network tasks can help applications towards the globally optimal end-to-end performance. However, none of the existing abstractions can provide fine-grained information for co-scheduling. We propose MXDAG, an abstraction to treat both compute and network tasks explicitly. It can capture the dependencies and interactions of both compute and network tasks leading to improved application performance.

中文翻译：

---

MXDAG：集群应用程序的混合抽象

分布式应用程序，例如数据库查询和分布式训练，由计算和网络任务组成。基于 DAG 的抽象主要针对计算任务，没有明确的网络级调度。相比之下，Coflow 抽象在计算任务之间共同调度网络流，但缺乏应用程序 DAG 的端到端视图。由于这两种类型的任务之间存在依赖性和相互作用，因此只考虑其中一种是次优的。我们认为，计算和网络任务的协同调度可以帮助应用程序实现全局最优的端到端性能。然而，现有的抽象都不能为协同调度提供细粒度的信息。我们提出了 MXDAG，这是一种显式处理计算和网络任务的抽象。