We consider a natural scheduling problem which arises in many distributed computing frameworks. Jobs with diverse resource demands (e.g. memory requirements) arrive over time and must be served by a cluster of servers. To improve throughput and delay, the scheduler can pack as many jobs as possible in each server, however the sum of the jobs’ resource demands cannot exceed the server’s capacity. Motivated by the increasing complexity of workloads in shared clusters, we consider a setting where jobs’ resource demands belong to a very large set of diverse types, or in the extreme case even infinitely many types, i.e. resource demands are drawn from a general unknown distribution over a possibly continuous support. The application of classical scheduling approaches that crucially rely on a predefined finite set of types is discouraging in this high (or infinite) type setting. We first characterize a fundamental limit on the maximum throughput in such setting. We then develop oblivious scheduling algorithms, based on Best-Fit and Universal Partitioning , that have low complexity and can achieve at least 1/2 and 2/3 of the maximum throughput respectively, without the knowledge of the resource demand distribution . Extensive simulation results, using both synthetic and real traffic traces, are presented to verify the performance of our algorithms.

中文翻译：

---

高通量垃圾箱打包：在集群中调度具有随机资源需求的作业

我们考虑在许多分布式计算框架中都会出现的自然调度问题。具有多种资源需求（例如内存需求）的作业会随着时间的推移而到达，并且必须由服务器集群来服务。为了提高吞吐量和延迟，调度程序可以在每个服务器中打包尽可能多的作业，但是这些作业的资源需求总和不能超过服务器的容量。受共享集群中工作负载日益复杂化的推动，我们考虑一种设置，其中作业的资源需求属于非常大量的各种类型，或者在极端情况下甚至是无限多种类型，即资源需求是从总体上得出的。未知分配可能持续的支持。在这种高（或无限）类型设置中，不鼓励使用严重依赖于预定义的有限类型集的经典调度方法。我们首先描述在这种情况下最大吞吐量的基本限制。然后，我们基于最合适 和 通用分区 ， 具有 低复杂度 并可以实现 至少 最大吞吐量的分别为1/2和2/3， 没有资源需求分配的知识 。提出了使用综合和真实流量跟踪的广泛仿真结果，以验证我们算法的性能。