The development of grid and workflow technologies has enabled complex, loosely coupled scientific applications to be executed on distributed resources. Many of these applications consist of large numbers of short-duration tasks whose runtimes are heavily influenced by delays in the execution environment. Such applications often perform poorly on the grid because of the large scheduling overheads commonly found in grids. In this paper we present a provisioning system based on multi-level scheduling that improves workflow runtime by reducing scheduling overheads. The system reserves resources for the exclusive use of the application, and gives applications control over scheduling policies. We describe our experiences with the system when running a suite of real workflow-based applications including in astronomy, earthquake science, and genomics. Provisioning resources with Corral ahead of the workflow execution, reduced the runtime of the astronomy application by up to 78% (45% on average) and of a genome mapping application by an order of magnitude when compared to traditional methods. We also show how provisioning can benefit applications both on a small local cluster as well as a large-scale campus resource.

中文翻译：

---

使用Corral进行科学工作流资源调配的经验

网格和工作流技术的发展使复杂，松散耦合的科学应用程序可以在分布式资源上执行。其中许多应用程序由大量的短期任务组成，这些任务的运行时受到执行环境中延迟的严重影响。由于网格中通常存在较大的调度开销，因此此类应用程序在网格上的性能通常很差。在本文中，我们提出了一种基于多层调度的资源调配系统，该系统通过减少调度开销来改善工作流的运行时间。系统保留专用于应用程序的资源，并为应用程序提供调度策略的控制权。当运行一套基于工作流程的实际应用程序时，包括天文学，地震科学和基因组学，我们将介绍我们在系统中的经验。与传统方法相比，在工作流执行之前通过Corral进行资源配置可将天文学应用程序的运行时间减少多达78％（平均为45％），并将基因组图谱应用程序的运行时间减少了一个数量级。我们还展示了预配如何使小型本地集群和大型园区资源上的应用程序受益。