Twister2 is an open-source big data hosting environment designed to process both batch and streaming data at scale. Twister2 runs jobs in both high-performance computing (HPC) and big data clusters. It provides a cross-platform resource scheduler to run jobs in diverse environments. Twister2 is designed with a layered architecture to support various clusters and big data problems. In this paper, we present the cross-platform resource scheduler of Twister2. We identify required services and explain implementation details. We present job startup delays for single jobs and multiple concurrent jobs in Kubernetes and OpenMPI clusters. We compare job startup delays for Twister2 and Spark at a Kubernetes cluster. In addition, we compare the performance of terasort algorithm on Kubernetes and bare metal clusters at AWS cloud.

中文翻译：

---

Twister2 大数据跨平台资源调度器

Twister2 是一个开源大数据托管环境，旨在大规模处理批处理和流数据。Twister2 在高性能计算 (HPC) 和大数据集群中运行作业。它提供了一个跨平台的资源调度器来在不同的环境中运行作业。Twister2 采用分层架构设计，以支持各种集群和大数据问题。在本文中，我们介绍了 Twister2 的跨平台资源调度器。我们确定所需的服务并解释实施细节。我们展示了 Kubernetes 和 OpenMPI 集群中单个作业和多个并发作业的作业启动延迟。我们比较了 Kubernetes 集群中 Twister2 和 Spark 的作业启动延迟。此外，我们比较了 terasort 算法在 Kubernetes 和 AWS 云上的裸机集群上的性能。