Many emerging cyber-physical systems, such as autonomous vehicles and robots, rely heavily on artificial intelligence and machine learning algorithms to perform important system operations. Since these highly parallel applications are computationally intensive, they need to be accelerated by graphics processing units (GPUs) to meet stringent timing constraints. However, despite the wide adoption of GPUs, efficiently scheduling multiple GPU applications while providing rigorous real-time guarantees remains a challenge. In this paper, we propose RTGPU, which can schedule the execution of multiple GPU applications in real-time to meet hard deadlines. Each GPU application can have multiple CPU execution and memory copy segments, as well as GPU kernels. We start with a model to explicitly account for the CPU and memory copy segments of these applications. We then consider the GPU architecture in the development of a precise timing model for the GPU kernels and leverage a technique known as persistent threads to implement fine-grained kernel scheduling with improved performance through interleaved execution. Next, we propose a general method for scheduling parallel GPU applications in real time. Finally, to schedule multiple parallel GPU applications, we propose a practical real-time scheduling algorithm based on federated scheduling and grid search (for GPU kernel segments) with uniprocessor fixed priority scheduling (for multiple CPU and memory copy segments). Our approach provides superior schedulability compared with previous work, and gives real-time guarantees to meet hard deadlines for multiple GPU applications according to comprehensive validation and evaluation on a real NVIDIA GTX1080Ti GPU system.

中文翻译：

---

RTGPU：具有精细粒度利用的硬期限并行任务实时GPU调度

许多新兴的网络物理系统，例如自动驾驶汽车和机器人，都严重依赖人工智能和机器学习算法来执行重要的系统操作。由于这些高度并行的应用程序需要大量计算，因此需要图形处理单元（GPU）对其进行加速，以满足严格的时序约束。但是，尽管GPU被广泛采用，但是在提供严格的实时保证的同时有效地调度多个GPU应用仍然是一个挑战。在本文中，我们提出了RTGPU，它可以实时调度多个GPU应用程序的执行，以适应严格的期限。每个GPU应用程序可以具有多个CPU执行和内存复制段，以及GPU内核。我们从一个模型开始，以明确说明这些应用程序的CPU和内存复制段。然后，我们在为GPU内核开发精确的时序模型时考虑了GPU体系结构，并利用称为持久线程的技术来实现细粒度的内核调度，并通过交错执行来提高性能。接下来，我们提出了一种实时调度并行GPU应用程序的通用方法。最后，为了调度多个并行GPU应用，我们提出了一种实用的实时调度算法，该算法基于联合调度和网格搜索（针对GPU内核段）以及单处理器固定优先级调度（针对多个CPU和内存副本段）。与以前的工作相比，我们的方法具有出色的可调度性，