Mobile services are becoming more diverse, making them have different demands on communications, computing, and caching (3C) resources in mobile systems. Unlike the traditional work that considers only one type of service, this paper designs a unified framework to characterize the different kinds of services, and jointly optimizes the 3C resources of the base station (BS) and mobile devices to provide differentiated quality of service (QoS) for diverse services. In the proposed framework, we model the task required by the mobile device to be generated at the BS, the mobile device, or both of them, which means the requested tasks are served through different paths, consuming different bandwidth, computing and caching resources. Since diverse services have different QoS, we formulate a multi-objective programming (MOP) to optimize the allocation of the 3C resources for minimizing the total delay while maximizing the number of executed tasks requested by the mobile devices. We transform the MOP problem as a multi-objective Markov decision process (MO-MDP) and design a multi-objective proximal policy optimization (MO-PPO) algorithm to solve the MO-MDP. The proposed MO-PPO first trains two sub-policies separately for the two objectives, and then combines them to search for Pareto dominating solutions. By alternately perform the separate training and the combination, we can finally obtain a set of Pareto optimal solutions and the corresponding Pareto front. Simulation results show that the proposed MO-PPO outperforms traditional methods in finding a higher-quality set of Pareto optimal solutions and can more appropriately allocate 3C resources to different types of services.

中文翻译：

---

通过多目标近端策略优化为多样化服务分配移动通信、计算和缓存资源


移动服务变得更加多样化，使得它们对移动系统中的通信、计算和缓存（3C）资源有不同的需求。与仅考虑一种服务的传统工作不同，本文设计了一个统一的框架来表征不同类型的服务，并联合优化基站（BS）和移动设备的3C资源，以提供差异化​​的服务质量（QoS） ）提供多样化的服务。在所提出的框架中，我们对移动设备所需的任务在基站、移动设备或两者上生成进行建模，这意味着请求的任务通过不同的路径提供服务，消耗不同的带宽、计算和缓存资源。由于不同的服务具有不同的 QoS，我们制定了多目标规划（MOP）来优化 3C 资源的分配，以最小化总延迟，同时最大化移动设备请求的执行任务数量。我们将MOP问题转化为多目标马尔可夫决策过程（MO-MDP），并设计一种多目标近端策略优化（MO-PPO）算法来求解MO-MDP。所提出的 MO-PPO 首先针对两个目标分别训练两个子策略，然后将它们组合起来寻找帕累托主导解决方案。通过交替进行单独训练和组合，最终可以得到一组Pareto最优解以及对应的Pareto前沿。仿真结果表明，所提出的MO-PPO在寻找更高质量的Pareto最优解集方面优于传统方法，并且可以更适当地将3C资源分配给不同类型的服务。