DYVERSE: DYnamic VERtical Scaling in multi-tenant Edge environments

Highlights

• Lightweight vertical scaling techniques are suitable for multi-tenant Edge systems.

• Performance of Edge applications improves with vertical scaling techniques applied.

• Dynamic priority management of Edge applications further improves the performance.

• System-aware dynamic vertical scaling method is the most effective technique.

Abstract

Multi-tenancy in resource-constrained environments is a key challenge in Edge computing. In this paper, we develop ‘DYVERSE: DYnamic VERtical Scaling in Edge’ environments, which is the first light-weight and dynamic vertical scaling mechanism for managing resources allocated to applications for facilitating multi-tenancy in Edge environments. To enable dynamic vertical scaling, one static and three dynamic priority management approaches that are workload-aware, community-aware and system-aware, respectively are proposed. This research advocates that dynamic vertical scaling and priority management approaches reduce Service Level Objective (SLO) violation rates. An online-game and a face detection workload in a Cloud-Edge test-bed are used to validate the research. The merit of DYVERSE is that there is only a sub-second overhead per Edge server when 32 Edge servers are deployed on a single Edge node. When compared to executing applications on the Edge servers without dynamic vertical scaling, static priorities and dynamic priorities reduce SLO violation rates of requests by up to 4% and 12% for the online game, respectively, and in both cases 6% for the face detection workload. Moreover, for both workloads, the system-aware dynamic vertical scaling method effectively reduces the latency of non-violated requests, when compared to other methods.

Introduction

The vision of next-generation distributed computing is to harness the network edge for computing [1], [2]. In contrast to servicing all user requests from the Cloud, a workload may be distributed across the Cloud and nodes, such as routers and switches or micro data centres, that are located at the edge of the network [3], [4], [5].

Fig. 1 shows a three-tier Edge computing architecture. The Cloud tier is represented by data centres that provide compute resources for workloads. The Edge tier uses nodes that are closer to users. These include: (i) traffic routing nodes — existing nodes that route Internet traffic, for example, Wi-Fi routers, which may be augmented with additional compute resources, and (ii) dedicated nodes — additional micro data centres, for example, cloudlets. Workloads hosted on the Edge could either be the same or a subset of functionalities of those hosted on the Cloud based on the availability of resources at the Edge. The end device tier represents user devices and sensors; 29 billion of these are estimated to be connected to the Internet by 2022.¹

In Edge computing, end devices are connected to Edge nodes instead of directly to servers in the Cloud. The benefits of distributing a workload across the Cloud and the Edge have already been established in the literature. They include reduced communication latencies and reduced traffic to the Cloud, which in turn improves response times and Quality-of-Service (QoS) [6], [7].

There are challenges in achieving the vision of using Edge computing for distributing Cloud workloads, especially when Edge nodes are resource constrained as in traffic routing nodes. This paper aims to investigate the problem of supporting multi-tenancy in Edge computing nodes with limited hardware resources when compared to the Cloud. In this case, the servers of multiple workloads hosted on the same Edge node are anticipated to compete for insufficient resources [8]. Similar to Cloud computing, multiple workloads are expected to share hardware resources in Edge computing [9] since it would not be cost-effective to customise individual Edge computing nodes for a specific application [10]. Although when multiple workloads are asking for computing resources from a resource-constrained Edge node, a few of the workloads may continue using Cloud servers. This would minimise the opportunity to leverage the edge of the network for reducing the distance of data transfer and subsequently improving the QoS of applications [11], [12]. Therefore, it is important to efficiently support multi-tenancy on the Edge.

Multi-tenancy on the Cloud is well researched to host multiple Virtual Machines (VMs) on the same underlying hardware [13], [14]. Nonetheless, it needs to be revisited in the context of Edge computing since: (i) Edge resources have limited processing capabilities, due to small form factor and low-power processors when compared to data centre resources [15]. (ii) The Edge is a more transient environment (availability of resources change over time and may be available only for short periods) compared to the Cloud [16].

Multi-tenancy causes resources contention. Mechanisms employed on the Cloud to mitigate resource contention include vertical scaling, which is a process of allocating/deallocating resources to/from workloads so that multiple workloads can coexist [17]. These solutions are not suited for Edge environments since: (i) The mechanisms to monitor and optimise the allocation/deallocation of resources to meet user-defined objectives, specified as Service Level Objectives (SLOs) are typically computationally intensive [18], [19]. (ii) Predictive models used for estimating resource demands will have insufficient data for training [20], [21]. Edge services are expected to have short life cycles and may result in insufficient data to feed into an accurate Machine-Learning (ML) model; (iii) Edge environments are expected to have more transient system states compared to the Cloud and Edge workloads may run in a come-and-go style. Solutions designed for the Cloud platforms do not consider this. Therefore, a light-weight vertical scaling mechanism to facilitate multi-tenancy on the Edge is required, which is proposed and presented in this article.

As the execution of a workload progresses in a multi-tenant environment, there may be workloads that require more or fewer resources to meet their SLOs. If required resources are not available for Edge workloads, then SLOs will be violated. A static resource provisioning method is unsuitable given the frequent changes in an Edge environment and resource scaling is therefore required. The research in this paper proposes a dynamic vertical scaling mechanism that can be employed in a multi-tenant and resource-limited Edge environment.

The proposed mechanism is underpinned by a model that accounts for static priorities (set before the execution of a workload) and dynamic priorities (that changes during execution) of workloads on the Edge. While priorities have been exploited in Cloud computing [22], [23], we investigate it in the context of Edge computing in this article. The Edge is expected to be a premium service for Cloud workloads, and therefore, selecting Edge service users becomes important. We propose three dynamic priorities that are workload-aware, community-aware and system-aware to this end. We hypothesise that dynamic vertical scaling along with priority management approaches will improve the QoS of multi-tenant workloads and reduce SLO violation rates.

This paper makes the following novel contributions:

• The development of a framework for supporting multi-tenancy in a resource-constrained Edge computing node. In this framework, the Edge service QoS maximisation problem in a three-tier environment is formulated by considering the SLO violation rate, server co-location, dynamic vertical scaling, and priorities of workloads.

• The design of the dynamic priority approaches for managing Edge applications in a multi-tenant environment, which accounts for Edge-specific characteristics and different economic models. Currently, research using priority management (for example the decision for offloading or task queuing) focuses on the pre-deployment phase. DYVERSE, on the other hand, applies priority management after deployment.

• The development of the lightweight dynamic vertical scaling mechanism that adjusts resource allocations for prioritised Edge applications after deployment. Existing resource management techniques are heavyweight (require significant processing) since they are computationally intensive [24]. DYVERSE offers resource management suited for resource-constrained Edge nodes.

• The evaluation of DYVERSE on two different use-cases in a realistic experimental environment. Much of existing research on resource management for Edge computing is evaluated using simulators [25], [26], [27]. The benefits of applying DYVERSE in an Edge node is on the contrary presented in a test-bed with a location-based mobile game and a real-time face detection application.

The feasibility of the proposed priority management approaches and dynamic vertical scaling mechanism is validated using an online-game and a face detection workload in an Edge environment. These workloads are a natural fit for using the Edge since they are latency critical — the response time is affected by the distance between the user device and the server. The merit of DYVERSE is observed in that they only have a sub-second overhead per Edge server when 32 servers are deployed on an Edge node. Additionally, we observe that scaling using static and dynamic priorities reduces the SLO violation rates of user requests by up to 4% and 12% for the online game (6% for the face detection workload) respectively, when compared to executing workloads on the Edge node without dynamic vertical scaling. Moreover, the proposed dynamic vertical scaling with the system-aware dynamic priority approach improves the latencies of requests that are not violated. The key result is that our initial hypothesis is confirmed.

The remainder of this paper is organised as follows. Section 2 presents the background and develops the problem model. Section 3 proposes one static and three dynamic priority management approaches for a multi-tenant Edge environment. Section 4 presents a dynamic vertical scaling mechanism for Edge nodes. Section 5 experimentally evaluates DYVERSE against a catalogue of metrics — system overhead, SLO violation rate and latency. Section 6 highlights the related work. Section 7 concludes this paper.