Performance optimization for parallel systems with shared DWM via retiming, loop scheduling, and data placement

Abstract

Domain Wall Memory (DWM) as an ideal candidate for replacing traditional memories especially in parallel systems, has many desirable characteristics such as low leakage power, high density and low access latency. However, due to the tape-like architecture of DWM, shift operations have a vital impact on performance. Considering data-intensive applications with massive loops and arrays, increasing parallelism of loops, appropriate loop scheduling and data placement on DWM will significantly improve the performance of parallel systems. This paper explores optimizing performance of parallel systems through retiming, loop scheduling and data placement especially when the data are arrays. It proposes Integer Linear Programming (ILP) formulation and Scheduling While Placing (SWP) algorithm to generate optimal or nearly optimal loop scheduling and data placement with minimum execution time. The experimental results show that SWP and ILP can effectively reduce execution time when compared with greedy List Scheduling First Access First Place (LF) algorithm. Besides, this paper proposes Threshold Retiming Repetition (TRR) algorithm to combine the retiming technique with SWP and ILP. The experimental results show that SWP+TRR and ILP+TRR can further reduce the execution time when compared to results without retiming.

Introduction

With the development of big data and artificial intelligence techniques, memory with large capacity and high access performance are required in modern computing systems. Domain Wall Memory (DWM) is a novel non-volatile memory with high access performance, high density and low power leakage [1]. For data-massive applications, DWM can meet the requirements of access speed, capacity and power consumption [2]. When compared to traditional memory, for example DRAM, DWM shows the same high density with close-to-zero standby power and it becomes an ideal candidate for future memory.

In previous works, since the write lifetime of NVM is limited, many researches aim at reducing the number of write operations when NVM is selected in hybrid memory [3], [4]. DWM has no write endurance problem [18], however, before accessing data in DWM, data in nanowires need to be shifted to align to read/write port, which is called shift operation. Shift operation will decrease the access speed of DWM and decrease system performance further. Therefore, reducing shift operations of DWM can significantly improve the performance. From hardware perspective, adding extra read/write ports to reduce the average distance between data and its closest head is considered in work [5], [6]. This enables higher access speed of DWM, however, more ports lower density which is a primary advantage of DMW. From software aspect, when given the number of ports, work [7] designs a compiler technique to reallocate the accessed data, which reduces the access time on DWM. The improvement of this technique is limited since the data access sequence is fixed. There is also a new work [17] figuring out a heuristic way to reschedule the sequence of instructions and reallocate data according to the intimacy between accessed data groups and DWM ports. It reduces number of shift operation to a great degree, however, it only considers single core system for simplicity. When it comes to multi-core system, there is at most one core shifting domains to align data to read/write ports while other are achieving inner chip operations, like calculating two registers, or waiting for shifting DWM later. Besides, different cores are able to access the same data at the same time. This introduces complexity when rescheduling instructions in programs. Moreover, big data and artificial intelligence system often repeat plenty of calculations on matrixes [20], [21], [22]. We can regard these operations as loops and each matrix as composition of a series of arrays. It’s necessary to think about how arrays are organized on DMW when data are accessed in loop programs repeatedly. Take these into account, in this paper, we target on parallel systems which may have multiple processing units or mutiple processors. In this kind of systems, single head DWM is used as a shared scratch-pad memory. We explore improving performance of system through retiming, loop scheduling and data placement when the data are arrays.

The main contributions of this paper are: 1) We explore retiming, loop scheduling and data placement simultaneously for loop program in parallel systems with shared single head DWM. 2) We propose Integer Linear Programming (ILP) formulation to generate optimal instructions schedule and data placement with minimum execution time. 3) We design a heuristic algorithm, Scheduling While Placing (SWP) algorithm, to generate nearly optimal loop schedule and data placement. 4) We propose the Threshold Retiming Repetition (TRR) algorithm which can be combined with ILP or SWP to further reduce the execution time of loop program.

The experimental results show that our ILP algorithm and heuristic SWP algorithm can effectively reduce execution time. When these two algorithms used with proposed TRR algorithm, the average performance of SWP indicates more room for improvement than ILP and the result of SWP+TRR is closer to the optimal result of ILP+TRR algorithm.

The rest of this paper is organized as follows: the architecture of DWM and related definitions are presented in section II. Section III introduces a motivation example to show the main ideas of this paper. We propose ILP formulation, TRR algorithm, and SWP algorithm in following three sections. Section VII analyzes the experimental results. Finally we conclude this paper in section VIII.