Improving Density Peaks Clustering through GPU acceleration

Highlights

• A vectorized VP-Tree layout and parallel construction method for VP-Tree is designed.

• A new method for parallel Density Peaks Clustering computation with GPU acceleration is proposed.

• A GPU-friendly VP-Tree dynamic update scheme is designed to support incremental clustering.

• A GPU-friendly incremental DPC method based on the dynamic VP-Tree is designed.

Abstract

Density Peaks Clustering (DPC) is a recently proposed clustering algorithm that has distinct advantages over existing clustering algorithms, which has already been used in a wide range of applications. However, DPC requires computing the distance between every pair of input points, therefore incurring quadratic computation overhead, which is prohibitive for large data sets. To address this efficiency problem, we propose to use GPU to accelerate DPC. We exploit a spatial index structure VP-Tree to efficiently maintain the data points and propose a GPU-friendly parallel VP-Tree construction algorithm. Based on the constructed VP-Tree, we propose a GPU-Accelerated DPC algorithm GDPC, in which the all-pair computation in DPC is greatly accelerated. Furthermore, in order to process dynamic evolving datasets, we propose an incremental GDPC algorithm, Incremental GDPC. Our results show that GDPC can achieve over 5.3-148.9X acceleration compared to the state-of-the-art GPU-based, multicore-based, and distributed DPC implementations, a 2.3-40.5X acceleration compared to the state-of-the-art incremental DPC algorithm.

Introduction

Data clustering is one of the most fundamental problems in many real-world applications, such as recommender systems, social networks, image processing, and bioinformatics. Basically, it groups a set of objects based on the similarities of objects such that objects in the same group (i.e., cluster) are more similar to each other than to those in other groups. Many different clustering algorithms have been proposed in the literature such as Kmeans [1] and DBSCAN [2]. There are also many research efforts to improve the efficiency of clustering to handle massive data [3], [4], [5].

Density Peaks Clustering (DPC) [6] is a novel clustering algorithm proposed recently. Given a set of points, DPC computes two metrics for every point $p$: (i) the local density $\rho$, which is the number of points within a specified distance from $p$; and (ii) the dependent distance $\delta$, which is the minimum distance from $p$ to other points with higher densities. It is observed that the center of a cluster sees the highest local density among its neighboring points (i.e., density center), and has a relatively large distance from other points with higher densities (i.e., far away from other density centers). Thus, cluster centers can be identified as points with both large $\rho$ and large $\delta$. With these identified cluster centers and the density center dependency trees extracted during the dependent distance $\delta$’s computation, the point-to-center relationship, or in other words, the point-to-cluster assignment (clustering results), can be discovered.

Compared with previous clustering algorithms, DPC has many advantages. (1) Unlike Kmeans, DPC does not require a pre-specified number of clusters. (2) DPC does not assume the clusters to be “balls” in space and supports arbitrarily shaped clusters. (3) DPC is more deterministic, since the clustering results have been shown to be robust against the initial choice of algorithm parameters. (4) The extraction of ($\rho$, $\delta$) provides a two-dimensional representation of the input data, which can be in very high dimensions, so that it is easier for users to gain new insights from the two-dimensional representation plot of the data. Due to its effectiveness and novelty, DPC has already been employed in a wide range of applications, such as neuroscience [7], geoscience [8], and computer vision [9].

While DPC is attractive for its effectiveness and its simplicity, the application of DPC is limited by its computational cost. In order to obtain the density values $\rho$, DPC computes the distance between every pair of points. That is, given $N$ points in the input data set, its computational cost is $O\left(N^2\right)$. Moreover, in order to obtain the dependent distance values $\delta$, a global sort operation on all points based on their density values (with computational cost $O\left(Nlog\left(N\right)\right)$) and $\frac{N(N-1)}{2}$ compare operations are required. As a result, it can be very time consuming to perform DPC on large data sets.

In the past few years, several research efforts have been put on accelerating DPC. LSH-DDP [3], EDDPC [10] and FDDP [11] leverage distributed approaches to help DPC handle large scale datasets. EDMStream [12] improves DPC by efficiently maintaining a novel in-memory dependent-tree structure. Ex-DPC and S-Approx-DPC [13] accelerate DPC efficiency by leveraging multi-core processing.

The recent advance in GPU technology is offering great prospects in parallel computation [14], [15]. With up to 80 GB GPU memory size [16], it is possible to use GPU to process large-scale data. There exist several related work have been devoted to accelerate DPC using GPU’s parallelization ability. Li et al. [17] propose a thread/block model and shared memory designs to accelerate the distance matrix computation. CUDA-DP [18] also integrates GPU’s parallelization ability and improves data locality to increase performance. However, these methods only focus on employing GPU’s many-core features to accelerate DPC, without paying attention to utilizing spatial index structures that can greatly filter out a large number of unnecessary all-pair computations.

In this paper, we exploit a spatial index structure vantage point tree (VP-Tree) [19] to help efficiently maintain clustering data. With VP-Tree, data points are partitioned into “hypershells” with decreasing radius. Comparing with other spatial index structures (such as KD-Tree [20] and Ball-Tree [21]), VP-Tree is more appropriate in DPC algorithm, because the decreasing-radius hypershell structure in VP-Tree can well support the point density computation (that obtains nearby points within a predefined radius of a point) and the dependent distance computation (that obtains the distance to the nearest neighbor with higher density). Besides, VP-Tree is more suitable for clustering high-dimensional data [22]. More importantly, the construction of VP-Tree and the search of VP-Tree can be well parallelized to adapt to GPU’s many-core architecture. Based on the GPU-based VP-Tree, we propose GDPC algorithm, where the density $\rho$ and the dependent distance $\delta$ can be efficiently calculated by querying the index structure and lots of unnecessary distance measurements (between faraway points) can be avoided.

On the other hand, new data are produced every day and the data distribution may evolve overtime. As a result, the clustering results should continuously evolve correspondingly. The dynamically evolving feature of data drives us to seek an incremental clustering approach. Rather than re-performing DPC on the whole updated data sets, the incremental clustering algorithm leverages previous clustering results and only updates the affected point-to-cluster assignments. Considering the massive size of data sets, the incremental processing approach is desirable especially in production usage. Therefore, we further propose Incremental GDPC to support incremental clustering, which extends GDPC in the following aspects. (1) We design a GPU-friendly dynamic VP-Tree index update scheme that reduces the number of tree traversals and eliminates the write-write conflicts in GPU’s many core computations. (2) Based on this dynamic VP-Tree index, we propose Incremental GDPC which can efficiently update the clustering results.

To sum up, we list our contributions in the following.

• GPU-Accelerated VP-Tree Construction. We design a vectorized VP-Tree layout to adapt to GPU architecture and take full advantage of GPU parallelism to speed up the VP-Tree index construction.

• GPU-Accelerated DPC Implementation. We propose to use the VP-Tree index to improve the efficiency of all-pair computation and rely on this index to avoid unnecessary computations in DPC’s density evaluation and dependent distance evaluation with GPU’s parallel computation support.

• GPU-Accelerated Incremental DPC Update. We provide incremental clustering support for dynamically evolving data by designing the GPU-friendly incremental update methods of density and dependent distance based on the dynamic VP-Tree.

We perform experiments on various real-world datasets and compare with a state-of-the-art GPU-based DPC algorithm CUDA-DP [18], a multicore-based parallel DPC algorithm S-Approx-DPC [13], and a distributed DPC implementation LSH-DDP [3]. Our results show that our GDPC can achieve 5.3–17.8X speedup over CUDA-DP, 43–148.9X speedup over S-Approx-DPC and 44.8–78.8X speedup over LSH-DDP. We further perform experiments on evolving datasets and compare with the state-of-the-art incremental DPC algorithm EDMStream [12]. Our results show that our Incremental GDPC can achieve 2.3–40.5X speedup over EDMStream.

The remainder of the paper is organized as follows. Section 2 describes the background on DPC and GPU’s architecture. Section 3 presents the GPU-accelerated VP-Tree construction and query methods. Section 4 proposes our GPU-accelerated DPC algorithm GDPC. Section 5 introduces how we maintain the dynamic VP-Tree on GPU and proposes Incremental GDPC. Section 6 reports the experimental results. Section 7 discusses related work and Section 8 concludes the paper.