Is-ClusterMPP: clustering algorithm through point processes and influence space towards high-dimensional data

Abstract

Clustering via marked point processes and influence space, Is-ClusterMPP, is a new unsupervised clustering algorithm through adaptive MCMC sampling of a marked point processes of interacting balls. The designed Gibbs energy cost function makes use of k-influence space information. It detects clusters of different shapes, sizes and unbalanced local densities. It aims at dealing also with high-dimensional datasets. By using the k-influence space, Is-ClusterMPP solves the problem of local heterogeneity in densities and prevents the impact of the global density in the detection of unbalanced classes. This concept reduces also the input values amount. The curse of dimensionality is handled by using a local subspace clustering principal embedded in a weighted similarity metric. Balls covering data points are constituting a configuration sampled from a marked point process (MPP). Due to the choice of the energy function, they tends to cover neighboring data, which share the same cluster. The statistical model of random balls is sampled through a Monte Carlo Markovian dynamical approach. The energy is balancing different goals. (1) The data driven objective function is provided according to k-influence space. Data in a high-dense region are favored to be covered by a ball. (2) An interaction part in the energy prevents the balls full overlap phenomenon and favors connected groups of balls. The algorithm through Markov dynamics, does converge towards configurations sampled from the MPP model. This algorithm has been applied in real benchmarks through gene expression data set of various sizes. Different experiments have been done to compare Is-ClusterMPP against the most well-known clustering algorithms and its efficiency is claimed.

Appendices

Annexe A: IS-ClusterMPP algorithm’s pseudo-code

Here is the pseudo-code transcription of IS-ClusterMPP algorithm. Algorithm 2 is iterated to implement the dynamical evolution. Algorithm 1 is called to implement the jumps: adding to and cancelling balls from the sampled configuration. When Algorithm 2 stops, then Algorithm 3 is called. To affect a cluster label to the remaining not covered data points, Algorithm 4 is run at the end. In Algorithm 1 $\text {size}({\underline{\omega }}^{i}) >0$ denotes the number of balls in the configuration ${\underline{\omega }}^{i}$. In Algorithm 2,—Jump simulation—means a call to Algorithm 1.

The functions used on the following algorithms are defined as:

• Not-converged The convergence of the simulation process is verified by the stabilization of the number of balls, i.e, no changes in the configurations size.

• Extract cc(x) this function identified the set of connected components cc(x).

• $x_{j}$Is covered by $cc_{i}$ if there is one or more balls $\omega _{p}(x_{p}, r_{p})$ such that $dist(x_{j}, x_{p}) \le r_{p}$, this function returns true.

• Minimal distance(P, NP) This function have as input two observations sets and it returns two observations, one from each input set, which have the smallest distance.

Annexe B: Proof of the energy function local stability

The local stability means, it exists $\varLambda \in {\mathbb {R}}$, such that for any ${\underline{w}}$ configuration and any $\xi$ object added to the configuration ${\underline{w}}$, the ratio of the probabilities, which is the conditional intensity, satisfies

$$\begin{aligned} \frac{p({\underline{w}} \cup \xi )}{p({\underline{w}})} \le \varLambda . \end{aligned}$$

Since $p({\underline{w}}) \propto e^{-U({\underline{\omega }})}$, it is equivalent to it exists $\varLambda \in {\mathbb {R}}$, such that for any ${\underline{\omega }}$ configuration and any $\xi$ object added to the configuration ${\underline{\omega }}$,

$$\begin{aligned} \varDelta U({\underline{\omega }};\xi ):= U ({\underline{\omega }}\cup \xi ) - U ({\underline{\omega }}) \end{aligned}$$

is uniformly bounded from below.

In the model considered, $\xi$ denotes a new ball to be added to the configuration ${\underline{\omega }}$,

$$\begin{aligned} \varDelta U({\underline{\omega }};\xi )= & {} V(\xi ) + \left( |cc ({\underline{\omega }})| - |cc ({\underline{\omega }}\cup \xi )| \right) \sum _{\omega _i(x_i,r_i) \in {\underline{\omega }}} \ | \text {IS}_{k}(x_i)| - |cc ({\underline{\omega }}\cup \xi )| \text {IS}_{k}(\xi ) \nonumber \\&+ \log \frac{|{\tilde{n}}({{\underline{\omega }}})|}{|{\tilde{n}}({{\underline{\omega }}\cup \xi })| } - \log \delta \ \frac{n_{\text {overlap}} ({\underline{\omega }}\cup \xi )}{n_{\text {overlap}} ({\underline{\omega }})}. \end{aligned}$$

(12)

Since $|\text {IS}_{k}(x)| \le k$, it holds $V(\xi )\ge (\frac{1}{k}-k) \ \text {e}$.

It holds

$$\begin{aligned} -1 \le \left( |cc ({\underline{\omega }})| - |cc ({\underline{\omega }}\cup \xi )| \right) \le |cc ({\underline{\omega }})| -1 \end{aligned}$$

and using the fact that $|cc ({\underline{\omega }})|$ is at most the number of balls in the configuration, the second and third parts are uniformly bounded from below.

Remark that the (random) number $N({\underline{\omega }})$ of balls in a configuration ${\underline{\omega }}$, sampled from the chosen model of point process, is Poisson distributed with a parameter which is finite since we consider point processes inside a compact.

Finally remark: $|{\tilde{n}}({{\underline{\omega }}})| \ge |{\tilde{n}}({{\underline{\omega }}\cup \xi })|$ and, with $\delta <1$, $n_{\text {overlap}} ({\underline{\omega }}\cup \xi ) \ge n_{\text {overlap}} ({\underline{\omega }})$.