Introduction

Given dataset points Xn as genes, x1,x2,x3,…,xn, in d dimensional space say Rd, clustering process can be clearly stated as thus:

We are required to find partition subsets X1, X2, X3,…, Xk\(\forall\)xi, i = 1,2,3,…,.n, such that every gene falls into one of the subsets and no xi falls into two or more subsets.

Partitions X1, X2, X3,…, Xk satisfy the following: X1\(\cup\) X2\(\cup\) X3 … \(\cup\) Xk = X and Xi\(\cap\) Xj = 0 \(\forall\) i \(\ne\) j, where \(\cup\) represents union and \(\cap\) represents intersection.

In addition, we cluster to form subsets with the goal that data points xi that are similar as much as possible belongs to same group. This require a similarity measure (or dissimilarity measure) usually given in form of values to represent the degree of resemblance or natural association between one data and another [1,2,3,4,5]. The converse indicates dissimilarity measure ρ which satisfies the following condition:

ρ(xi,xi) = 0 and ρ(xi,xj) \(\ne\) 0 \(\forall\) i \(\ne\) j.

The use of clustering algorithms covers all aspects of studies ranging from Science and Engineering to Social Sciences and Humanities. In genomics, clustering is applied in data analysis to put genes into groups called clusters so that the degree of similarity or level of association is strong between gene membership of one cluster or same cluster and weak between gene membership of dissimilar or different clusters. Several genomic studies find usefulness in the application of hierarchical clustering and variants including the agglomerative and divisive clustering [6]. However, in partitional clustering, we have QT (Quality Threshold) clustering [7], Self Organising Map (SOM) [8] and Standard k-means [1, 4] which continue to evolve in recent years [9, 10] for analysis of high dimensional data. In k-means clustering, given a set of n data points in d-dimensional space Rd and k, the task is to determine a set of k points in Rd, called centers, so as to minimize the mean squared distance from each data point to its nearest center [4, 11].

The k-means method has been shown to be effective in producing good clustering results for many practical applications such as data compression and vector quantization and pattern classification [12]. Clustering is applicable in data mining and knowledge discovery [13, 14], and it is useful in image segmentation [15, 16]. In bioinformatic analysis of gene expression [17] and other high throughput studies, the application of clustering date as far back as old as bioinformatics [18, 19] itself and currently the use of clustering seems inevitable [20]. For efficient and effective analysis of microarray data, Osamor et al. [21] recently developed a novel Pearson correlation-based Metric Matrices k-means (MMk-means) with a better run-time O(nk2) than the Traditional k-means and other variants of k-means algorithm like Overlapped and Enhanced k-means algorithms developed by Fahim et al. [11].

Big data now forms an important part of life sciences, technological developments in omics have brought about them been considered in the big data domain. The word big data implies an immense amount of unstructured or structured data which can not be processed using conventional technology and which is characterized by rapid production speed, great variety, and extreme heterogeneity. High throughput data which encompass a great deal of omics data generate large volume of data which can be analyzed to yeild useful information, these data volume can be characterized with big data production [22].

Different assumptions and statistics are made by different investigators in the development of available clustering algorithms and this perhaps account for some conflicting clustering result obtained from several algorithms over same data. This can also be a function of several choices available to the users at the time of clustering [23]. To this end, clustering results may be rendered unreliable unless they are passed through a cluster quality validation check.

Revisiting cluster index model

There is no specific definition of a cluster that has been widely agreed upon. But objects in a cluster should be identical or compatible and objects should be dissimilar in different clusters. This means maximizing the similarity within a cluster and reducing similarities between clusters [24, 25]. Clustering is among the methods used, specifically in genomics, for gaining insight into biological methods. Clustering can of course be seen in other fields of biological data processing [26]. Clustering algorithms is intended to classify clusters or groups of objects, which are more like one another than other clusters. This type of data analysis approach is closely linked to the task of generating a data model, that entails establishing a simple collection of properties that provides an understandable explanation of crucial aspects of a dataset [24]. In general, clustering approaches give further insight into complex data.

An early cluster index model was given by Rand [27] and this was followed by the work of Fowlkes and Mallows [10]. Rand statistic was first adjusted by Morey and Agresti [28, 29] with some level of error [30] before a more accurate adjustment was later done by Hubert and Arabie [31] to what is now popularly known as Hubert and Arabie Adjusted Rand Index (ARIHA) [31]. Several authors in clustering community seem to believe that ARIHA is comparatively the preferred index to most other measures [30, 32]. Krieger and Green [23] proposed a model named SEGWAY based on Rand criterion measure to provide single clustering for a set of separate partitions in a database while Tseng and Kao [9] proposed a clustering algorithm christened, correlation search technique (CST) for analysis of data from gene expression.

Several algorithms have evolved [33] and more are still being implemented at increasing pace with unequal matching pace in the development and implementation of validity measures to ensure that appropriate results are obtained [34]. Few works done on cluster validity are either in the internal cluster validity measure [35, 36] or external validity measure which is our emphasis in this article. However, tools that provide easy application of these cluster indexes are scarce and the few created are either not freely available or the programming language used for implementation are obsolete and no longer compatible or supported by current software architectures. Some common examples includes ANSI COBOL program, Clustisz for IBM 370 DOS created in 1975 by McClain and Rao [37], and the 1996 BASIC program of Saltstone and Stange [38] whose cluster quality validation implementation are now obsolete and publicly unavailable online. These tool in their current state, are no longer compatible with current software environment hence the need for a new tool that is compatible, platform independent, easy-to-use, friendly, and publicly available as proposed in this work. We were able to improve the reliability of clustering results through the implementation and use of a Java and Spreadsheet-based cluster validity tool called OsamorSoft using the clustering index model of Hubert and Arabie Adjusted Rand Index (ARIHA). The proposed solution constitutes DNA_Omatrix and OsamorSpreadSheet used to respectively generate a matching matrix and evaluate the level of agreement between the cluster output of two k-means algorithms. It could also be useful in determining the cluster quality of each algorithm by comparing it with clusters of known structure as benchmark or gold standard algorithm.

Methodology

Basic concept

  • If there are two clustering algorithms 1 and algorithm 2, each with its cluster output to be compared to one another, then the output of these algorithms represents the cluster partitions with cluster i (i = 1,2,3 …, I) and cluster j (j = 1,2,3 …, J).

  • If there is a gold standard cluster upon whose benchmark is to be used to determine the quality of clustering of another cluster algorithm, then the gold standard cluster structure should be known.

  • Each of these partitions has clusters(subsets/groups) and the union of these subsets is the total number of genes N being clustered.

  • The intersection of any two subsets may or may not be 0 for the number of matching genes in a pair of subsets which helps to generate the matching matrix H.

  • Matching matrix is a text file which when opened using a Spreadsheet shows clearly the number of elements that are found in same intersecting clusters forming the square matrix.

  • ARIHA is Hubert and Arabie Adjusted Rand Index which is an improved form of Rand Index, adjudged as more accurate and improved Rand Index is also computed by four defined parameters represented by a, b, c, and d.

  • OsamorSoft constitutes the entire software comprising DNA_OMatrix and OsamorSpreadSheet.

  • DNA_OMatrix is a Java module that generates a matching matrix that constitute the combination of algorithm 1 vs algorithm 2.

  • OsamorSpreadSheet is an integrated Spreadsheet embedded with formula to automatically compute ARIHA without human aid.

Clustering validation model

One interesting clustering validation task is comparative method validity for determining the quality of clusters between two algorithms [5, 27, 30, 31, 39]. Assuming there are two partitions of h data points being clustered by two algorithms and we are interested to know how closely their resultant output agree, then we need to obtain a matching matrix H.

The matching matrix H = {hij} representing the number of genes in cluster i (i = 1,2,3 …, I) where i represents subsets from algorithm 1 and cluster j (j = 1,2,3 …, J) where j represents subsets from algorithm 2 [32].

The details of the formula and variables of Hubert and Arabie Adjusted Rand Index presented in Eq. 1, 2, 3, 4, 5, 6 are further described in Hubert and Arabie [31], Yeung and Ruzzo [40], Steinley [30], Warren [32], Santos and Embrechts [41]

$$a\; = \;\sum\limits_{r\; = \;1}^{R} {\sum\limits_{c\; = \;1}^{C} {\;\left( {\mathop {t_{rc} }\limits_{2} } \right)} } \; = \;\left( {\sum\limits_{r\; = \;1}^{R} {\sum\limits_{c\; = \;1}^{C} {t_{rc}^{2} \;} } - \;n} \right)/2$$
(1)
$$b\; = \;\sum\limits_{r\; = \;1}^{R} {\left( {\mathop {t_{r} }\limits_{2} } \right)} \; = \;\left( {\sum\limits_{r\; = \;1}^{R} {t_{r}^{2} } - \;\sum\limits_{r\; = \;1}^{R} {\sum\limits_{c\; = \;1}^{C} {t_{rc}^{2} } } } \right)/2$$
(2)
$$c\; = \;\sum\limits_{c\; = \;1}^{C} {\left( {\mathop {t_{c} }\limits_{2} } \right)} \; = \;\left( {\sum\limits_{c\; = \;1}^{C} {t_{c}^{2} } - \;\sum\limits_{r\; = \;1}^{R} {\sum\limits_{c\; = \;1}^{C} {t_{rc}^{2} } } } \right)/2$$
(3)
$$d\; = \;\left( {\mathop n\limits_{2} } \right)\; - \;a\; - \;b\; - c\; = \;\left( {\mathop n\limits_{2} } \right)\; - \;\sum\limits_{r\; = \;1}^{R} {\;\left( {\mathop {t_{r} }\limits_{2} } \right)\; - \;} \sum\limits_{c\; = \;1}^{C} {\;\left( {\mathop {t_{c} }\limits_{2} } \right)\; + \;a}$$
(4)
$$d\; = \;\left( {\sum\limits_{r\; = \;1}^{R} {\sum\limits_{c\; = \;1}^{C} {{\text{t}}_{rc}^{2} \; + \;n^{2} \; - \;\sum\limits_{r\; = \;1}^{R} {t_{r}^{2} \; - \;} \sum\limits_{c\; = \;1}^{C} {t_{c}^{2} } } } } \right)/2$$
(5)
$$ARI_{HA} \; = \;\frac{{\left( {\mathop N\limits_{2} } \right)\;(a\; + \;d)\; - \;\left[ {(a\; + \;b)\;(a\; + \;c)\; + \;(c\; + \;d)\;(b\; + \;d)} \right]}}{{\left( {\mathop N\limits_{2} } \right)^{Z} \; - \;\left[ {(a\; + \;b)\;(a\; + \;c)\; + \;(c\; + \;d)\;(b\; + \;d)} \right]}}$$
(6)

.

Microarray data

The clustering of genomic data has been an outstanding approach to dealing with high-dimensional data provided by high-throughput technologies, such as microarrays of gene expression. With the advent of new technologies, such as next-generation sequencing, microarrays and eQTL mapping, large volume of high throughput biological data in terabytes are now produced [26]. The DNA microarray is one of the widely used techniques for evaluating millions of gene expressions simultaneously and microarray data were stored for any further analysis in publicly accessible repositories such as the Gene Expression Omnibus (GEO) [26].

Owing to its large number of features and the limited sample sizes, microarray data classification is a challenging task for many researchers [42, 43]. This type of data is used to collect tissue and cell sample information about the variations in gene expression that may aid in the diagnosis of a disease or to differentiate certain tumor types [42]. The microarray databases used most frequently in the literature were studied in [42] and the data characteristics issues such as their complexity and data imbalances were highlighted, hence, microarray data arising from image analysis output was used in the course of the research.

Design and implementation

Due to the need to have a clearer definition of the computed ARIHA values so that outcomes can be unambiguously related to specific biological meaning and interpretation, we designed and evolved a 4-stage cluster validation value categorization. This is a modification of the Kappa statistics categorization which has some challenges in relating its outcome to biological interpretation. We programmatically counted the number of cluster membership obtained from clustering experiment of malaria microarray data using four algorithms namely Traditional k-means, Enhanced k-means, Overlapped k-means [11] and MMkmeans as described in Osamor et al. [21], [44] and use it to observe a trend. This guided us to set heuristics scale similar to Steinley [30] with stages namely Excellent, Moderate, Fair and Poor for description of ARIHA as in Table 1.

Table 1 Set of Heuristics for validating clustering technique
Full size table

Our method accepts cluster output of high-throughput data from two algorithms to be compared i.e. Algorithm 1 and Algorithm 2. The DNA_Omatrix tool accepts each cluster as a single file, truncates it into individual files equivalent to the number of clusters created. DNA_Omatrix processes it further to generate a matching matrix by finding the intersection of every 2 combination of clusters from the algorithms. This matrix is in turn pasted into the OsamorSpreadSheet which automatically computes the sum of squares of the matching matrix. Initially, the values of ARIHA obtained were not very informative, hence we proposed the need to set a threshold for appropriate performance category. Our heuristics threshold are four categories ranging from Excellent with greater than or equal to 0.90 to less than 0.65 for poor level of agreement between the algorithms being compared. Despite differences in techniques, the value description is a bit similar to Kappa statistic proposed by Cohen [45, 46] but differs by having 4 sets instead of 6 sets of heuristics as present in Kappa statistics.

DNA_Omatrix module

This module is implemented in Java which is platform-independent. This is an improvement of an earlier implementation where Saltstone and Stange [38] used the older basic programming language. Our solution uses two buttons “first browse button” to select the cluster file from Algorithm 1 and the “second browse button” to select another cluster file from Algorithm 2. Pressing the Compute Matrix button invokes the sorting and intersection sub-module to collate genes of all possible paired combination of Algorithm 1 vs Algorithm 2 clusters and generate a matching matrix. Matching matrix is a text file which when opened using a Spreadsheet bears the number of items in same clusters being compared.

OsamorSpreadSheet

OsamorSpreadSheet is an integrated Spreadsheet embedded with formula to automatically compute ARIHA without human aid. This module automatically computes the sum of squares, columns and rows totals. It accepts matching matrices from DNA_Omatrix and displays result as soon as the matrix is placed on it. Its computation is in 4 phases which can be described by pasting of matching matrix(Phase1), computation of sum of squares (Phase 2) and completion of other statistics computation for variables a, b, c, and d represented by Phase 3. These inputs are used to finally compute and display the final value of ARIHA and its validity description in Phase 4.

Results and discussion

Several packages and programming languages have been used to implement clustering algorithms. Optimization and changes in existing methods are normally introduced during the development and implementation of these codes, this in turn leads to new versions of the original methods with diverse improvements, to make such codes more effective [47]. For assessing the similarities in clustering algorithms, several measures have been identified, [24] compared the normalized mutual information, Fowlkes-Mallows index, Adjusted Rand index and Jaccard index. Furthermore, the work of Shirkhorshidi et al. [48] studied the influence of Similarity and Dissimilarity Measures on clustering while Zhang and Fang [49] investigated missing data and imputation on clustering validity. In these cases, emphasized was not focused on ARIHA. These problems happen when the expected value of the RI of two random partition does not take a constant value (zero for example) or the Rand statistic approaches its upper limit of unity as the number of cluster increases.

To test our method, firstly, we used example data of T1 and T2 matching matrix as described in Steinley [30]. This data of T1 and T2 fits dynamically into a 4 × 4 matrix of OsamorSpreadSheet. We may optionally hide the unused empty rows and columns using the spreadsheet’s “hide” and “unhide” feature to obtain a good sizable interface which is optional. At the completion of the experiment, the results of ARIHA as obtained for T1 = 0.245588. Similarly, we also obtained ARIHA for T2 = 0.7401 following same approach. Our result is in line with the output manually generated by Steinley [30]. This implies that the T2 has moderate validity compared to T1 matrix with very poor validity.

Secondly, in order to ensure that further evaluation is attained using a high-throughput dataset, we employed the use of microarray data of Bozdech et al. [50, 51] containing 4915 genes by 49 time points and that of Le Roch et al. [52] containing 5119 genes by 17 time points. Following the description as specified in Osamor et al. [21], we clustered these datasets independently using MMk-means, Enhanced k-means, Traditional k-means and Overlapped k-means algorithms. DNA_Omatrix was used to correctly obtain the matching matrices and this was subsequently followed by the use of OsamorSpreadSheet to compute the ARIHA and upon evaluation, we obtained similar result in line with Osamor et al. [21]. This result further confirm that our method as described in OsamorSoft is appreciably good to serve as a useful tool for comparing clustering qualities of algorithms especially where matching matrices can be obtained. Interestingly, the method is also able to give clue to the suitable number of k clusters to be applied from various checks of ARIHA and ultimately make biological results easier to analyse and interpret. This is achieved by comparing the various k-cluster runs against a gold standard clustering for cluster quality checks as judged by the ARIHA value. It is worthy to recall that Xu et al. [53] recently tried to formulated a unified validity index framework but their work is restricted to only hierarchical clustering having proposed two synthetical clustering validity (SCV) indexes where minimum spanning tree was utilized to compute the compactness of intra-cluster with a view to minimise deficiencies in accuracy for existing validity indexes. We emphasize here that the methodology described represents a big data solution as could be encapsulated in the methodology pipeline shown in Fig. 1.

Fig. 1
figure 1

The proposed pipeline allows raw big data access into both Algorithm 1 and 2 whose cluster partitions is intended to be evaluated. The DNA_O Matrix Java program accepts (see Fig. 2a, b), these clustered partitions separately and processes it to generate a Matching matrix that serves as input into OsamorSpreadsheet for computation of model statististics and AHIHA. Based on the computed values of the statistics, a decision is taken by the system to conclude one choice out of the 4 defined level of evaluation of similarity ranging from value of 0–1 as specified in Table 1

Full size image
Fig. 2
figure 2

a Interface of DNA_Omatrix with (b) Matching matrix for Bozdech et al.(2003) 3D7 Data comparing two algorithms (Traditional k-means vs Overlapped k-means)

Full size image

It is important to state clearly again that the methodology involves the use of big data because the dataset used is a large microarray data originating from imaging technology for gene expression in the biological domain. This is to say that what constitutes what can be classified as a big data varies from domain to domain but the major attribute of a big data is that it is large and maybe structured or unstructured and imaging technology output and application usually constitute a big data problem as seen in this work. The spreadsheet design is capable of computing AHIHA for 20x20 clusters on a screen viewport (See Fig. 3a–c).

Fig. 3
figure 3figure 3

a Phase 1 of OsamorSpreadSheet processing k = 15 with the 20x20 Matching matrix design, for Bozdech et al. [50] 3D7 data running Traditional K means vs Overlapped k-means. b Phase 2 of OsamorSpreadSheet with Automatic Computation of Sum of Squares. c Phase 3 of OsamorSpreadSheet completes the computation of other statistics required to display the ARIHA and the appropriate description of cluster validity according to our proposed design

Full size image

However, if higher dimension clusters are required, the worksheet is increased by simple insertion of additional rows and columns in the worksheet up to k = 16,384. An empty excel spreadsheet that serves as a single worksheet is made up of 1,048,576 rows by 16,384 columns, hence this Osamorspreadsheet application is able to process big data in the magnitude of specified by the capacity of the rows and column in the worksheet which transcends to a huge volume of data with its attendant simplicity in design and convenience to a wider user audience.

Conclusion

The reproducibility of ARIHA results of Steinley et al. [30] and Osamor et al. [21] indicate that the design and automation of our method for the computation of ARIHA is empirically proven to be adequate. To the best of our knowledge, Kappa statistics’ description set of 6, seems too many for describing a small range 0–1 for precise report desired for cluster agreement between two algorithms. From the result of our malaria microarray data analysis, our heuristics are more precise, compact and likely to serve as a fairly good enhancement for Kappa statistics’ assigned descriptions of values, notwithstanding the differences in techniques. Our interesting contribution is the development of a software solution tailored as an off-the-shelf application package. Spreadsheet guarantees the likelihood of its wide usage and applicability due to ease-of-use. Surely, an application resident on a spreadsheet like OsamorSpreadSheet will command wider usage against tools that may require program compilation, system troubleshooting and/or complications often associated with character user interface applications.

Furthermore, another contribution may be the use of this tool to give an idea on the suitable number of clusters for an optimum initial k cluster value selection thereby, adding to the solution of finding the value of initial k cluster value problem. The method gives clue to the suitable number of k clusters to be applied from various checks of ARIHA and ultimately make biological results easier to interpret. It is also important to accommodate the core idea and intuition that good clustering produces stable clustering result [53, 54] but since this does not occur in most cases, OsamorSoft tool becomes very handy measure to ensure robustness against the randomness of clustering results. Further work is desirable in evolving a framework to automate the integration of several other cluster validity schemes for easy comparative analysis and also testing the solution in HDFS or Spark or test the Java application on a distributed system.