Accelerator for supervised neighborhood based attribute reduction

https://doi.org/10.1016/j.ijar.2019.12.013Get rights and content

Abstract

In neighborhood rough set, radius is a key factor. Different radii may generate different neighborhood relations for discriminating samples. Unfortunately, it is possible that two samples with different labels are regarded as indistinguishable, mainly because the neighborhood relation does not always provide satisfactory discriminating performance. Moreover, it should be noticed that the process of obtaining reducts in terms of multiple different radii is very time-consuming, mainly because different radii imply different reducts and those reducts should be searched, respectively. To solve the above problems, not only a supervised neighborhood relation is proposed for obtaining better discriminating performance, but also an accelerator is designed to speed up the process of obtaining reducts. Firstly, both intra-class radius and inter-class radius are proposed to distinguish samples. Different from the previous approaches, the labels of samples are taken into account and then this is why our approach is referred to as the supervised neighborhood based strategy. Secondly, from the viewpoint of the variation of radius, an accelerator is designed which aims to quickly obtain multiple radii based reducts. Such mechanism is based on the consideration that the reduct in terms of the previous radius may guide the process of obtaining the reduct in terms of the current radius. The experimental results over 12 UCI data sets show the following: 1) compared with the traditional and pseudo-label neighborhood based reducts, our supervised neighborhood based reducts can provide higher classification accuracies; 2) our accelerator can significantly reduce the elapsed time for obtaining reducts. This study suggests new trends for considering neighborhood rough set related topics.

Introduction

Presently, the neighborhood rough set [15], [48] has gained a substantial attention in the field of Granular Computing [11], [29], [41], [55], [57], [72], mainly because neighborhood rough set provides us with an effective way which can distinguish samples in terms of different granularities [21], [23], [26], [30], [31], [33]. Such different granularities can be acquired by using different values of radius because the distances between samples are invariant. Therefore, radius is a key factor and the values of radius may affect the neighborhood related results immediately. For example, if a smaller value of radius is used, then more samples will be separated from each other; if a greater value of radius is used, then less samples will be separated from each other.

Unfortunately, by using the neighborhood relation, samples with different labels may fall into the same neighborhood. This is mainly because the neighborhood relation obtained by a given radius is not good enough for providing satisfactory discriminating performance. From this point of view, Yang et al. [63] have proposed a pseudo-label strategy for re-constructing the neighborhood relation; Liu et al. [32] also have designed a pseudo-label strategy by using Label Propagation Algorithm, note that such strategy is applied to semi-supervised learning. Nevertheless, not only is it a time-consuming process for generating pseudo labels of samples, but also the information provided by pseudo labels may be incorrect which will lead to lower quality of neighborhood rough approximations. For instance, in Ref. [53], [63], the pseudo labels of samples are obtained by using the K-means clustering approach, obviously, such a process will take lots of time. Moreover, due to the imprecise results of clustering [1], [2], samples with the same true label may be equipped with different pseudo labels. To solve these problems, it is necessary for us to design a new strategy to study neighborhood rough set.

In view of this, a novel strategy which considers the true labels of samples will be explored. Different from the pseudo-label approach, our new strategy will be realized by using the true labels instead of the pseudo labels of samples. Obviously, without the process of generating pseudo labels, the time complexity of computing neighborhood rough set will not be increased. Furthermore, through using the true labels of samples, the case may be partially avoided, which samples are separated from each other while they have same label. In the context of this paper, our new approach will be referred to as the supervised neighborhood based strategy. The main thinking of our supervised neighborhood is: considering the true labels of samples, and then analyzing the relationship between samples with respect to two different cases, i.e., samples are with the same label and samples are with different labels. In other words, samples with the same label can be regarded as the case of intra-class [42] while samples with different labels can be regarded as the case of inter-class [42]. Consequently, it is reasonable for us to construct neighborhood by using two different radii: 1) intra-class radius is used to discriminate the samples with the characteristic of intra-class; 2) inter-class radius is used to discriminate the samples with the characteristic of inter-class. Note that the value of inter-class radius should be smaller than that of intra-class radius. Otherwise, more samples with different labels will fall into the same neighborhood and such a result is undesirable.

Up to now, it has been demonstrated that attribute reduction [4], [14], [27], [36], [43], [44], [52], [67], [70], [76] is one of the most important topics in the field of rough set [9], [10], [24], [37], [40], [45], [49], [51], [54], [56], [61], [62], [74], [75]. The reason can be contributed to the fact that attribute reduction provides us with the detailed explanations of selecting attributes for reducing the dimensionality of data. From the viewpoint of neighborhood rough set, the structure of neighborhood is closely related to the results of reduct. For example, the reducts derived by using different values of radius may be so different. This is mainly because a smaller value of radius may generate a smaller neighborhood, which indicates more samples will be separated from each other; a greater value of radius may generate a greater neighborhood, which indicates more samples can be regarded as indistinguishable. Thus, some challenges about neighborhood rough set based attribute reduction should be addressed.

  • 1.

    The radius used in neighborhood rough set may lead to powerless reduct. This is mainly because such radius does not take the information offered by the label into account and then the neighborhood related measures may be lower quality. For example, given a radius, if the distance between two samples is less than or equal to the radius, then such two samples will fall into the same neighborhood though the labels of them are different. Immediately, if the discriminating performance of neighborhood relation is required to be preserved in attribute reduction, then the derived reduct will not distinguish such two samples, either. From this point of view, it is worth studying the supervised neighborhood based attribute reduction.

  • 2.

    The single radius based attribute reduction cannot reflect the variation tendency of the performances of reducts. That is, if one and only one radius is used, then only the reduct related to such radius can be obtained. Immediately, it is impossible for us to observe the variation tendency of the performances of reducts. For such a reason, it is necessary to address the problem of attribute reduction in terms of multiple different radii.

  • 3.

    The previous searching strategy may lead to higher time consumption if multiple radii related attribute reduction is considered. For instance, if the reducts based on n different radii are required, then the used searching process will be repeated n times. If the number of used radii is increased further, then the time consumption will also be increased. Therefore, how to accelerate the process of finding multiple radii based reducts should also be paid much attention to.

From discussions above, considering the information provided by the true labels of samples, the supervised neighborhood based attribute reduction will be studied. Moreover, to further speed up the process of finding multiple radii based reducts, an acceleration strategy will be introduced into our searching process. Different from the traditional process, i.e., computing the reduct one by one based on the same searching steps in terms of different radii, our accelerator will be realized through considering the variation of radius, that is, the reduct obtained by using previous radius will guide the searching of reduct in terms of current radius.

The main contributions of this work can be attributed to the following: 1) to improve the discriminating performance of neighborhood relation, a supervised neighborhood relation will be proposed; 2) to obtain the reduct with higher generalization performance, the supervised neighborhood based attribute reduction will be defined and explored; 3) to overcome the limitations of obtaining supervised neighborhood based reduct in terms of multiple different radii, an acceleration strategy will be proposed.

The rest of this paper is organized as follows. The basic notions of neighborhood rough set are introduced in Section 2. The supervised neighborhood relation, supervised neighborhood based attribute reduction and our acceleration strategy will be proposed in Section 3. The experimental results and detailed comparisons are discussed in Section 4. Section 5 includes the conclusion and perspectives for future researches.

Section snippets

Neighborhood relation

Generally, a decision system can be denoted as DS=<U,AT,d>, in which U is the set of samples, AT is the set of condition attributes and d is the decision attribute. xiU and bAT, b(xi) indicates the value of xi over condition attribute b, d(xi) indicates the value of xi over decision attribute d, i.e., the label of xi.

Given a decision system DS, assuming that the values of decision attribute are categorical, then an equivalence relation over d can be defined asIND({d})={(xi,xj)U×U:d(xi)=d(xj

Supervised neighborhood based rough set and attribute reduction

Following the traditional neighborhood relation shown in Eqs. (2) and (3), the size of the neighborhood is directly determined by the given radius and the distance between two samples. However, this method may be powerless in characterizing whether the samples with different labels are similar or not. The reason may attribute to the fact that two samples with different labels will fall into the same neighborhood if an unsuitable radius is employed. To further improve the discriminating

Experimental analyses

To demonstrate the effectiveness of our supervised neighborhood strategy and accelerator, 12 UCI data sets are selected to conduct the experiments. The details of these data are shown in Table 1.

In the context of this paper, not only is 5-fold cross validation employed, but also 20 different intra-class radii δI are used, they are 0.05, 0.1, ⋯, 1, the corresponding inter-class radii δO are set to be “0.5δI”, “0.6δI”, “0.7δI”, “0.8δI”, “0.9δI”.

Moreover, two types of the neighborhood based

Conclusions and future perspectives

In this paper, the framework of supervised neighborhood has been introduced into attribute reduction. Different from previous researches, our supervised neighborhood is constructed by using intra-class radius and inter-class radius through considering the information offered by label. Furthermore, to find the corresponding reducts over multiple radii, an acceleration strategy has been designed. Different from the naive strategy to obtain reducts in terms of multiple different radii, our

Declaration of Competing Interest

We wish to confirm that there are no known conflicts of interest associated with this publication and there has been no significant financial support for this work that could have influenced its outcome.

Acknowledgement

This work is supported by the Natural Science Foundation of China (Nos. 61572242, 61906078), the Key Laboratory of Data Science and Intelligence Application, Fujian Province University (No. D1901) and the Postgraduate Research & Practice Innovation Program of Jiangsu Province (No. KYCX19_1715).

References (76)

  • Z.H. Jiang et al.

    Accelerator for multi-granularity attribute reduction

    Knowl.-Based Syst.

    (2019)
  • H.R. Ju et al.

    Cost-sensitive rough set approach

    Inf. Sci.

    (2016)
  • Y.C. Ko et al.

    An evidential analytics for buried information in big data samples: case study of semiconductor manufacturing

    Inf. Sci.

    (2019)
  • J.H. Li et al.

    Concept learning via granular computing: a cognitive viewpoint

    Inf. Sci.

    (2015)
  • J.H. Li et al.

    A comparative study of multigranulation rough sets and concept lattices via rule acquisition

    Knowl.-Based Syst.

    (2016)
  • K.Y. Liu et al.

    Rough set based semi-supervised feature selection via ensemble selector

    Knowl.-Based Syst.

    (2019)
  • K.Y. Liu et al.

    An efficient selector for multi-granularity attribute reduction

    Inf. Sci.

    (2019)
  • F. Min et al.

    Test-cost-sensitive attribute reduction

    Inf. Sci.

    (2011)
  • F. Min et al.

    Feature selection with test cost constraint

    Int. J. Approx. Reason.

    (2014)
  • Y.H. Qian et al.

    An efficient accelerator for attribute reduction from incomplete data in rough set framework

    Pattern Recognit.

    (2011)
  • Y.H. Qian et al.

    Positive approximation: an accelerator for attribute reduction in rough set theory

    Artif. Intell.

    (2010)
  • Y.H. She et al.

    A multiple-valued logic approach for multigranulation rough set model

    Int. J. Approx. Reason.

    (2017)
  • R. Susmaga et al.

    Generation of rough sets reducts and constructs based on inter-class and intra-class information

    Fuzzy Sets Syst.

    (2015)
  • C.Z. Wang et al.

    Fuzzy rough set-based attribute reduction using distance measures

    Knowl.-Based Syst.

    (2019)
  • W.Z. Wu et al.

    On rule acquisition in incomplete multi-scale decision tables

    Inf. Sci.

    (2017)
  • W. Wei et al.

    Compacted decision tables based attribute reduction

    Knowl.-Based Syst.

    (2015)
  • W.H. Xu et al.

    Generalized multigranulation double-quantitative decision-theoretic rough set

    Knowl.-Based Syst.

    (2016)
  • W.H. Xu et al.

    A novel approach to information fusion in multi-source datasets: a granular computing viewpoint

    Inf. Sci.

    (2017)
  • S.P. Xu et al.

    Prediction of protein structural classed by decreasing nearest neighbor error rate

  • S.P. Xu et al.

    Multi-label learning with label-specific feature reduction

    Knowl.-Based Syst.

    (2016)
  • X. Yang et al.

    A sequential three-way approach to multi-class decision

    Int. J. Approx. Reason.

    (2019)
  • X.B. Yang et al.

    Pseudo-label neighborhood rough set: measures and attribute reductions

    Int. J. Approx. Reason.

    (2019)
  • X.B. Yang et al.

    Test cost sensitive multigranulation rough set: model and minimal cost selection

    Inf. Sci.

    (2013)
  • X.B. Yang et al.

    Updating multigranulation rough approximations with increasing of granular structures

    Knowl.-Based Syst.

    (2014)
  • X.B. Yang et al.

    Ensemble selector for attribute reduction

    Appl. Soft Comput.

    (2018)
  • Y.Y. Yao et al.

    Class-specific attribute reducts in rough set theory

    Inf. Sci.

    (2017)
  • Y.Y. Yao et al.

    Discernibility matrix simplification for constructing attribute reducts

    Inf. Sci.

    (2009)
  • Q.H. Zhang et al.

    A dynamic three-way decision model based on the updating of attribute values

    Knowl.-Based Syst.

    (2018)
  • Cited by (0)

    View full text