Skip to main content
Log in

Class-specific cost-sensitive boosting weighted ELM for class imbalance learning

  • Regular Research Paper
  • Published:
Memetic Computing Aims and scope Submit manuscript

Abstract

Class imbalance problem happens when the training dataset contains significantly fewer instances of one class compared to another class. Traditional classification algorithms like extreme learning machine (ELM) and support vector machine (SVM) are biased towards the majority class. They minimize the least squares error due to which the minority class instances are usually misclassified. The classification algorithms like weighted ELM (WELM) and weighted SVM minimize the weighted least squares error to address the class imbalance problem effectively. The variants of WELM like boosting WELM, ensemble WELM etc. further improve the performance of WELM by employing ensemble method. This work proposes class-specific AdaC1, class-specific AdaC2 and class-specific AdaC3 algorithms for addressing the class imbalance problem more effectively. This work employs kernelized WELM as the component classifier to make the proposed ensemble. The proposed classifier ensembles are the variants of AdaC1, AdaC2 and AdaC3 algorithms. The cost-sensitive boosting classifiers AdaC1, AdaC2 and AdaC3 assign initial weights to the training instances without considering class skewness. This work assigns the initial weights to the training instances based on the class skewness. Moreover, the proposed ensemble rescales the weights assigned to the instances of each class after each iteration such that the total weight assigned to the instances belonging to each class remains equal. The proposed algorithms are assessed by employing the benchmark real-world imbalanced datasets downloaded from the KEEL dataset repository. The superiority of the proposed work over the other state-of-the-art ensemble methods for the class imbalance learning can be observed from the experimental results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. https://www.dei.unipd.it/node/2357.

References

  1. Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70(2):513–529

    Google Scholar 

  2. Huang GB, Zhou H, Ding X, Zhang R (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern Part B (Cybern) 42(2):513–529

    Article  Google Scholar 

  3. Li K, Kong X, Lu Z, Wenyin L, Yin J (2014) Boosting weighted ELM for imbalanced learning. Neurocomputing 128:15–21

    Article  Google Scholar 

  4. Xiao W, Zhang J, Li Y, Zhang S, Yang W (2017) Class-specific cost regulation extreme learning machine for imbalanced classification. Neurocomputing 261:70–82

    Article  Google Scholar 

  5. Zhang Y, Liu B, Cai J, Zhang S (2017) Ensemble weighted extreme learning machine for imbalanced data classification based on differential evolution. Neural Comput Appl 28:259–267

    Article  Google Scholar 

  6. Zong W, Huang GB, Chen Y (2013) Weighted extreme learning machine for imbalance learning. Neurocomputing 101:229–242

    Article  Google Scholar 

  7. Schapire RE, Singer Y (1999) Improved boosting algorithms using confidence-rated predictions. Mach Learn 37(3):297–336

    Article  MATH  Google Scholar 

  8. Sun Y, Kamel MS, Wong AKC, Wang Y (2007) Cost-sensitive boosting for classification of imbalanced data. Pattern Recogn 40(12):3358–3378

    Article  MATH  Google Scholar 

  9. Fan W, Stolfo SJ, Zhang J, Chan PK (1999) Adacost: misclassification cost-sensitive boosting. In: Proceedings of the sixteenth international conference on machine learning, ICML’99. Morgan Kaufmann, pp 97–105

  10. Ting KM (2000) A comparative study of cost-sensitive boosting algorithms. In: Proceedings of the seventeenth international conference on machine learning, ICML ’00. Morgan Kaufmann, pp 983–990

  11. Sun Y, Wong AKC, Wang Y (2005) Parameter inference of cost-sensitive boosting algorithms. Springer, Berlin, pp 21–30

    Google Scholar 

  12. Liu XY, Wu J, Zhou ZH (2009) Exploratory undersampling for class-imbalance learning. IEEE Trans Syst Man Cybern Part B (Cybern) 39(2):539–550

    Article  Google Scholar 

  13. Seiffert C, Khoshgoftaar TM, Hulse JV, Napolitano A (2010) Rusboost: a hybrid approach to alleviating class imbalance. IEEE Trans Syst Man Cybern Part A Syst Hum 40(1):185–197

    Article  Google Scholar 

  14. Nanni L, Fantozzi C, Lazzarini N (2015) Coupling different methods for overcoming the class imbalance problem. Neurocomputing 158(C):48–61

    Article  Google Scholar 

  15. Hu S, Liang Y, Ma L, He Y (2009) Msmote: improving classification performance when training data is imbalanced. In: 2009 second international workshop on computer science and engineering, vol 2, pp 13–17

  16. He H, Garcia EA (2009) Learning from imbalanced data. IEEE Trans Knowl Data Eng 21(9):1263–1284

    Article  Google Scholar 

  17. Brown I, Mues C (2012) An experimental comparison of classification algorithms for imbalanced credit scoring data sets. Expert Syst with Appl 39(3):3446–3453

    Article  Google Scholar 

  18. Krawczyk B, Galar M, Jele L, Herrera F (2016) Evolutionary undersampling boosting for imbalanced classification of breast cancer malignancy. Appl Soft Comput 38(C):714–726

    Article  Google Scholar 

  19. Galar M, Fernandez A, Barrenechea E, Bustince H, Herrera F (2012) A review on ensembles for the class imbalance problem: bagging-, boosting-, and hybrid-based approaches. IEEE Trans Syst Man Cybern Part C (Appl Rev) 42(4):463–484

    Article  Google Scholar 

  20. Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP (2002) Smote: synthetic minority over-sampling technique. J Artif Int Res 16(1):321–357

    MATH  Google Scholar 

  21. Yang X, Song Q, Wang Y (2007) A weighted support vector machine for data classification. Int J Pattern Recogn Artif Intell 21(05):961–976

    Article  Google Scholar 

  22. Dietterich TG (2000) Ensemble methods in machine learning. In: Multiple classifier systems. Springer, pp 1–15

  23. Chawla NV, Lazarevic A, Hall LO, Bowyer KW (2003) Smoteboost: Improving prediction of the minority class in boosting. In: Lavrač N, Gamberger D, Todorovski L, Blockeel H (eds) Knowledge discovery in databases: PKDD 2003, pp 107–119

  24. Deng W, Zheng Q, Chen L (2009) Regularized extreme learning machine. In: IEEE symposium on computational intelligence and data mining (CIDM 09), pp 389–395

  25. Fdez A, Fernandez A, Luengo J, Derrac J, Garcia S (2011) Keel data mining software tool: data set repository, integration of algorithms and experimental analysis framework. Multiple-valued logic. Soft Comput 17(2–3):255–287

    Google Scholar 

  26. Bradley AP (1997) The use of the area under the roc curve in the evaluation of machine learning algorithms. Pattern Recogn 30(7):1145–1159

    Article  Google Scholar 

  27. Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30

    MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Sanyam Shukla.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Raghuwanshi, B.S., Shukla, S. Class-specific cost-sensitive boosting weighted ELM for class imbalance learning. Memetic Comp. 11, 263–283 (2019). https://doi.org/10.1007/s12293-018-0267-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12293-018-0267-4

Keywords

Navigation