Skip to main content
SpringerLink
Log in

Optimal arrangements of hyperplanes for SVM-based multiclass classification

  • Regular Article
  • Published:
Advances in Data Analysis and Classification Aims and scope Submit manuscript

Abstract

In this paper, we present a novel SVM-based approach to construct multiclass classifiers by means of arrangements of hyperplanes. We propose different mixed integer (linear and non linear) programming formulations for the problem using extensions of widely used measures for misclassifying observations where the kernel trick can be adapted to be applicable. Some dimensionality reductions and variable fixing strategies are also developed for these models. An extensive battery of experiments has been run which reveal the powerfulness of our proposal as compared with other previously proposed methodologies.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  • Agarwal N, Balasubramanian VN, Jawahar C (2018) Improving multiclass classification by deep networks using DAGSVM and triplet loss. Pattern Recognit Lett 112:184–190

    Article  Google Scholar 

  • Allwein EL, Schapire RE, Singer Y (2001) Reducing multiclass to binary. Reducing multiclass to binary: a unifying approach for margin classifiers. J Mach Learn Res 1:113–141

    MathSciNet  MATH  Google Scholar 

  • Bagirov AM, Ugon J, Webb D, Ozturk G, Kasimbeyli R (2013) A novel piecewise linear classifier based on polyhedral conic and max-min separabilities. TOP 21(1):3–24

    Article  MathSciNet  Google Scholar 

  • Bahlmann C, Haasdonk B, Burkhardt H (2002) On-line handwriting recognition with support vector machines-a kernel approach. In: Eighth international workshop on frontiers in handwriting recognition, pp 49–54

  • Benders JF (1962) Partitioning procedures for solving mixed-variables programming problems. Numerische Mathematik 4(1):238–252

    Article  MathSciNet  Google Scholar 

  • Benítez-Peña S, Blanquero R, Carrizosa E, Ramírez-Cobo P (2018) On support vector machines under a multiple-cost scenario. Adv Data Anal Classif. https://doi.org/10.1007/s11634-018-0330-5

    Article  MATH  Google Scholar 

  • Bennett KP, Demiriz A (1999) Semi-supervised support vector machines. Adv Neural Inf Process Syst 11:368–374

    Google Scholar 

  • Blanco V, Ben Ali S, Puerto J (2014) Revisiting several problems and algorithms in continuous location with \(\ell _p\) norms. Comput Optim Appl 58(3):563–595

    Article  MathSciNet  Google Scholar 

  • Blanco V, Puerto J, Salmerón R (2018) Locating hyperplanes to fitting set of points: a general framework. Comput Oper Res 95:172–193

    Article  MathSciNet  Google Scholar 

  • Blanco V, Japón A, Puerto J (2019) Optimal arrangements of hyperplanes for multiclass classification. arXiv preprint: arXiv:1810.09167

  • Blanco V, Puerto J, Rodríguez-Chía A M (2019) On \(\ell _p \)-Support Vector Machines and Multidimensional Kernels. arXiv preprint: arXiv:1711.10332

  • Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297

    MATH  Google Scholar 

  • Cover T, Hart P (1967) Nearest neighbor pattern classification. IEEE Trans Inf Theory 13:21–27

    Article  Google Scholar 

  • Crammer K, Singer Y (2001) On the algorithmic implementation of multiclass kernel-based vector machines. J Mach Learn Res 2:265–292

    MATH  Google Scholar 

  • Dietterich TG, Bakiri G (1995) Solving multiclass learning problems via error-correcting output codes. J Artif Intell Res 2:263–286

    Article  Google Scholar 

  • Geoffrion AM (1972) Generalized Benders decomposition. J Optim Theory Appl 10(4):237–260

    Article  MathSciNet  Google Scholar 

  • Ghaddar B, Naoum-Sawaya J (2018) High dimensional data classification and feature selection using support vector machines. Eur J Oper Res 265(3):993–1004

    Article  MathSciNet  Google Scholar 

  • Guermeur Y, Monfrini E (2011) A quadratic loss multi-class SVM for which a radius-margin bound applies. Informatica 22(1):73–96

    MathSciNet  MATH  Google Scholar 

  • Harris T (2013) Quantitative credit risk assessment using support vector machines: broad versus narrow default definitions. Expert Syst Appl 40(11):4404–4413

    Article  Google Scholar 

  • Horn D, Demircioglu A, Bischl B, Glasmachers T, Weihs C (2016) A comparative study on large scale kernelized support vector machines. Adv Data Anal Classif 12(4):867–883

    Article  MathSciNet  Google Scholar 

  • Ikeda K, Murata N (2005) Geometrical properties of Nu support vector machines with different norms. Neural Comput 17(11):2508–2529

    Article  MathSciNet  Google Scholar 

  • Ikeda K, Murata N (2005) Effects of norms on learning properties of support vector machines. Proc IEEE Int Conf Acoust Speech Signal Process 5:241–244

    Google Scholar 

  • Kašćelan V, Kašćelan L, Novović Burić M (2016) A nonparametric data mining approach for risk prediction in car insurance: a case study from the montenegrin market. Econ Res Ekonomska istraživanja 29(1):545–558

    Article  Google Scholar 

  • Labbé M, Martínez-Merino LI, Rodríguez-Chía AM (2018) Mixed integer linear programming for feature selection in support vector machine. Discrete Appl Math 261:276–304

    Article  MathSciNet  Google Scholar 

  • Lauer F, Guermeur Y (2011) MSVMpack: a multi-class support vector machine package. J Mach Learn Res 12:2269–2272

    MathSciNet  MATH  Google Scholar 

  • Lee Y, Lin Y, Wahba G (2004) Multicategory support vector machines: theory and application to the classification of microarray data and satellite radiance data. J Am Stat Assoc 99(465):67–81

    Article  MathSciNet  Google Scholar 

  • Lewis D.D (1998) Naive (bayes) at forty: the independence assumption in information retrieval. In: European conference on machine learning, pp 4–15

    Chapter  Google Scholar 

  • Lichman M (2013) UCI machine learning repository. https://archive.ics.uci.edu. Accessed Oct 2018

  • López J, Maldonado S, Carrasco M (2018) Double regularization methods for robust feature selection and SVM classification via DC programming. Inf Sci 429:377–389

    Article  MathSciNet  Google Scholar 

  • Majid A, Ali S, Iqbal M, Kausar N (2014) Prediction of human breast and colon cancers from imbalanced data using nearest neighbor and support vector machines. Comput Methods Progr Biomed 113(3):792–808

    Article  Google Scholar 

  • Maldonado S, Pérez J, Weber R, Labbé M (2014) Feature selection for support vector machines via mixed integer linear programming. Inf Sci 279:163–175

    Article  MathSciNet  Google Scholar 

  • Mangasarian OL (1999) Arbitrary-norm separating plane. Oper Res Lett 24(1–2):15–23

    Article  MathSciNet  Google Scholar 

  • Meyer D, Dimitriadou E, Hornik K, Weingessel A, Leisch F (2017) e1071: misc functions of the department of statistics, probability theory group (Formerly: E1071), TU Wien. R package version 1.6-8. https://CRAN.R-project.org/package=e1071. Accessed Oct 2018

  • Ortigosa-Hernández J, Inza I, Lozano JA (2016) Semisupervised multiclass classification problems with scarcity of labeled data: a theoretical study. IEEE Trans Neural Netw Learn Syst 27(12):2602–2614

    Article  MathSciNet  Google Scholar 

  • Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825–2830

    MathSciNet  MATH  Google Scholar 

  • Platt JC, Cristianini N, Shawe-Taylor J (2000) Large margin DAGs for multiclass classification. In: Solla SA, Leen TK, Mülle K (eds) Advances in neural information processing systems, vol 12. The MIT Press, Cambridge, pp 547–553

    Google Scholar 

  • Radhimeenakshi S (2016) Classification and prediction of heart disease risk using data mining techniques of support vector machine and artificial neural network. In: International conference on computing for sustainable global development, INDIACom, pp 3107–3111

  • Tang X, Xu A (2016) Multi-class classification using kernel density estimation on k-nearest neighbours. Electron Lett 52(8):600–602

    Article  Google Scholar 

  • Üney F, Türkay M (2006) A mixed-integer programming approach to multi-class data classification problem. Eur J Oper Res 173(3):910–920

    Article  MathSciNet  Google Scholar 

  • van den Burg GJJ, Groenen PJF (2016) GenSVM: a generalized multiclass support vector machine. J Mach Learn Res 17(225):1–42

    MathSciNet  MATH  Google Scholar 

  • Weston J, Watkins C (1999) Support vector machines for multi-class pattern recognition. In: European symposium on artificial neural networks, pp 219–224

Download references

Acknowledgements

The authors were partially supported by the research Project MTM2016-74983-C2-1-R (MINECO, Spain). The first author has been also supported by Project PP2016-PIP06 (Universidad de Granada) and the research group SEJ-534 (Junta de Andalucía). We would also like to thank the three anonymous referees for their helpful and constructive comments that greatly contributed to improving the final version of the paper.

Author information

Authors and Affiliations

  1. IEMath-GR, Universidad de Granada, 18001, Granada, Spain

    Víctor Blanco

  2. IMUS, Universidad de Sevilla, 41012, Sevilla, Spain

    Alberto Japón & Justo Puerto

Authors
  1. Víctor Blanco
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alberto Japón
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Justo Puerto
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Víctor Blanco.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Blanco, V., Japón, A. & Puerto, J. Optimal arrangements of hyperplanes for SVM-based multiclass classification. Adv Data Anal Classif 14, 175–199 (2020). https://doi.org/10.1007/s11634-019-00367-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11634-019-00367-6

Keywords

Mathematics Subject Classification

Search

Navigation