Skip to main content

Advertisement

SpringerLink
Log in

Quantum-Behaved Particle Swarm Optimization for Parameter Optimization of Support Vector Machine

  • Published:
Journal of Classification Aims and scope Submit manuscript

Abstract

Support vector machine (SVM) parameters such as penalty parameter and kernel parameters have a great influence on the complexity and accuracy of SVM model. In this paper, quantum-behaved particle swarm optimization (QPSO) has been employed to optimize the parameters of SVM, so that the classification error can be reduced. To evaluate the proposed model (QPSO-SVM), the experiment adopted seven standard classification datasets which are obtained from UCI machine learning data repository. For verification, the results of the QPSO-SVM algorithm are compared with the standard PSO, and genetic algorithm (GA) which is one of the well-known optimization algorithms. Moreover, the results of QPSO are compared with the grid search, which is a conventional method of searching parameter values. The experimental results demonstrated that the proposed model is capable to find the optimal values of the SVM parameters. The results also showed lower classification error rates compared with standard PSO and GA algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Ali, S., & Smith, K. (2003). Automatic parameter selection for polynomial kernel. In Proceedings of IEEE International Conference on Information Reuse and Integration, (IRI 2003), Lens, France, October, (Vol. 27-29 pp. 243–249).

  • Bashir, Z., & El-Hawary, M. (2009). Applying wavelets to short-term load forecasting using PSO-based neural networks. IEEE Transactions on Power Systems, 24(1), 20–27.

    Article  Google Scholar 

  • Ben-Hur, A., Ong, C.S., Sonnenburg, S., Schölkopf, B., Rätsch, G. (2008). Support vector machines and kernels for computational biology. PLoS Comput Biol, 4(10), e1000173.

    Article  Google Scholar 

  • Blake, C., & Merz, C.J. (1998). {UCI} repository of machine learning databases repository of machine learning databases.

  • Byvatov, E., & Schneider, G. (2002). Support vector machine applications in bioinformatics. Applied Bioinformatics, 2(2), 67–77.

    Google Scholar 

  • Cai, Y., Sun, J., Wang, J., Ding, Y., Tian, N., Liao, X., et al. (2008). Optimizing the codon usage of synthetic gene with QPSO algorithm. Journal of Theoretical Biology, 254(1), 123–127.

    Article  MathSciNet  MATH  Google Scholar 

  • Chander, A., Chatterjee, A., Siarry, P. (2011). A new social and momentum component adaptive PSO algorithm for image segmentation. Expert Systems with Applications, 38(5), 4998–5004.

    Article  Google Scholar 

  • Chapelle, O., Vapnik, V., Bousquet, O. (2002). Choosing multiple parameters for support vector machines. Machine Learning, 46(1-3), 131–159.

    Article  MATH  Google Scholar 

  • Clerc, M., & Kennedy, J. (2002). The particle swarm-explosion, stability, and convergence in a multidimensional complex space. IEEE Transactions on Evolutionary Computation, 6(1), 58–73.

    Article  Google Scholar 

  • Dudani, S.A. (1976). The distance-weighted k-nearest-neighbor rule. IEEE Transactions on Systems, Man, and Cybernetics, SMC-6(4), 325–327.

    Article  Google Scholar 

  • Eberhart, R., & Kennedy, J. (1995). A new optimizer using particle swarm theory. In Proceedings of the Sixth International Symposium on Micro Machine and Human Science, 1995. MHS’95. pp. 39–43.

  • Friedrichs, F., & Igel, C. (2005). Evolutionary tuning of multiple SVM parameters. Neurocomputing, 64, 107–117.

    Article  Google Scholar 

  • Hassan, R., Cohanim, B., De, Weck, Venter, O. (2005). G A comparison of particle swarm optimization and the genetic algorithm. In Proceedings of the 1st AIAA multidisciplinary design optimization specialist conference, Honolulu, Hawaii, April 23-26 (pp. 1–13).

  • He, H., & Garcia, E.A. (2009). Learning from imbalanced data. IEEE Transactions on Knowledge and Data Engineering, 21(9), 1263–1284.

    Article  Google Scholar 

  • Huang, C.L., & Wang, C.J. (2006). A GA-based feature selection and parameters optimizationfor support vector machines. Expert Systems with Applications, 31(2), 231–240.

    Article  Google Scholar 

  • Kecman, V. (2001). Learning and soft computing: support vector machines, neural networks, and fuzzy logic models. Cambridge: MIT Press.

    MATH  Google Scholar 

  • Keerthi, S.S., & Lin, C.J. (2003). Asymptotic behaviors of support vector machines with Gaussian kernel. Neural Computation, 15(7), 1667–1689.

    Article  MATH  Google Scholar 

  • Kennedy, J. (2004). Probability and dynamics in the particle swarm. In Congress on Evolutionary Computation.

  • Kennedy, J. (2005). Dynamic-probabilistic particle swarms. In Proceedings of the 7th Annual Conference On Genetic And Evolutionary Computation, pp. 201–207.

  • Kennedy, J. (2010). Particle swarm optimization. In Encyclopedia of Machine Learning. Springer, pp. 760–766.

  • Krohling, R.A. (2004). Gaussian swarm: a novel particle swarm optimization algorithm. In Proceedings of IEEE Conference on Cybernetics and Intelligent Systems, (Vol. 1 pp. 372–376).

  • Krohling, R.A., & Santos Coelho, L. dos. (2006). PSO-E: particle swarm with exponential distribution. In 2006 IEEE International Conference on Evolutionary Computation, pp 1428–1433.

  • Li, S., Wang, R., Hu, W., Sun, J. (2007). A new QPSO based BP neural network for face detection. In Fuzzy Information and Engineerin. Springer, pp. 355–363.

  • Liang, J.J., Qin, A.K., Suganthan, P.N. (2006). Comprehensive learning particle swarm optimizer for global optimization of multimodal functions. IEEE Transactions on Evolutionary Computation, 10(3), 281–295.

    Article  Google Scholar 

  • Lin, S.W., Ying, K.C., Chen, S.C., Lee, Z.J. (2008). Particle swarm optimization for parameter determination and feature selection of support vector machines. Expert Systems with Applications, 35(4), 1817–1824.

    Article  Google Scholar 

  • Liu, B., Wang, L., Jin, Y.H., Tang, F. (2005). Huang D X. Chaos, Solitons & Fractals, 25(5), 1261–1271.

    Article  Google Scholar 

  • Liu, J., Xu, W., Sun, J. (2005). Quantum-behaved particle swarm optimization with mutation operator. In Proceedings of the 17th IEEE International Conference on Tools with Artificial Intelligence (ICTAI’05).

  • Maitra, M., & Chatterjee, A. (2008). A hybrid cooperative–comprehensive learning based PSO algorithm for image segmentation using multilevel thresholding. Expert Systems with Applications, 34(2), 1341– 1350.

    Article  Google Scholar 

  • Merwe, D., Van der, Engelbrecht, A.P. (2003). Data clustering using particle swarm optimization. In The 2003 Congress on Evolutionary Computation(CEC’03), (Vol. 1 pp. 215–220).

  • Mikki, S.M., & Kishk, A.A. (2006). Quantum particle swarm optimization for electromagnetics. IEEE Transactions on Antennas and Propagation, 54(10), 2764–2775.

    Article  Google Scholar 

  • Miyatake, M., Veerachary, M., Toriumi, F., Fujii, N., Ko, H. (2011). Maximum power point tracking of multiple photovoltaic arrays: a PSO approach. IEEE Transactions on Aerospace and Electronic Systems, 47(1), 367–380.

    Article  Google Scholar 

  • Omkar, S., Khandelwal, R., Ananth, T., Naik, G.N. (2009). Quantum behaved Particle Swarm Optimization (QPSO) for multi-objective design optimization of composite structures. Expert Systems with Applications, 36(8), 11312–11322.

    Article  Google Scholar 

  • Pal, S.K., & Mitra, S. (1992). Multilayer perceptron, fuzzy sets, and classification. IEEE Transactions on Neural Networks, 3(5), 683–697.

    Article  Google Scholar 

  • Panda, S., & Padhy, N.P. (2008). Optimal location and controller design of STATCOM for power system stability improvement using PSO. Journal of the Franklin Institute, 345(2), 166–181.

    Article  MATH  Google Scholar 

  • Reynolds, C.W. (1987). Flocks, herds and schools: a distributed behavioral model. ACM Siggraph Computer Graphics, 21(4), 25–34.

    Article  Google Scholar 

  • Richer, T.J., & Blackwell, T.M. (2006). The Lévy particle swarmvy particle swarm. In 2006 IEEE International Conference on Evolutionary Computation, pp. 808–815.

  • Santos Coelho, L. dos. (2010). Gaussian quantum-behaved particle swarm optimization approaches for constrained engineering design problems. Expert Systems with Applications, 37(2), 1676–1683.

    Article  Google Scholar 

  • Santos Coelho, L. dos, & Krohling, R.A. (2005). Predictive controller tuning using modified particle swarm optimization based on Cauchy and Gaussian distributions. In Soft Computing: Methodologies and Applications. Springer, pp. 287–298.

  • Scholköpf, B., & Smola, A.J. (2001). Learning with kernels: support vector machines, regularization, optimization, and beyond. Cambridge: MIT Press.

    Google Scholar 

  • Subasi, A. (2013). Classification of EMG signals using PSO optimized SVM for diagnosis of neuromuscular disorders. Computers in Biology and Medicine, 43(5), 576–586.

    Article  Google Scholar 

  • Sun, J., Fang, W., Palade, V., Wu, X. (2011). Quantum-behaved particle swarm optimization with Gaussian distributed local attractor point. Applied Mathematics and Computation, 218(7), 3763–3775.

    Article  MATH  Google Scholar 

  • Sun, J., Fang, W., Wu, X., Palade, V. (2012). Quantum-behaved particle swarm optimization: analysis of individual particle behavior and parameter selection. Evolutionary Computation, 20(3), 349–393.

    Article  Google Scholar 

  • Sun, J., & Feng, B. (2004). Particle swarm optimization with particles having quantum behavior. In Congress on Evolutionary Computation.

  • Sun, J., Xu, W., Feng, B. (2004). A global search strategy of quantum-behaved particle swarm optimization. In Proceedings of IEEE Conference on Cybernetics and Intelligent Systems, (Vol. 1 pp. 111–116).

  • Sun, J., Xu, W., Feng, B. (2005). Adaptive parameter control for quantum-behaved particle swarm optimization on individual level. In Proceedings of IEEE International Conference on Systems, Man and Cybernetics, (Vol. 4 pp. 3049–3054).

  • Tharwat, A. (2016). Linear vs. quadratic discriminant analysis classifier: a tutorial. International Journal of Applied Pattern Recognition, 3(2), 145–180.

    Article  Google Scholar 

  • Tharwat, A. (2016). Principal component analysis-a tutorial. International Journal of Applied Pattern Recognition, 3(3), 197–240.

    Article  Google Scholar 

  • Tharwat, A., Gabel, T., Hassanien, A.E. (2017). Parameter optimization of support vector machine using dragonfly algorithm. In International Conference on Advanced Intelligent Systems and Informatics, pp. 309–319.

  • Tharwat, A., Gaber, T., Ibrahim, A. (2017). Linear discriminant analysis: a detailed tutorial. AI Communications, 30(2), 169–190.

    Article  MathSciNet  Google Scholar 

  • Tharwat, A., & Hassanien, A.E. (2018). Chaotic antlion algorithm for parameter optimization of support vector machine. Applied Intelligence, 48(3), 670–686.

    Article  Google Scholar 

  • Tharwat, A., Hassanien, A.E., Elnaghi, B.E. (2016). A BA-based algorithm for parameter optimization of support vector machine. Pattern Recognition Letters.

  • Tharwat, A., Hemedan, A.A., Hassanien, A.E., Thomas G. (2018). A biometric-based model for fish species classification. Fisheries Research, 204, 324–336.

    Article  Google Scholar 

  • Tharwat, A., & Moemen, Y.S. (2017). Classification of toxicity effects of biotransformed hepatic drugs using whale optimized support vector machines. Journal of Biomedical Informatics, 68, 132–149.

    Article  Google Scholar 

  • Wang, G., & Guo, L. (2013). A novel hybrid bat algorithm with harmony search for global numerical optimization. Journal of Applied Mathematics.

  • Wang, L. (2005). Support vector machines: theory and applications, vol. 177. Berlin: Springer.

    Book  Google Scholar 

  • Wu, C.H., Tzeng, G.H., Lin, R.H. (2009). A novel hybrid genetic algorithm for kernel function and parameter optimization in support vector regression. Expert Systems with Applications, 36(3), 4725–4735.

    Article  Google Scholar 

  • Xi, M., & Sun, J. (2008). An improved quantum-behaved particle swarm optimization algorithm with weighted mean best position. Applied Mathematics and Computation, 205(2), 751–759.

    Article  MATH  Google Scholar 

  • Xinchao, Z. (2010). A perturbed particle swarm algorithm for numerical optimization. Applied Soft Computing, 10(1), 119–124.

    Article  Google Scholar 

  • Yang, X.S. (2014). Nature-inspired optimization algorithms, 1st edn. Amsterdam: Elsevier.

    MATH  Google Scholar 

  • Zhang, X., & Chen, X. (2010). An ACO-based algorithm for parameter optimization of support vector machines. Expert Systems with Applications, 37(9), 6618–6628.

    Article  Google Scholar 

  • Zhang, Y., & Zhang, P. (2015). Machine training and parameter settings with social emotional optimization algorithm for support vector machine. Pattern Recognition Letters, 54, 36–42.

    Article  Google Scholar 

  • Zhao, M., Fu, C., Ji, L., Tang, K. (2011). Feature selection and parameter optimization for support vector machines: a new approach based on genetic algorithm with feature chromosomes. Expert Systems with Applications, 38(5), 5197–5204.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Computer Science and Engineering, Frankfurt University of Applied Sciences, 60318, Frankfurt am Main, Germany

    Alaa Tharwat

  2. Faculty of Computers and Information, Cairo University, Giza, Egypt

    Aboul Ella Hassanien

  3. Scientific Research Group in Egypt (SRGE), Giza, Egypt

    Aboul Ella Hassanien

Authors
  1. Alaa Tharwat
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aboul Ella Hassanien
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alaa Tharwat.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tharwat, A., Hassanien, A.E. Quantum-Behaved Particle Swarm Optimization for Parameter Optimization of Support Vector Machine. J Classif 36, 576–598 (2019). https://doi.org/10.1007/s00357-018-9299-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00357-018-9299-1

Keywords

Search

Navigation