Skip to main content
SpringerLink
Log in

Competitive swarm optimizer with mutated agents for finding optimal designs for nonlinear regression models with multiple interacting factors

  • Regular Research Paper
  • Published:
Memetic Computing Aims and scope Submit manuscript

Abstract

This paper proposes a novel enhancement for competitive swarm optimizer (CSO) by mutating loser particles (agents) from the swarm to increase the swarm diversity and improve space exploration capability, namely competitive swarm optimizer with mutated agents (CSO-MA). The selection mechanism is carried out so that it does not retard the search if agents are exploring in promising areas. Simulation results show that CSO-MA has a better exploration–exploitation balance than CSO and generally outperforms CSO, which is one of the state-of-the-art metaheuristic algorithms for optimization. We show additionally that it also generally outperforms swarm based types of algorithms and an exemplary and popular non-swarm based algorithm called Cuckoo search, without requiring a lot more CPU time. We apply CSO-MA to find a c-optimal approximate design for a high-dimensional optimal design problem when other swarm algorithms were not able to. As applications, we use the CSO-MA to search various optimal designs for a series of high-dimensional statistical models. The proposed CSO-MA algorithm is a general-purpose optimizing tool and can be directly amended to find other types of optimal designs for nonlinear models, including optimal exact designs under a convex or non-convex criterion.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Bansal JC, Singh PK, Saraswat M, Verma A, Jadon SS, Abraham A (2011) Inertia weight strategies in particle swarm optimization. In: 2011 third world congress on Nature and biologically inspired computing (NaBIC). IEEE, pp 633–640

  2. Berger MPF, Wong WK (2005) Applied optimal designs. Wiley, Chichester

    MATH  Google Scholar 

  3. Berger MPF, Wong WK (2009) An introduction to optimal designs for social and biomedical research. Wiley, Chichester

    MATH  Google Scholar 

  4. Campos M, Krohling RA, Enriquez I (2014) Bare bones particle swarm optimization with scale matrix adaptation. IEEE Trans Cybern 44(9):1567–1578

    Google Scholar 

  5. Carlisle A, Dozier G (2000) Adapting particle swarm optimization to dynamic environments. Int Conf Artif Intell 1:429–434

    Google Scholar 

  6. Chen W-N, Zhang J, Lin Y, Chen N, Zhan Z-H, Chung HS-H, Li Y, Shi Y-H (2013) Particle swarm optimization with an aging leader and challengers. IEEE Trans Evolut Comput 17(2):241–258

    Google Scholar 

  7. Chen X, Ong Y-S, Lim M-H, Tan KC (2011) A multi-facet survey on memetic computation. IEEE Trans Evolut Comput 15(5):591–607

    Google Scholar 

  8. Cheng R, Jin Y (2015) A competitive swarm optimizer for large scale optimization. IEEE Trans Cybern 45(2):191–204

    Google Scholar 

  9. Chi R, Su Y, Zhang D, Chi X, Zhang H (2019) A hybridization of cuckoo search and particle swarm optimization for solving optimization problems. Neural Comput Appl 31(1):653–670

    Google Scholar 

  10. Chi Y, Sun F, Jiang L, Yu C, Zhang P (2012) Elastic boundary for particle swarm optimization. In: International conference in swarm intelligence. Springer, pp 125–132

  11. Deb K, Pratap A, Agarwal S, Meyarivan TAMT (2002) A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans Evolut Comput 6(2):182–197

    Google Scholar 

  12. Eberhart RC, Shi Y (2000) Comparing inertia weights and constriction factors in particle swarm optimization. In: Proceedings of the 2000 congress on evolutionary computation, vol 1. IEEE, pp 84–88

  13. Eberhart RC, Kennedy J (1995) Particle swarm optimization. In: Proceedings of the IEEE international conference on neural networks, vol 4, pp 1942–1948

  14. Eberhart RC, Shi Y (1998) Comparison between genetic algorithms and particle swarm optimization. In: International conference on evolutionary programming. Springer, pp 611–616

  15. Gang M, Wei Z, Xiaolin C (2012) A novel particle swarm optimization algorithm based on particle migration. Appl Math Comput 218(11):6620–6626

    MathSciNet  MATH  Google Scholar 

  16. Gao L, Qian W, Li X, Wang J (2010) Application of memetic algorithm in assembly sequence planning. Int J Adv Manuf Technol 49(9–12):1175–1184

    Google Scholar 

  17. Ghamisi P, Benediktsson JA (2015) Feature selection based on hybridization of genetic algorithm and particle swarm optimization. IEEE Geosci Remote Sens Lett 12(2):309–313

    Google Scholar 

  18. Shenkai G, Cheng R, Jin Y (2018) Feature selection for high-dimensional classification using a competitive swarm optimizer. Soft Comput 22(3):811–822

    Google Scholar 

  19. Haupt RL, Haupt SE (2004) Practical genetic algorithms. Wiley, Hoboken

    MATH  Google Scholar 

  20. Higashi N, Iba H (2003) Particle swarm optimization with Gaussian mutation. In: Swarm intelligence symposium, 2003. SIS’03. Proceedings of the 2003 IEEE. IEEE, pp 72–79

  21. Ye WLH, Li Z (2013) Convergence analysis of particle swarm optimizer and its improved algorithm based on velocity differential evolution. Comput Intell Neurosci 2013:7

    Google Scholar 

  22. Hsieh S-T, Sun T-Y, Liu C-C, Tsai S-J (2008) Solving large scale global optimization using improved particle swarm optimizer. In: Evolutionary computation, 2008. CEC 2008 (IEEE world congress on computational intelligence). IEEE, pp 1777–1784

  23. Ishaque K, Salam Z (2013) A deterministic particle swarm optimization maximum power point tracker for photovoltaic system under partial shading condition. IEEE Trans Ind Electron 60(8):3195–3206

    Google Scholar 

  24. Ishaque K, Salam Z, Amjad M, Mekhilef S (2012) An improved particle swarm optimization (PSO)-based MPPT for PV with reduced steady-state oscillation. IEEE Trans Power Electron 27(8):3627–3638

    Google Scholar 

  25. Kalantzis G, Apte A, Radke R, Jackson A (2013 ) A reduced order memetic algorithm for constraint optimization in radiation therapy treatment planning. In: 2013 14th ACIS international conference on software engineering, artificial intelligence, networking and parallel/distributed computing. IEEE, pp 225–230

  26. Kennedy J (1999) Small worlds and mega-minds: effects of neighborhood topology on particle swarm performance. In: Proceedings of the 1999 congress on evolutionary computation, 1999. CEC 99, vol 3. IEEE, pp 1931–1938

  27. Kennedy J, Mendes R (2002) Population structure and particle swarm performance. In: Proceedings of the 2002 congress on evolutionary computation, 2002. CEC’02, vol 2. IEEE, pp 1671–1676

  28. Krasnogor N, Smith J (2005) A tutorial for competent memetic algorithms: model, taxonomy, and design issues. IEEE Trans Evol Comput 9(5):474–488

    Google Scholar 

  29. Kumarappan N, Arulraj R (2016) Optimal installation of multiple DG units using competitive swarm optimizer (CSO) algorithm. In: 2016 IEEE congress on evolutionary computation (CEC). IEEE, pp 3955–3960

  30. Lall S, Jaggi S, Varghese E, Varghese C, Bhowmik A (2018) An algorithmic approach to construct d-optimal saturated designs for logistic model. J Stat Comput Simul 88(6):1191–1199

    MathSciNet  MATH  Google Scholar 

  31. Larsen RB, Jouffroy J, Lassen B (2016) On the premature convergence of particle swarm optimization. In: 2016 European control conference (ECC), pp 1922–1927

  32. Leung Y-W, Wang Y (2001) An orthogonal genetic algorithm with quantization for global numerical optimization. IEEE Trans Evol Comput 5(1):41–53

    Google Scholar 

  33. Li X, Yao X (2012) Cooperatively coevolving particle swarms for large scale optimization. IEEE Trans Evol Comput 16(2):210–224

    Google Scholar 

  34. Liang J-J, Suganthan PN (2005) Dynamic multi-swarm particle swarm optimizer with local search. In: The 2005 IEEE congress on evolutionary computation, 2005, vol 1. IEEE, pp 522–528

  35. Liu B, Wang L, Jin Y-H, Tang F, Huang D-X (2005) Improved particle swarm optimization combined with chaos. Chaos Solitons Fractals 25(5):1261–1271

    MATH  Google Scholar 

  36. Liu C, Wen-Bo D, Wang W-X (2014) Particle swarm optimization with scale-free interactions. PLoS ONE 9(5):e97822

    Google Scholar 

  37. McDermott J (2020) When and why metaheuristics researchers can ignore “no free lunch” theorems. SN Comput Sci (in press)

  38. Meng K, Wang HG, Dong ZY, Wong KP (2010) Quantum-inspired particle swarm optimization for valve-point economic load dispatch. IEEE Trans Power Syst 25(1):215–222

    Google Scholar 

  39. Mohapatra P, Das KN, Roy S (2017) A modified competitive swarm optimizer for large scale optimization problems. Appl Soft Comput 59:340–362

    Google Scholar 

  40. Moore J, Chapman R (1999) Application of particle swarm to multiobjective optimization. Department of Computer Science and Software Engineering, Auburn University

  41. Morris GM, Goodsell DS, Halliday RS, Huey R, Hart WE, Belew RK, Olson AJ et al (1998) Automated docking using a Lamarckian genetic algorithm and an empirical binding free energy function. J Comput Chem 19(14):1639–1662

    Google Scholar 

  42. Nakisa B, Rastgoo MN, Norodin MJ et al (2014) Balancing exploration and exploitation in particle swarm optimization on search tasking. Res J Appl Sci Eng Technol 8(12):1429–1434

    Google Scholar 

  43. Nezami OM, Bahrampour A, Jamshidlou P (2013) Dynamic diversity enhancement in particle swarm optimization (DDEPSO) algorithm for preventing from premature convergence. Procedia Comput Sci 24:54–65

    Google Scholar 

  44. Olorunda O, Engelbrecht AP (2008) Measuring exploration/exploitation in particle swarms using swarm diversity. In: 2008 IEEE congress on evolutionary computation (IEEE world congress on computational intelligence). IEEE, pp 1128–1134

  45. Ong Y-S, Lim M-H, Zhu N, Wong K-W (2006) Classification of adaptive memetic algorithms: a comparative study. IEEE Trans Syst Man Cybern Part B (Cybern) 36(1):141–152

    Google Scholar 

  46. Pázman A (1986) Foundations of optimum experimental design, vol 14. Springer, Berlin

    MATH  Google Scholar 

  47. Pehlivanoglu YV (2013) A new particle swarm optimization method enhanced with a periodic mutation strategy and neural networks. IEEE Trans Evolut Comput 17(3):436–452

    Google Scholar 

  48. Pukelsheim F (2006) Optimal design of experiments. SIAM, Philadelphia

    MATH  Google Scholar 

  49. Ratnaweera AS, Halgamuge SK, Watson HC (2004) Self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients. IEEE Trans Evolut Comput 8(3):240–255

    Google Scholar 

  50. Robinson J, Sinton S, Rahmat-Samii Y (2002) Particle swarm, genetic algorithm, and their hybrids: optimization of a profiled corrugated horn antenna. In: Antennas and propagation society international symposium, 2002. IEEE, vol 1. IEEE, pp 314–317

  51. Rodríguez-Torreblanca C, Rodríguez-Díaz JM (2007) Locally D- and c-optimal designs for poisson and negative binomial regression models. Metrika 66(2):161–172

    MathSciNet  MATH  Google Scholar 

  52. Ros R, Hansen N (2008 ) A simple modification in CMA-ES achieving linear time and space complexity. In: International conference on parallel problem solving from nature. Springer, pp 296–305

  53. Shi Y, Eberhart RC (1998) Parameter selection in particle swarm optimization. In: International conference on evolutionary programming. Springer, pp 591–600

  54. Shi Y, Eberhart RC (2001) Fuzzy adaptive particle swarm optimization. In: Proceedings of the 2001 congress on evolutionary computation, 2001, vol 1. IEEE, pp 101–106

  55. Song X, Zhang Y, Guo Y, Sun X, Wang Y (2019) Variable-size cooperative coevolutionary particle swarm optimization for feature selection on high-dimensional data. IEEE Trans Evolut Comput 14(8):1–14

    Google Scholar 

  56. Storn R, Price K (1997) Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces. J Glob Optim 11(4):341–359

    MathSciNet  MATH  Google Scholar 

  57. Suganthan PN (1999) Particle swarm optimiser with neighbourhood operator. In: Proceedings of the 1999 congress on evolutionary computation, 1999. CEC 99, vol 3. IEEE, pp 1958–1962

  58. Sun C, Ding J, Zeng J, Jin Y (2016) A fitness approximation assisted competitive swarm optimizer for large scale expensive optimization problems. Memet Comput 10(2):123–34

    Google Scholar 

  59. Sun J, Palade V, Xiao-Jun W, Fang W, Wang Z (2014) Solving the power economic dispatch problem with generator constraints by random drift particle swarm optimization. IEEE Trans Ind Inf 10(1):222–232

    Google Scholar 

  60. Tang K, Yáo X, Suganthan PN, MacNish C, Chen YP, Chen CM, Yang Z (2007) Benchmark functions for the CEC’2008 special session and competition on large scale global optimization, vol 24. Nature Inspired Computation and Applications Laboratory, USTC, China

  61. Taormina R, Chau K-W (2015) Data-driven input variable selection for rainfall-runoff modeling using binary-coded particle swarm optimization and extreme learning machines. J Hydrol 529:1617–1632

    Google Scholar 

  62. Tian J, Yu W, Xie S (2008) An ant colony optimization algorithm for image edge detection. In: IEEE congress on evolutionary computation, 2008. CEC 2008 (IEEE world congress on computational intelligence). IEEE, pp 751–756

  63. Trelea IC (2003) The particle swarm optimization algorithm: convergence analysis and parameter selection. Inf Process Lett 85(6):317–325

    MathSciNet  MATH  Google Scholar 

  64. Valenzuela J, Smith AE (2002) A seeded memetic algorithm for large unit commitment problems. J Heurist 8(2):173–195

    Google Scholar 

  65. Whitley D (1994) A genetic algorithm tutorial. Stat Comput 4(2):65–85

    Google Scholar 

  66. Willan AR, Pinto EM (2005) The value of information and optimal clinical trial design. Stat Med 24(12):1791–1806

    MathSciNet  Google Scholar 

  67. Worasucheep C (2010) A particle swarm optimization for high-dimensional function optimization. In: ECTI-CON2010: the 2010 ECTI international conference on electrical engineering/electronics, computer, telecommunications and information technology, pp 1045–1049

  68. Xinchao Z (2010) A perturbed particle swarm algorithm for numerical optimization. Appl Soft Comput 10(1):119–124

    Google Scholar 

  69. Xiong G, Shi D (2018) Orthogonal learning competitive swarm optimizer for economic dispatch problems. Appl Soft Comput 66:134–148

    Google Scholar 

  70. Xu G, Wu Z-H, Jiang M-Z (2015) Premature convergence of standard particle swarm optimisation algorithm based on Markov chain analysis. Int J Wirel Mobile Comput 9(4):377–382

    Google Scholar 

  71. Yang Q, Chen WN, Gu T, Zhang H, Yuan H, Kwong S, Zhang J (2019) A distributed swarm optimizer with adaptive communication for large-scale optimization. IEEE Trans Cybern. https://doi.org/10.1109/TCYB.2019.2904543

    Article  Google Scholar 

  72. Yang X-S, Deb S (2009) Cuckoo search via Lévy flights. In: 2009 World congress on nature & biologically inspired computing (NaBIC). IEEE, pp 210–214

  73. Yang Z, Tang K, Yao X (2008) Multilevel cooperative coevolution for large scale optimization. In: IEEE congress on evolutionary computation, 2008. CEC 2008 (IEEE world congress on computational intelligence). IEEE, pp 1663–1670

  74. Yang Z, Tang K, Yao X (2008) Self-adaptive differential evolution with neighborhood search. In: IEEE congress on evolutionary computation, 2008. CEC 2008 (IEEE world congress on computational intelligence). IEEE, pp 1110–1116

  75. Zhang Q, Cheng H, Ye Z, Wang Z (2017) A competitive swarm optimizer integrated with Cauchy and Gaussian mutation for large scale optimization. In: 2017 36th Chinese control conference (CCC). IEEE, pp 9829–9834

  76. Zhang W-X, Chen W-N, Zhang J (2016) A dynamic competitive swarm optimizer based-on entropy for large scale optimization. In: 2016 eighth international conference on advanced computational intelligence (ICACI). IEEE, pp 365–371

  77. Zhou J, Fang W, Wu X, Sun J, Cheng S (2016) An opposition-based learning competitive particle swarm optimizer. In: 2016 IEEE congress on evolutionary computation (CEC). IEEE, pp 515–521

  78. Zibakhsh A, Abadeh MS (2013) Gene selection for cancer tumor detection using a novel memetic algorithm with a multi-view fitness function. Eng Appl Artif Intell 26(4):1274–1281

    Google Scholar 

  79. Zitzler E, Deb K, Thiele L (2000) Comparison of multiobjective evolutionary algorithms: empirical results. Evol Comput 8(2):173–195

    Google Scholar 

Download references

Acknowledgements

Both Wong and Zhang were partially supported by a grant from the National Institute of General Medical Sciences of the National Institutes of Health under Award Number R01GM107639. The contents are solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. Tan was partially supported by the Key Project of Science and Technology Innovation 2030 supported by the Ministry of Science and Technology of China (Grant No. 2018AAA0101301).

Author information

Authors and Affiliations

  1. Department of Biostatistics, University of California at Los Angeles, Los Angeles, CA, 90095-1772, USA

    Zizhao Zhang & Weng Kee Wong

  2. Department of Computer Science, City University of Hong Kong, Kowloon, Hong Kong

    Kay Chen Tan

Authors
  1. Zizhao Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Weng Kee Wong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kay Chen Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Weng Kee Wong.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, Z., Wong, W.K. & Tan, K.C. Competitive swarm optimizer with mutated agents for finding optimal designs for nonlinear regression models with multiple interacting factors. Memetic Comp. 12, 219–233 (2020). https://doi.org/10.1007/s12293-020-00305-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12293-020-00305-6

Keywords

Search

Navigation