Skip to main content

Advertisement

SpringerLink
Log in

Local search in speciation-based bloat control for genetic programming

  • Published:
Genetic Programming and Evolvable Machines Aims and scope Submit manuscript

Abstract

This work presents a unique genetic programming (GP) approach that integrates a numerical local search method and a bloat-control mechanism to address some of the main issues with traditional GP. The former provides a directed search operator to work in conjunction with standard syntax operators that perform more exploration in design space, while the latter controls code growth by maintaining program diversity through speciation. The system can produce highly parsimonious solutions, thus reducing the cost of performing the local optimization process. The proposal is extensively evaluated using real-world problems from diverse domains, and the behavior of the search is analyzed from several different perspectives, including how species evolve, the effect of the local search process and the interpretability of the results. Results show that the proposed approach compares favorably with a standard approach, and that the hybrid algorithm can be used as a viable alternative for solving real-world symbolic regression problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Notes

  1. We must state that the terms species and speciation are used in a technical sense, following their use in NEAT, and are not meant to imply that they closely resemble their biological namesakes.

  2. Source code: https://github.com/saarahy/NGP-LS.

  3. Basically an aggregate of the size of all the individuals evaluated by a search.

References

  1. M. Affenzeller, S.M. Winkler, B. Burlacu, G. Kronberger, M. Kommenda, S. Wagner, Dynamic observation of genotypic and phenotypic diversity for different symbolic regression GP variants, in Proceedings of the Genetic and Evolutionary Computation Conference Companion, GECCO ’17 (ACM, New York, 2017), pp. 1553–1558

  2. N. Agarwal, B. Bullins, E. Hazan, Second-order stochastic optimization for machine learning in linear time. J. Mach. Learn. Res. 18(1), 4148–4187 (2017)

    MathSciNet  MATH  Google Scholar 

  3. S. Angra, S. Ahuja, Machine learning and its applications: a review, in 2017 International Conference on Big Data Analytics and Computational Intelligence (ICBDAC) (2017), pp. 57–60

  4. D. Applegate, B. Mayfield, An analysis of exchanging fitness cases with population size in symbolic regression genetic programming with respect to the computational model. in 2013 IEEE Congress on Evolutionary Computation (2013), pp. 3111–3116

  5. R.M.A. Azad, C. Ryan, A simple approach to lifetime learning in genetic programming-based symbolic regression. Evol. Comput. 22(2), 287–317 (2014)

    Article  Google Scholar 

  6. S. Bleuler, J. Bader, E. Zitzler, Reducing Bloat in GP with Multiple Objectives (Springer, Berlin, 2008), pp. 177–200

    Google Scholar 

  7. R.H. Byrd, R.B. Schnabel, G.A. Shultz, A trust region algorithm for nonlinearly constrained optimization. SIAM J. Numer. Anal. 24(5), 1152–1170 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  8. M. Castelli, L. Trujillo, L. Vanneschi, A. Popovič, Prediction of energy performance of residential buildings: a genetic programming approach. Energy Build. 102, 67–74 (2015)

    Article  Google Scholar 

  9. S. Chand, M. Wagner, Evolutionary many-objective optimization: a quick-start guide. Surv. Oper. Res. Manag. Sci. 20(2), 35–42 (2015)

    MathSciNet  Google Scholar 

  10. X. Chen, Y.-S. Ong, M.-H. Lim, K.C. Tan, A multi-facet survey on memetic computation. IEEE Trans. Evol. Comput. 15(5), 591–607 (2011)

    Article  Google Scholar 

  11. J.-S. Chou, C.-F. Tsai, Concrete compressive strength analysis using a combined classification and regression technique. Autom. Constr. 24, 52–60 (2012)

    Article  Google Scholar 

  12. V.V. de Melo, W. Banzhaf, Improving the prediction of material properties of concrete using kaizen programming with simulated annealing. Neurocomputing 246, 25–44 (2017)

    Article  Google Scholar 

  13. S. Dignum, R. Poli, Operator equalisation and bloat free GP, in Genetic Programming: 11th European Conference, EuroGP 2008 (Springer, Berlin, 2008), pp. 110–121

  14. J. Enríquez-Zárate, L. Trujillo, S. de Lara, M. Castelli, E. Z-Flores, L. Muñoz, A. Popovič, Automatic modeling of a gas turbine using genetic programming: an experimental study. Appl. Soft Comput. 50, 212–222 (2017)

    Article  Google Scholar 

  15. O.F. Ertuĝrul, A novel type of activation function in artificial neural networks: trained activation function. Neural Netw. 99, 148–157 (2018)

    Article  Google Scholar 

  16. F.-A. Fortin, F.-M. De Rainville, M.-A. Gardner, M. Parizeau, C. Gagné, DEAP: evolutionary algorithms made easy. J. Mach. Learn. Res. 13, 2171–2175 (2012)

    MathSciNet  MATH  Google Scholar 

  17. J.E. Hernández-Beltran, V.H. Díaz-Ramirez, L. Trujillo, P. Legrand, Design of estimators for restoration of images degraded by haze using genetic programming. Swarm Evol. Comput. 44, 49–63 (2019)

    Article  Google Scholar 

  18. T.K. Ho, Random decision forests, in Proceedings of 3rd International Conference on Document Analysis and Recognition, vol. 1 (1995), pp. 278–282

  19. P. Juárez-Smith, L. Trujillo, Integrating local search within neat-GP, in Proceedings of the 2016 on Genetic and Evolutionary Computation Conference Companion, GECCO ’16 Companion (ACM, New York, 2016), pp. 993–996

  20. S.S. Kim, K.C. Kwak, Development of quantum-based adaptive neuro-fuzzy networks. IEEE Trans. Syst. Man Cybern. B (Cyberne.) 40(1), 91–100 (2010)

    Article  Google Scholar 

  21. M. Kommenda, G. Kronberger, S.M. Winkler, M. Affenzeller, S. Wagner, Effects of constant optimization by nonlinear least squares minimization in symbolic regression, in Genetic and Evolutionary Computation Conference, GECCO ’13, Amsterdam, The Netherlands, July 6–10, 2013, Companion Material Proceedings (2013), pp. 1121–1128

  22. J.R. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection (MIT Press, Cambridge, 1992)

    MATH  Google Scholar 

  23. J.R. Koza, Human-competitive results produced by genetic programming. Genet. Program. Evolvable Mach. 11(3–4), 251–284 (2010)

    Article  Google Scholar 

  24. W.B. Langdon, P. Riccardo, Foundations of Genetic Programming (Springer, Berlin, 2002)

    Book  MATH  Google Scholar 

  25. D. Medernach, J. Fitzgerald, R.M.A. Azad, C. Ryan, A new wave: a dynamic approach to genetic programming, in Proceedings of the Genetic and Evolutionary Computation Conference 2016, GECCO ’16 (ACM, New York, 2016), pp. 757–764

  26. A. Moraglio, K. Krawiec, C.G. Johnson, Geometric semantic genetic programming, in Proceedings of the 12th International Conference on Parallel Problem Solving from Nature—Volume Part I (Springer, Berlin, 2012), pp. 21–31

  27. G. Olague, L. Trujillo, Evolutionary-computer-assisted design of image operators that detect interest points using genetic programming. Image Vis. Comput. 29(7), 484–498 (2011)

    Article  Google Scholar 

  28. I. Ortigosa, R. López, J. Garcia, A neural networks approach to residuary resistance of sailing yachts prediction, in Proceedings of the International Conference on Marine Engineering MARINE (2007), p. 250

  29. R. Poli, W.B. Langdon, S. Dignum, On the limiting distribution of program sizes in tree-based genetic programming, in Genetic Programming (Springer, Berlin, 2007), pp. 193–204

  30. R. Poli, W.B. Langdon, N.F. McPhee, A Field Guide to Genetic Programming (Lulu Enterprises, Morrisville, 2008)

    Google Scholar 

  31. J.R. Quinlan, Combining instance-based and model-based learning, in Machine Learning, Proceedings of the Tenth International Conference, University of Massachusetts, Amherst, MA, USA, June 27–29, 1993 (1993), pp. 236–243

  32. S.S. Roy, R. Roy, V.E. Balas, Estimating heating load in buildings using multivariate adaptive regression splines, extreme learning machine, a hybrid model of MARS and ELM. Renew. Sustain. Energy Rev. 82, 4256–4268 (2018)

    Article  Google Scholar 

  33. Y.-H. Shao, C.-H. Zhang, Z.-M. Yang, L. Jing, N.-Y. Deng, An \(\epsilon \)-twin support vector machine for regression. Neural Comput. Appl. 23(1), 175–185 (2013)

    Article  Google Scholar 

  34. S. Silva, Reassembling operator equalisation: a secret revealed, in Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation, GECCO ’11 (ACM, New York, 2011), pp. 1395–1402

  35. S. Silva, E. Costa, Dynamic limits for bloat control in genetic programming and a review of past and current bloat theories. Genet. Program. Evolvable Mach. 10(2), 141–179 (2009)

    Article  Google Scholar 

  36. D. Sorensen, Newton’s method with a model trust region modification. SIAM J. Numer. Anal. 16, 409–426 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  37. K.O. Stanley, R. Miikkulainen, Evolving neural networks through augmenting topologies. Evol. Comput. 10(2), 99–127 (2002)

    Article  Google Scholar 

  38. L. Trujillo, P. Legrand, G. Olague, J. LéVy-VéHel, Evolving estimators of the pointwise Hölder exponent with genetic programming. Inf. Sci. 209, 61–79 (2012)

    Article  Google Scholar 

  39. L. Trujillo, L. Muñoz, E. Galván-López, S. Silva, Neat genetic programming: controlling bloat naturally. Inf. Sci. 333, 21–43 (2016)

    Article  Google Scholar 

  40. A. Tsanas, A. Xifara, Accurate quantitative estimation of energy performance of residential buildings using statistical machine learning tools. Energy Build. 49, 560–567 (2012)

    Article  Google Scholar 

  41. E.J. Vladislavleva, G.F. Smits, D. den Hertog, Order of nonlinearity as a complexity measure for models generated by symbolic regression via pareto genetic programming. IEEE Trans. Evol. Comput. 13(2), 333–349 (2009)

    Article  Google Scholar 

  42. I.-C. Yeh, Modeling of strength of high-performance concrete using artificial neural networks. Cem. Concr. Res. 28(12), 1797–1808 (1998)

    Article  Google Scholar 

  43. E. Z-Flores, L. Trujillo, O. Schütze, P. Legrand, A local search approach to genetic programming for binary classification, in Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation, GECCO ’15 (ACM, New York, 2015), pp. 1151–1158

  44. E. Z-Flores, L. Trujillo, O. Schütze, P. Legrand, EVOLVE—A Bridge Between Probability, Set Oriented Numerics, and Evolutionary Computation V. Chapter, Evaluating the Effects of Local Search in Genetic Programming (Springer, New York, 2014)

    Google Scholar 

  45. E. Z-Flores, M. Abatal, A. Bassam, L. Trujillo, P. Juárez-Smith, Y.E. Hamzaoui, Modeling the adsorption of phenols and nitrophenols by activated carbon using genetic programming. J. Clean. Prod. 161, 860–870 (2017)

    Article  Google Scholar 

Download references

Acknowledgements

This work was funded by CONACYT (Mexico) project no. FC-2015-2/944 Aprendizaje evolutivo a gran escala, and TecNM (Mexico) project no. 6826-18-p. Second author was supported by CONACYT doctoral scholarship 332554. The authors would like to thank Spanish Ministry of Economy, Industry and Competitiveness and European Regional Development Fund (FEDER) under projects TIN2014-56494-C4-4-P (Ephemec) and TIN2017-85727-C4-4-P (DeepBio); Junta de Extremadura Project IB16035 Regional Government of Extremadura, Consejeria of Economy and Infrastructure, FEDER.

Author information

Authors and Affiliations

  1. Tecnológico Nacional de México/Instituto Tecnológico de Tijuana, Tijuana, BC, C.P. 22430, Mexico

    Perla Juárez-Smith, Leonardo Trujillo & Mario García-Valdez

  2. Universidad de Extremadura, Badajoz, Spain

    Francisco Fernández de Vega & Francisco Chávez

Authors
  1. Perla Juárez-Smith
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Leonardo Trujillo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mario García-Valdez
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Francisco Fernández de Vega
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Francisco Chávez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Leonardo Trujillo.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Juárez-Smith, P., Trujillo, L., García-Valdez, M. et al. Local search in speciation-based bloat control for genetic programming. Genet Program Evolvable Mach 20, 351–384 (2019). https://doi.org/10.1007/s10710-019-09351-7

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10710-019-09351-7

Keywords

Search

Navigation