Skip to main content

Advertisement

SpringerLink
Log in

Multi-Objective Memetic Algorithms with Tree-Based Genetic Programming and Local Search for Symbolic Regression

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Symbolic regression is to search the space of mathematical expressions to find a model that best fits a given dataset. As genetic programming (GP) with the tree representation can represent solutions as expression trees, it is popularly-used for regression. However, GP tends to evolve unnecessarily large programs (known as bloat), causing excessive use of CPU time/memory and evolving solutions with poor generalization ability. Moreover, even though the importance of local search has been proved in augmenting the search ability of GP (termed as memetic algorithms), local search is underused in GP-based methods. This work aims to handle the above problems simultaneously. To control bloat, a multi-objective (MO) technique (NSGA-II, Non-dominant Sorting Genetic Algorithm) is selected to incorporate with GP, forming a multi-objective GP (MOGP). Moreover, three mutation-based local search operators are designed and incorporated with MOGP respectively to form three multi-objective memetic algorithms (MOMA), i.e. MOMA_MR (MOMA with Mutation-based Random search), MOMA_MF (MOMA with Mutation-based Function search) and MOMA_MC (MOMA with Mutation-based Constant search). The proposed methods are tested on both benchmark functions and real-world applications, and are compared with both GP-based (i.e. GP and MOGP) and nonGP-based symbolic regression methods. Compared with GP-based methods, the proposed methods can reduce the risk of bloat with the evolved solutions significantly smaller than GP solutions, and the local search strategies introduced in the proposed methods can improve their search ability with the evolved solutions dominating MOGP solutions. In addition, among the three proposed methods, MOMA_MR performs best in RMSE for testing, yet it consumes more training time than others. Moreover, compared with six reference nonGP-based symbolic regression methods, MOMA_MR generally performs better than or similar to them consistently.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. Memetic algorithms (MAs) can be regarded as a class of population-based stochastic local search methods, which combines population-based global search and local search [3, 39].

  2. Koza is a pioneer in solving complex optimization tasks based on GP [9, 11].

  3. For a set of objectives, solution A is claimed to dominate solution B, when A is better than or equal to B in all objectives, and A is better than B in at least one objective [4, 38].

  4. Solutions that are not dominated by any others in the population are called as non-dominated solutions [4, 38].

  5. Weka is a set of machine learning algorithms for solving real-world data mining tasks [7].

References

  1. Alexandropoulos SAN, Aridas CK, Kotsiantis SB, Vrahatis MN (2019) Multi-objective evolutionary optimization algorithms for machine learning: a recent survey. In: Approximation and optimization, vol 145. Springer, Cham

  2. Azzouz R, Bechikh S, Said LB (2017) Dynamic multi-objective optimization using evolutionary algorithms: a survey. In: Recent advances in evolutionary multi-objective optimization, vol 20. Springer, Cham

  3. Boryczka U, Szwarc K (2019) Selected variants of a memetic algorithm for jsp - a comparative study. Int J Prod Res 44:1–16

    Google Scholar 

  4. Deb K, Pratap A, Agarwal S, Meyarivan T (2002) A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans Evolut Comput 6(2):182–197

    Article  Google Scholar 

  5. Doerr B, Kötzing T, Lagodzinski JAG, Lengler J (2018) Bounding bloat in genetic programming. In: genetic and evolutionary computation conference

  6. Dua D, Graff C (2017) UCI machine learning repository . http://archive.ics.uci.edu/ml

  7. Frank E, Hall MA, Witten IH (2016) Data Mining: practical machine learning tools and techniques (Fourth Edition)

  8. Izadi Rad H, Feng J, Iba H (2018) GP-RVM: Genetic programing-based symbolic regression using relevance vector machine. In: International conference on systems, man, and cybernetics (SMC)

  9. Jie L, Jia H, Verma N, Jha NK (2018) Genetic programming for energy-efficient and energy-scalable approximate feature computation in embedded inference systems. IEEE Trans Comput 67:222–236

    Article  MathSciNet  Google Scholar 

  10. Kordestani JK, Firouzjaee HA, Meybodi MR (2018) An adaptive bi-flight cuckoo search with variable nests for continuous dynamic optimization problems. Appl Intell 48(1):97–117

    Article  Google Scholar 

  11. Koza JR (1992) Genetic programming: on the programming of computers by means of natural selection. MIT press, Cambridge

    MATH  Google Scholar 

  12. Koza JR (2014) Genetic programming ii: automatic discovery of reusable programs. Artif Life 1(4):439–441

    Google Scholar 

  13. Kronberger G, Kammerer L, Burlacu B, Winkler SM, Kommenda M, Affenzeller M (2019) Cluster analysis of a symbolic regression search space. In: Genetic programming theory and practice XVI. Springer, Cham

  14. Li B, Li J, Ke T, Xin Y (2015) Many-objective evolutionary algorithms:a survey. ACM Comput Surv 48(1):1–35

    Article  Google Scholar 

  15. Li B, Yang X (2019) Xuan H (2019) A hybrid simulated annealing heuristic for multistage heterogeneous fleet scheduling with fleet sizing decisions. J Adv Trans 10:1–19

    Google Scholar 

  16. Liang Y, Zhang M, Browne WN (2016) Figure-ground image segmentation using genetic programming and feature selection. In: IEEE congress on evolutionary computation (CEC), pp. 3839–3846. IEEE

  17. Liang Y, Zhang M, Browne WN (2016) Multi-objective genetic programming for figure-ground image segmentation. In: Australasian conference on artificial life and computational intelligence, pp. 134–146. Springer

  18. Liang Y, Zhang M, Browne WN (2017) Image feature selection using genetic programming for figure-ground segmentation. Eng Appl Artif Intell 62:96–108

    Article  Google Scholar 

  19. Liao CC, Ting CK (2018) A novel integer-coded memetic algorithm for the set\(k\)-cover problem in wireless sensor networks. IEEE Trans Cybern 48(8):1–14

    Article  MathSciNet  Google Scholar 

  20. Luke S, Panait L (2006) A comparison of bloat control methods for genetic programming. Evolut Comput 14(3):309–344

    Article  Google Scholar 

  21. Luo J, Yun Y, Liu Q, Xia L, Gao K (2018) A new hybrid memetic multi-objective optimization algorithm for multi-objective optimization. Inform Sci 448:164–186

    Article  MathSciNet  Google Scholar 

  22. de Melo VV (2014) Kaizen programming. In: GECCO ’14: Proceedings of the 2014 conference on Genetic and evolutionary computation, pp. 895–902

  23. Murrietamendoza A, Ternisien L, Beuze B, Botez RM (2018) Aircraft vertical route optimization by beam search and initial search space reduction. J Aerosp Inf Syst 15(5):1–15

    Google Scholar 

  24. Neri F, Cotta C et al (2012) Handbook of memetic algorithms [M]. Springer Berlin Heidelberg

  25. Pawlak TP, Krawiec K (2018) Competent geometric semantic genetic programming for symbolic regression and boolean function synthesis. Evolut Comput 26(2):1–36

    Article  Google Scholar 

  26. Poli R, Langdon WB, Mcphee NF (2008) A field guide to genetic programming. lulu.com. [S.L.] : Lulu Press (lulu.com), 2008. 10(2), 229–230

  27. Poli R, Mcphee NF (2014) Parsimony pressure made easy: Solving the problem of bloat in gp. Theory and Principled Methods for the Design of Metaheuristics pp. 181–204

  28. Quade M, Gout J (2018) Abel M (2019) Glyph: Symbolic regression tools. Journal of Open Research Software 7(5923)

  29. Shao L, Liu L, Li X (2014) Feature learning for image classification via multiobjective genetic programming. IEEE Trans Neural Netw Learn Syst 25(7):1359–1371

    Article  Google Scholar 

  30. Sonccoálvarez JL, Muñoz DM, Ayalarincón M (2018) Opposition-based memetic algorithm and hybrid approach for sorting permutations by reversals. Evolut Comput 27:229–265

    Article  Google Scholar 

  31. Trujillo L, Z-Flores E, Juarez P et al (2018) Local search is underused in genetic programming. In: Genetic and evolutionary computation XIV. Springer, Cham, pp 119–137

  32. Wang B, Singh HK, Ray T (2015) A multi-objective genetic programming approach to uncover explicit and implicit equations from data. In: IEEE congress on evolutionary computation (CEC), pp. 1129–1136. IEEE

  33. Wang P, Tang K, Weise T, Tsang EPK, Yao X (2014) Multiobjective genetic programming for maximizing ROC performance. Neurocomputing 125(3):102–118

    Article  Google Scholar 

  34. Xu G, Ying L, Lohse N, Pessemier TD, Joseph W (2018) Energy and labor aware production scheduling for industrial demand response using adaptive multi-objective memetic algorithm. IEEE Trans Ind Inf 15:942–953

    Google Scholar 

  35. Xu J, Liu SC, Zhao C, Wu J, Lin WC, Yu PW (2019) An iterated local search and tabu search for two-parallel machine scheduling problem to minimize the maximum total completion time. J Inf Optim Sci 40(3):751–766

    MathSciNet  Google Scholar 

  36. Z-Flores E, Trujillo L, Schütze O, Legrand P (2014) Evaluating the effects of local search in genetic programming. Springer International Publishing, Berlin

    Book  Google Scholar 

  37. Zhang C Genetic programming for symbolic regression. Available at https://pdfs.semanticscholar.org/e5ee/ddd04b8344fd4f39a5836be686886c80df13.pdf

  38. Zhang H, Dong Y, Xie X, Hu S, Weng S (2018) Pareto-dominance based adaptive multi-objective optimization for hydrothermal coordinated scheduling with environmental emission. Appl Soft Comput 69:270–287

    Article  Google Scholar 

  39. Zhang K, Cai Y, Fu S, Zhang H (2019) Multiobjective memetic algorithm based on adaptive local search chains for vehicle routing problem with time windows. Evolut Intell 3:1–12

    Google Scholar 

  40. Zhao Z, He X, Feng L (2017) An improved multi-objective memetic algorithm for bi-objective permutation flow shop scheduling. In: International conference on service systems and service management

  41. Zhong J, Liang F, Cai W, Ong YS (2018) Multifactorial genetic programming for symbolic regression problems. IEEE Trans Syst Man Cybern Syst 50:4492–4505

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Tianjin Key Laboratory of Autonomous Intelligent Technology and System, Tiangong University, Tianjin, 300387, China

    Jiayu Liang

  2. School of Computer Science and Software, Nanjing University of Information Science and Technology, Nanjing, 210044, China

    Yu Xue

Authors
  1. Jiayu Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yu Xue
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jiayu Liang.

Ethics declarations

Conflict of interest

Authors declares that she has no conflict of interest.

Funding

This study was funded by National Natural Science Foundation of China (Grant Number 61902281 and Grant Number 61876089).

Human or animal participants

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liang, J., Xue, Y. Multi-Objective Memetic Algorithms with Tree-Based Genetic Programming and Local Search for Symbolic Regression. Neural Process Lett 53, 2197–2219 (2021). https://doi.org/10.1007/s11063-021-10497-8

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-021-10497-8

Keywords

Search

Navigation