Skip to main content
SpringerLink
Log in

VALIS: an evolutionary classification algorithm

  • Published:
Genetic Programming and Evolvable Machines Aims and scope Submit manuscript

Abstract

VALIS is an effective and robust classification algorithm with a focus on understandability. Its name stems from Vote-ALlocating Immune System, as it evolves a population of artificial antibodies that can bind to the input data, and performs classification through a voting process. In the beginning of the training, VALIS generates a set of random candidate antibodies; at each iteration, it selects the most useful ones to produce new candidates, while the least, are discarded; the process is iterated until a user-defined stopping condition. The paradigm allows the user to get a visual insight of the learning dynamics, helping to supervise the process, pinpoint problems, and tweak feature engineering. VALIS is tested against nine state-of-the-art classification algorithms on six popular benchmark problems; results demonstrate that it is competitive with well-established black-box techniques, and superior in specific corner cases.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. https://www.infovis-wiki.net.

  2. https://www.youtube.com/watch?v=nxW_ZtqqgXo.

  3. http://inversed.ru/AIS.htm.

  4. https://github.com/inversed-ru/VALIS.

  5. https://www.embarcadero.com/products/delphi.

  6. https://www.lazarus-ide.org/.

  7. http://archive.ics.uci.edu/ml/.

References

  1. U. Aickelin, D. Dasgupta, Artificial immune systems, in Search Methodologies, ed. by E.K. Burke, G. Kendall (Springer, Boston, MA, 2005)

    Google Scholar 

  2. N.S. Altman, An introduction to kernel and nearest-neighbor nonparametric regression. Am. Stat. 46(3), 175–185 (1992)

    MathSciNet  Google Scholar 

  3. T.W. Anderson, Classification by multivariate analysis. Psychometrika 16(1), 31–50 (1951). https://doi.org/10.1007/bf02313425

    Article  MathSciNet  Google Scholar 

  4. E. Bernadò-Mansilla, J.M. G.G, Accuracy-based learning classifier systems: models, analysis and applications to classification tasks. Evol. Comput. 11(3), 209–238 (2003)

    Article  Google Scholar 

  5. B.E. Boser, I.M. Guyon, V.N. Vapnik, A training algorithm for optimal margin classifiers, in Proceedings of the Fifth Annual Workshop on Computational Learning Theory (ACM, 1992), pp. 144–152

  6. L. Breiman, Pasting small votes for classification in large databases and on-line. Mach. Learn. 36(1–2), 85–103 (1999)

    Article  Google Scholar 

  7. L. Breiman, Random forests. Mach. Learn. 45(1), 5–32 (2001)

    Article  MATH  Google Scholar 

  8. J. Brownlee, Clonal Selection Theory and CLONALG: The Clonal Selection Classification Algorithm (CSCA) (Swinburne University of Technology, Melbourne, 2005)

    Google Scholar 

  9. F.M. Burnet, The Clonal Selection Theory of Acquired Immunity (Vanderbilt University Press, Nashville, 1959)

    Book  Google Scholar 

  10. M.V. Butz, S.W. W, An algorithmic description of XCS. Soft Comput. 6(3), 144–153 (2002)

    Article  MATH  Google Scholar 

  11. S. Cayzer, U. Aickelin, On the effects of idiotypic interactions for recommendation communities in artificial immune systems. Technical report (2008)

  12. P. Collet, E. Lutton, F. Raynal, M. Schoenauer, Polar IFS+ parisian genetic programming = efficient ifs inverse problem solving. Genet. Program. Evol. Mach. 1(4), 339–361 (2000)

    Article  MATH  Google Scholar 

  13. T.M. Cover, Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition. IEEE Trans. Electron. Comput. 3, 326–334 (1965)

    Article  MATH  Google Scholar 

  14. D.R. Cox, The regression analysis of binary sequences. J. R. Stat. Soc. Ser. B (Methodol.) 20(2), 215–242 (1958)

    MathSciNet  MATH  Google Scholar 

  15. L.N. de Castro, F.J. Von Zuben, Ainet: an artificial immune network for data analysis. Data Min. Heuristic Approach 1, 231–259 (2001)

    Google Scholar 

  16. L.N. de Castro, F.J.V. Zuben, Learning and optimization using the clonal selection principle. IEEE Trans. Evol. Comput. 6(3), 239–251 (2002). https://doi.org/10.1109/TEVC.2002.1011539

    Article  Google Scholar 

  17. J. Farmer, N.H. Packard, A.S. Perelson, The immune system, adaptation, and machine learning. Phys. D: Nonlinear Phenom. 22(1–3), 187–204 (1986). https://doi.org/10.1016/0167-2789(86)90240-x

    Article  MathSciNet  Google Scholar 

  18. S. Forrest, S.A. Hofmeyr, A. Somayaji, Computer immunology. Commun. ACM 40(10), 88–96 (1997)

    Article  Google Scholar 

  19. S. Forrest, A.S. Perelson, L. Allen, R. Cherukuri, Self-nonself discrimination in a computer, in Proceedings of the 1994 IEEE Symposium on Security and Privacy, SP ’94 (IEEE Computer Society, Washington, 1994), p. 202

  20. J. Greensmith, U. Aickelin, S. Cayzer, Detecting Danger: The Dendritic Cell Algorithm (Springer, London, 2008), pp. 89–112. https://doi.org/10.1007/978-1-84800-261-6_5

    Google Scholar 

  21. N.S. Halvaiee, M.K. Akbari, A novel model for credit card fraud detection using artificial immune systems. Appl. Soft Comput. 24, 40–49 (2014)

    Article  Google Scholar 

  22. J.H. Holland, Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control and Artificial Intelligence (The University of Michigan Press, 1975). http://books.google.it/books?id=YE5RAAAAMAAJ

  23. H. Hotelling, Analysis of a complex of statistical variables into principal components. J. Educ. Psychol. 24(6), 417 (1933)

    Article  MATH  Google Scholar 

  24. N.K. Jerne, Towards a network theory of the immune system. Annales d’immunologie 125, 373–389 (1974)

    Google Scholar 

  25. Z. Ji, D. Dasgupta, Real-valued negative selection algorithm with variable-sized detectors, in Genetic and Evolutionary Computation Conference (Springer, 2004), pp. 287–298

  26. P.M. Karpov, VALIS, a novel immune-inspired classification algorithm, in Proceedings of the 12th National Conference on Artificial Intelligence with International Participation CAI-2010, vol. 2 (Fizmatlit, 2010), pp. 325–334

  27. T.K. Landauer, P.W. Foltz, D. Laham, An introduction to latent semantic analysis. Discourse Process. 25(2–3), 259–284 (1998)

    Article  Google Scholar 

  28. P. Matzinger, Tolerance, danger, and the extended family. Annu. Rev. Immunol. 12(1), 991–1045 (1994)

    Article  Google Scholar 

  29. L. Meng, P. van der Putten, H. Wang, A comprehensive benchmark of the artificial immune recognition system (airs), in Proceedings of the First International Conference on Advanced Data Mining and Applications, ADMA’05 (Springer, Berlin, 2005), pp. 575–582. https://doi.org/10.1007/11527503_68

  30. R.S. Michalski, Learnable evolution model: evolutionary processes guided by machine learning. Mach. Learn. 38(1), 9–40 (2000)

    Article  MATH  Google Scholar 

  31. K. Pearson, Liii. On lines and planes of closest fit to systems of points in space. Lond. Edinb. Dublin Philos. Mag. J. Sci. 2(11), 559–572 (1901)

    Article  MATH  Google Scholar 

  32. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, E. Duchesnay, Scikit-learn: machine learning in python. J. Mach. Learn. Res. 12, 2825–2830 (2011)

    MathSciNet  MATH  Google Scholar 

  33. C.R. Rao, The utilization of multiple measurements in problems of biological classification. J. R. Stat. Soc. Ser. B (Methodol.) 10(2), 159–203 (1948)

    MathSciNet  MATH  Google Scholar 

  34. R. Shang, L. Qi, L. Jiao, R. Stolkin, Y. Li, Change detection in SAR images by artificial immune multi-objective clustering. Eng. Appl. Artif. Intell. 31, 53–67 (2014)

    Article  Google Scholar 

  35. D.N. Sotiropoulos, G.A. Tsihrintzis, Experimental evaluation of artificial immune system-based learning algorithms, in Machine Learning Paradigms (Springer, 2017), pp. 237–323

  36. G. Squillero, A. Tonda, Divergence of character and premature convergence: a survey of methodologies for promoting diversity in evolutionary optimization. Inf. Sci. 329, 782–799 (2016)

    Article  Google Scholar 

  37. J. Timmis, M. Neal, J. Hunt, An artificial immune system for data analysis. Biosystems 55(1), 143–150 (2000)

    Article  Google Scholar 

  38. A. Tonda, E. Lutton, G. Squillero, A benchmark for cooperative coevolution. Memet. Comput. 4(4), 263–277 (2012). https://doi.org/10.1007/s12293-012-0095-x

    Article  Google Scholar 

  39. A.M. Turing, Computing machinery and intelligence. Mind 59(236), 433–460 (1950)

    Article  MathSciNet  Google Scholar 

  40. B.H. Ulutas, S. Kulturel-Konak, A review of clonal selection algorithm and its applications. Artif. Intell. Rev. 36(2), 117–138 (2011)

    Article  Google Scholar 

  41. L. van der Maaten, G. Hinton, Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008)

    MATH  Google Scholar 

  42. A. Watkins, J. Timmis, L. Boggess, Artificial immune recognition system (airs): an immune-inspired supervised learning algorithm. Genet. Program. Evol. Mach. 5(3), 291–317 (2004)

    Article  Google Scholar 

  43. K.Q. Weinberger, J. Blitzer, L.K. Saul, Distance metric learning for large margin nearest neighbor classification, in Advances in Neural Information Processing Systems, vol. 18, ed. by Y. Weiss, P.B. Schölkopf, J.C. Platt (MIT Press, Cambridge, 2006), pp. 1473–1480

    Google Scholar 

  44. D. Whitley, T. Starkweather, C. Bogart, Genetic algorithms and neural networks: optimizing connections and connectivity. Parallel Comput. 14(3), 347–361 (1990)

    Article  Google Scholar 

  45. D.H. Wolpert, The lack of a priori distinctions between learning algorithms. Neural Comput. 8(7), 1341–1390 (1996)

    Article  Google Scholar 

  46. H. Zhang, The optimality of naive bayes. AA 1(2), 3 (2004)

    Google Scholar 

  47. J. Zhu, H. Zou, S. Rosset, T. Hastie, Multi-class adaboost. Stat. Its Interface 2(3), 349–360 (2009)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

Peter Karpov would like to thank Anton Shepelev, Brian Bucklew, Neil J. A. Sloane and Glenn Fiedler for financial support through his Patreon page (https://www.patreon.com/inversed).

Author information

Authors and Affiliations

  1. Podolsk, Russia

    Peter Karpov

  2. Politecnico di Torino — DAUIN, Corso Duca degli Abruzzi 24, 10129, Turin, Italy

    Giovanni Squillero

  3. UMR GMPA, INRA, AgroParisTech, Université Paris-Saclay, 1 av. Brétignières, 78850, Thiverval-Grignon, France

    Alberto Tonda

Authors
  1. Peter Karpov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Giovanni Squillero
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alberto Tonda
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Giovanni Squillero.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Karpov, P., Squillero, G. & Tonda, A. VALIS: an evolutionary classification algorithm. Genet Program Evolvable Mach 19, 453–471 (2018). https://doi.org/10.1007/s10710-018-9331-6

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10710-018-9331-6

Keywords

Search

Navigation