Skip to main content
SpringerLink
Log in

Distributed learning automata-based scheme for classification using novel pursuit scheme

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Learning Automata (LA) is a popular decision making mechanism to “determine the optimal action out of a set of allowable actions” (Agache and Oommen, IEEE Trans Syst Man Cybern-Part B Cybern 2002(6): 738–749, 2002). The distinguishing characteristic of automata-based learning is that the search for the optimising parameter vector is conducted in the space of probability distributions defined over the parameter space, rather than in the parameter space itself (Thathachar and Sastry, IEEE Trans Syst Man Cybern-Part B Cybern 32(6): 711–722, 2002). Recently, Goodwin and Yazidi pioneered the use of Ant Colony Optimisation (ACO) for solving classification problems (Goodwin and Yazidi 2016). In this paper, we propose a novel classifier based on the theory of LA. The classification problem is formulated as a deterministic optimization problem involving a team of LA that operate collectively to optimize an objective function. Although many LA algorithms have been devised in the literature, those LA schemes are not able to solve deterministic optimization problems as they suppose that the environment is stochastic. In this paper, we develop a novel pursuit LA which can be seen as the counterpart of the family of pursuit LA developed for stochastic environments (Agache and Oommen, IEEE Trans Syst Man Cybern Part B Cybern 32(6): 738–749, 2002). While classical pursuit LA are able to pursue the action with the highest reward estimate, our pursuit LA rather pursues the collection of actions that yield the highest performance. The theoretical analysis of the pursuit scheme does not follow classical LA proofs and can pave the way towards more schemes where LA can be applied to solve deterministic optimization problems. When applied to classification, the essence of our scheme is to search for a separator in the feature space by imposing a LA based random walk in a grid system. To each node in the gird we attach an LA, whose actions are the choice of the edges forming the separator. The walk is self-enclosing, i.e., a new random walk is started whenever the walker returns to starting node forming a closed classification path yielding a multiedged polygon. In our approach, the different LA attached at the different nodes search for a polygon that best encircles and separates each class. Based on the obtained polygons, we perform classification by labelling items encircled by a polygon as part of a class using ray casting function. Seen from a methodological perspective, PolyPursuit-LA has appealing properties compared to SVM. In fact, unlike PolyPursuit-LA, the SVM performance is dependent on the right choice of kernel function (e.g. Linear Kernel, Gaussian Kernel)— which is considered a “black art”. PolyPursuit-LA can find arbitrarily complex separators in the feature space. Experimental results from both synthetic and real-life data show that our scheme is able to perfectly separate both simple and complex patterns outperforming existing classifiers, including polynomial and linear SVM, without the need of any “kernel trick”. We believe that the results are impressive given the simplicity of PolyPursuit-LA compared to other approaches such as SVM.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. In the case of multi-class classification, more than one polygon is required.

  2. f(s) is formally defined in Section 5.

  3. The number of samples has negligible effect on the performance of the scheme.

References

  1. Agache M, Oommen BJ (2002) Generalized pursuit learning schemes: New families of continuous and discretized learning automata. IEEE Trans Syst Man Cybern-Part B Cybern 32(6):738–749

    Article  Google Scholar 

  2. Thathachar MAL, Sastry PS (2002) Varieties of learning automata: An overview. IEEE Trans Syst Man Cybern-Part B Cybern 32(6):711–722

    Article  Google Scholar 

  3. Goodwin M, Yazidi A (2016) Ant colony optimisation-based classification using two-dimensional polygons. In: International conference on swarm intelligence, Springer, pp 53–64

  4. Agache M, Oommen BJ (2002) Generalized pursuit learning schemes: New families of continuous and discretized learning automata. IEEE Trans Syst Man Cybern Part B Cybern 32(6):738–749

    Article  Google Scholar 

  5. Lakshmivarahan S (1981) Learning algorithms theory and applications. Springer-Verlag, Berlin

    Book  MATH  Google Scholar 

  6. Najim K, Poznyak AS (1994) Learning automata: Theory and applications. Pergamon Press, Oxford

    MATH  Google Scholar 

  7. Narendra KS, Thathachar MAL (1989) Learning automata: An introduction. New Jersey, Prentice-Hall, Inc.

    Google Scholar 

  8. Obaidat MS, Papadimitriou GI, Pomportsis AS (2002) Learning automata: Theory, paradigms, and applications. IEEE Trans Syst Man Cybern-Part B Cybern 32(6):706–709

    Article  Google Scholar 

  9. Poznyak AS, Najim K (1997) Learning automata and stochastic optimization. Springer-Verlag, Berlin

    MATH  Google Scholar 

  10. Thathachar MAL, Sastry PS (2003) Networks of learning automata: Techniques for online stochastic optimization. Kluwer Academic, Boston

    Google Scholar 

  11. Zhang J, Wang C, Zang D, Zhou M (2016) Incorporation of optimal computing budget allocation for ordinal optimization into learning automata. IEEE Trans Autom Sci Eng 13(2):1008–1017

    Article  Google Scholar 

  12. Tsetlin ML (1973) Automaton theory and the modeling of biological systems. Academic Press, New York

    MATH  Google Scholar 

  13. Misra S, Oommen BJ (2004) GPSPA: A new adaptive algorithm for maintaining shortest path routing trees in stochastic networks. Int J Commun Syst 17:963–984

    Article  Google Scholar 

  14. Obaidat MS, Papadimitriou GI, Pomportsis AS, Laskaridis HS (2002) Learning automata-based bus arbitration for shared-edium ATM switches. IEEE Trans Syst Man Cybern Part B 32:815– 820

    Article  Google Scholar 

  15. Oommen BJ, Roberts TD (2000) Continuous learning automata solutions to the capacity assignment problem. IEEE Trans Comput C-49:608–620

    Article  Google Scholar 

  16. Papadimitriou GI, Pomportsis AS (2000) Learning-automata-based TDMA protocols for broadcast communication systems with bursty traffic. IEEE Communication Letters, 107–109

  17. Atlassis AF, Loukas NH, Vasilakos AV (2000) The use of learning algorithms in ATM networks call admission control problem: A methodology. Comput Netw 34:341–353

    Article  Google Scholar 

  18. Atlassis AF, Vasilakos AV (2002) The use of reinforcement learning algorithms in traffic control of high speed networks. Advances in Computational Intelligence and Learning, 353–369

  19. Vasilakos AV, Saltouros MP, Atlassis AF, Pedrycz W (2003) Optimizing QoS routing in hierarchical ATM networks using computational intelligence techniques. IEEE Trans Syst Man Cybern Part C 33:297–312

    Article  Google Scholar 

  20. Seredynski F (1998) Distributed scheduling using simple learning machines. Eur J Oper Res 107:401–413

    Article  MATH  Google Scholar 

  21. Kabudian J, Meybodi MR, Homayounpour MM (2004) Applying continuous action reinforcement learning automata (CARLA) to global training of hidden markov models. In: Proceedings of the International Conference on Information Technology: Coding and Computing, ITCC’04, Las Vegas, Nevada, pp 638– 642

  22. Meybodi MR, Beigy H (2002) New learning automata based algorithms for adaptation of backpropagation algorithm pararmeters. Int J Neural Syst 12:45–67

    Article  Google Scholar 

  23. Unsal C, Kachroo P, Bay JS (1997) Simulation study of multiple intelligent vehicle control using stochastic learning automata. Trans Soc Comput Simul Int 14:193–210

    Google Scholar 

  24. Oommen BJ, Croix EDS (1995) Graph partitioning using learning automata. IEEE Trans Comput C-45:195–208

    MathSciNet  MATH  Google Scholar 

  25. Collins JJ, Chow CC, Imhoff TT (1995) Aperiodic stochastic resonance in excitable systems. Phys Rev E 52:R3321–R3324

    Article  Google Scholar 

  26. Cook RL (1986) Stochastic sampling in computer graphics. ACM Trans Graph 5:51–72

    Article  Google Scholar 

  27. Barzohar M, Cooper DB (1996) Automatic finding of main roads in aerial images by using geometric-stochastic models and estimation. IEEE Trans Pattern Anal Mach Intell 7:707–722

    Article  Google Scholar 

  28. Brandeau ML, Chiu SS (1989) An overview of representative problems in location research. Manag Sci 35:645–674

    Article  MathSciNet  MATH  Google Scholar 

  29. Bettstetter C, Hartenstein H, Pérez-Costa X (2004) Stochastic properties of the random waypoint mobility model. J Wirel Netw 10:555–567

    Article  Google Scholar 

  30. Rowlingson BS, Diggle PJ (1991) SPLANCS: Spatial point pattern analysis code in S-Plus, University of Lancaster, North West Regional Research Laboratory

  31. Paola M (1998) Digital simulation of wind field velocity. J Wind Eng Ind Aerodyn 74-76:91–109

    Article  Google Scholar 

  32. Cusumano JP, Kimble BW (1995) A stochastic interrogation method for experimental measurements of global dynamics and basin evolution: Application to a two-well oscillator. Nonlinear Dyn 8:213–235

    Article  Google Scholar 

  33. Baddeley A, Turner R (2005) Spatstat: An R package for analyzing spatial point patterns. J Stat Softw 12:1–42

    Article  Google Scholar 

  34. Caruana R, Niculescu-Mizil A (2006) An empirical comparison of supervised learning algorithms. In: Proceedings of the 23rd international conference on Machine learning, ACM, pp 161–168

  35. Caruana R, Karampatziakis N, Yessenalina A (2008) An empirical evaluation of supervised learning in high dimensions. In: Proceedings of the 25th international conference on Machine learning, ACM, pp 96–103

  36. Madjarov G, Kocev D, Gjorgjevikj D, Džeroski S (2012) An extensive experimental comparison of methods for multi-label learning. Pattern Recogn 45(9):3084–3104

    Article  Google Scholar 

  37. Dorigo M, Di Caro G (1999) Ant colony optimization: A new meta-heuristic. In: Proceedings of the 1999 congress on evolutionary computation-CEC99 (Cat. No. 99TH8406), vol 2, IEEE, pp 1470–1477

  38. Oommen BJ, Agache M (2001) Continuous and discretized pursuit learning schemes: Various algorithms and their comparison. IEEE Trans Syst Man Cybern-Part B Cybern 31:277–287

    Article  Google Scholar 

  39. Misra S, Oommen BJ (2005) Dynamic algorithms for the shortest path routing problem: learning automata-based solutions. IEEE Trans Syst Man Cybern-Part B Cybern 35(6):1179–1192

    Article  Google Scholar 

  40. Misra S, Oommen BJ (2006) An efficient dynamic algorithm for maintaining all-pairs shortest paths in stochastic networks. IEEE Trans Comput 55(6):686–702

    Article  Google Scholar 

  41. Li H, Mason L, Rabbat M (2009) Distributed adaptive diverse routing for voice-over-ip in service overlay networks. IEEE Trans Netw Serv Manag 6(3):175–189

    Article  Google Scholar 

  42. Mason L (1973) An optimal learning algorithm for s-model environments. IEEE Trans Autom Control 18(5):493–496

    Article  MATH  Google Scholar 

  43. Beigy H, Meybodi MR (2006) Utilizing distributed learning automata to solve stochastic shortest path problems. Int J Uncert Fuzziness Knowl-Based Syst 14(05):591–615

    Article  MathSciNet  MATH  Google Scholar 

  44. Torkestani JA, Meybodi MR (2010) An intelligent backbone formation algorithm for wireless ad hoc networks based on distributed learning automata. Comput Netw 54(5):826–843

    Article  MATH  Google Scholar 

  45. Torkestani JA, Meybodi MR (2012) Finding minimum weight connected dominating set in stochastic graph based on learning automata. Inform Sci 200:57–77

    Article  MathSciNet  MATH  Google Scholar 

  46. Torkestani JA, Meybodi MR (2012) A learning automata-based heuristic algorithm for solving the minimum spanning tree problem in stochastic graphs. J Supercomput 59(2):1035– 1054

    Article  Google Scholar 

  47. Thathachar MA, Sastry PS (1987) Learning optimal discriminant functions through a cooperative game of automata. IEEE Trans Syst Man Cybern 17(1):73–85

    Article  MathSciNet  MATH  Google Scholar 

  48. Santharam G, Sastry P, Thathachar M (1994) Continuous action set learning automata for stochastic optimization. J Franklin Inst 331(5):607–628

    Article  MathSciNet  MATH  Google Scholar 

  49. Sastry P, Thathachar M (1999) Learning automata algorithms for pattern classification. Sadhana 24(4):261–292

    Article  MathSciNet  MATH  Google Scholar 

  50. Zahiri S (2008) Learning automata based classifier. Pattern Recogn Lett 29(1):40–48

    Article  Google Scholar 

  51. Zeng X, Liu Z (2005) A learning automata based algorithm for optimization of continuous complex functions. Inform Sci 174(3):165–175

    Article  MathSciNet  MATH  Google Scholar 

  52. Howell M, Gordon T, Brandao F (2002) Genetic learning automata for function optimization. IEEE Trans Syst Man Cybern 32(6):804–815

    Article  Google Scholar 

  53. Bandyopadhyay S, Murthy CA, Pal SK (1995) Pattern classification with genetic algorithms. Pattern Recogn Lett 16(8):801–808

    Article  Google Scholar 

  54. Stützle T, López-Ibáñez M, Dorigo M (2011) A concise overview of applications of ant colony optimization, Wiley encyclopedia of operations research and management science

  55. Dorigo M, Birattari M, Stutzle T (2006) Ant colony optimization. IEEE Comput Intell Mag 1(4):28–39

    Article  Google Scholar 

  56. Gutjahr W (2000) A graph-based ant system and its convergence. Futur Gener Comput Syst 16(8):873–888

    Article  Google Scholar 

  57. D’Acierno L, Gallo M, Montella B (2012) An ant colony optimisation algorithm for solving the asymmetric traffic assignment problem. Eur J Oper Res 217(2):459–469

    Article  MathSciNet  MATH  Google Scholar 

  58. Goodwin M, Granmo O-C, Radianti J, Sarshar P, Glimsdal S (2013) Ant colony optimisation for planning safe escape routes. In: Recent trends in applied artificial intelligence, Springer, pp 53–62

  59. Goodwin M, Granmo O-C, Radianti J (2015) Escape planning in realistic fire scenarios with ant colony optimisation. Appl Intell 42(1):24–35

    Article  Google Scholar 

  60. Desell T, Clachar S, Higgins J, Wild B (2015) Evolving deep recurrent neural networks using ant colony optimization. In: Evolutionary computation in combinatorial optimization, Springer, pp 86–98

  61. Salama KM, Freitas AA (2015) Ant colony algorithms for constructing bayesian multi-net classifiers. Intell Data Anal 19(2):233–257

    Article  Google Scholar 

  62. Liu B, Abbas H, McKay B (2003) Classification rule discovery with ant colony optimization. In: IEEE/WIC international conference on intelligent agent technology, 2003. IAT 2003., IEEE, pp 83–88

  63. Parpinelli RS, Lopes HS, Freitas A, et al (2002) Data mining with an ant colony optimization algorithm. IEEE Trans Evol Comput 6(4):321–332

    Article  MATH  Google Scholar 

  64. Martens D, De Backer M, Haesen R, Vanthienen J, Snoeck M, Baesens B (2007) Classification with ant colony optimization. IEEE Trans Evol Comput 11(5):651–665

    Article  Google Scholar 

  65. Otero FE, Freitas A, Johnson CG (2008) Cant-miner: An ant colony classification algorithm to cope with continuous attributes. In: Ant colony optimization and swarm intelligence, Springer, pp 48–59

  66. Junior IC (2013) Data mining with ant colony algorithms. In: Intelligent computing theories and technology, Springer, pp 30–38

  67. Parpinelli RS, Lopes S, Freitas AA (2001) An ant colony based system for data mining: Applications to medical data. In: Proceedings of the genetic and evolutionary computation conference (GECCO-2001), Citeseer, pp 791–797

  68. Hota S, Satapathy P, Jagadev AK (2015) Modified ant colony optimization algorithm (mant-miner) for classification rule mining. In: Intelligent Computing, Communication and Devices, Springer, pp 267–275

  69. Özbakir L, Baykasoğlu A, Kulluk S, Yapıcı H (2009) Taco-miner: An ant colony based algorithm for rule extraction from trained neural networks. Expert Syst Appl 36(10):12295–12305

    Article  Google Scholar 

  70. Holden N, Freitas A (2004) Web page classification with an ant colony algorithm. In: Parallel problem solving from nature-PPSN VIII, Springer, pp 1092–1102

  71. Socha K, Blum C (2007) An ant colony optimization algorithm for continuous optimization: application to feed-forward neural network training. Neural Comput Appl 16(3):235–247

    Article  Google Scholar 

  72. Salama K, Abdelbar AM (2014) A novel ant colony algorithm for building neural network topologies. In: Swarm Intelligence, Springer, pp 1–12

  73. Salama KM, Abdelbar AM (2015) Learning neural network structures with ant colony algorithms, Swarm Intelligence, pp 1–37

  74. De Campos LM, Fernandez-Luna JM, Gámez JA, Puerta JM (2002) Ant colony optimization for learning bayesian networks. Int J Approx Reason 31(3):291–311

    Article  MathSciNet  MATH  Google Scholar 

  75. De Campos LM, Puerta J et al (2008) Learning bayesian networks by ant colony optimisation: searching in two different spaces. Mathware Soft Comput 9(3):251–268

    MathSciNet  MATH  Google Scholar 

  76. Daly R, Shen Q, Aitken S (2011) Learning bayesian networks: Approaches and issues. The Knowl Eng Rev 26(02):99–157

    Article  Google Scholar 

  77. Jun-Zhong J, ZHANG H-X, Ren-Bing H, Chun-Nian L (2009) A bayesian network learning algorithm based on independence test and ant colony optimization. Acta Automatica Sinica 35(3):281–288

    Google Scholar 

  78. Daly R, Shen Q (2014) Learning bayesian network equivalence classes with ant colony optimization. arXiv:1401.3464

  79. Juang C-F, Chang P-H (2010) Designing fuzzy-rule-based systems using continuous ant-colony optimization. IEEE Trans Fuzzy Syst 18(1):138–149

    Article  Google Scholar 

  80. Chan A, Freitas A (2006) A new classification-rule pruning procedure for an ant colony algorithm. In: Artificial evolution, Springer, pp 25–36

  81. Arora S, Singh S (2017) An effective hybrid butterfly optimization algorithm with artificial bee colony for numerical optimization. Changes 26:27

    Google Scholar 

  82. Restrepo AO, Rodríguez DE, Casas Mateus G, García P, Alonso CE, Montenegro Marín R, González C (2018) Hyperparameter optimization for image recognition over an ar-sandbox based on convolutional neural networks applying a previous phase of segmentation by color–space. Symmetry 10(12):743

    Article  Google Scholar 

  83. Meza J, Espitia H, Montenegro C, Crespo RG (2016) Statistical analysis of a multi-objective optimization algorithm based on a model of particles with vorticity behavior. Soft Comput 20(9):3521–3536

    Article  Google Scholar 

  84. Magdin M, Prikler F (2019) Are instructed emotional states suitable for classification? demonstration of how they can significantly influence the classification result in an automated recognition system. IJIMAI 5(4):141–147

    Article  Google Scholar 

  85. Thathachar MAL, Sastry PS (1985) A new approach to designing reinforcement schemes for learning automata, IEEE Transactions on Systems, Man and Cybernetics SMC-15

  86. Zhang X, Granmo O-C, Oommen BJ (2013) On incorporating the paradigms of discretization and bayesian estimation to create a new family of pursuit learning automata. Appl Intell 39(4):782–792

    Article  Google Scholar 

  87. Oommen BJ, Lanctôt JK (1990) Discretized pursuit learning automata. IEEE Trans Syst Man Cybern SMC-20 4:931–938

    Article  MathSciNet  MATH  Google Scholar 

  88. Gutjahr WJ (2002) Aco algorithms with guaranteed convergence to the optimal solution. Inf Process Lett 82(3):145–153

    Article  MathSciNet  MATH  Google Scholar 

  89. Thathachar MAL, Sastry PS (1987) Learning optimal discriminant functions through a cooperative game of automata. IEEE Trans Syst Man Cybern 17(1):73–85

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Centre for Artificial Intelligence Research, University of Agder, Kristiansand, Norway

    Morten Goodwin

  2. Department of Computer Science, Oslo Metropolitan University, Oslo, Norway

    Anis Yazidi

Authors
  1. Morten Goodwin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anis Yazidi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Morten Goodwin.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Goodwin, M., Yazidi, A. Distributed learning automata-based scheme for classification using novel pursuit scheme. Appl Intell 50, 2222–2238 (2020). https://doi.org/10.1007/s10489-019-01627-w

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-019-01627-w

Keywords

Search

Navigation