Skip to main content
SpringerLink
Log in

Toward competitive multi-agents in Polo game based on reinforcement learning

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The learning of agents in a competitive space such as a game is a challenging task. The aim of the proposed research is to improve the reinforcement learning techniques in a competitive multi-agent for the Polo game. First, the video dataset is prepared. Then, the rules of the Polo game are extracted as a class diagram. An architecture is designed for multi-agent team in the Polo game. Therefore, an algorithm is proposed for the temporal difference in the game belief space for improving reward catching. The reward function is implemented in the agent team. Finally, the research improvement is evaluated by increasing 31 units in comparison with previous work. Therefore, competitive learning in the agent team has been improved.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Ali MZ, Morghem A, Albadarneh J, Al-Gharaibeh R, Suganthan PN, Reynolds RG (2014) Cultural algorithm applied to the evolution of robotic soccer team tactics: a novel perspective, Congress on Evolutionary Computation, IEEE, pp 2180–2187

  2. Asis KD, Hernandez-Garcia JF, Holland GZ, Sutton RS (2018) multi-step reinforcement learning: a unifying algorithm, 31th AAAI Conference on Artificial Intelligence 32:2902–2909

  3. Baglivo A, Ponti FD, Luca DD, Guidazzoli A, Liguori MC (2013) X3D/X3DOM, Blender Game Engine and OSG4WEB: open source visualization for cultural heritage environments. Digital Heritage Int Congress, IEEE 2:711–718

  4. Banerjee B, Davis CE (2017) Multi-agent path finding with persistence conflicts. IEEE Trans Comput Intell AI in Games, IEEE 9:402–409

  5. Beysolow T (2019) Applied reinforcement learning with Python, Book, Springer

  6. Carmel D, Markovitch S (1998) Model-based learning of interaction strategies in multi-agent systems. Taylor & Francis Ltd 10:309–332

  7. Castelfranchi C, Lesperance Y (2000) Intelligent Agent VII–Agent Theories, Architecture and Languages, Springer

  8. Chen B, Zhang A, Cao L (2014) Autonomous intelligent decision-making system based on Bayesian SOM. Neural Netw Robot Soccer, Elsevier, Neurocomputing 128:447–458

    Google Scholar 

  9. Collazo MN, Cotta C, Fernandez-Leiva AJ (2014) Virtual player design using self-learning via competitive coevolutionary algorithms, Springer. Nat Comput 31:131–144

  10. Collazo MN, Porras CC, Fern’Andez-Leiva AJ (2016) Competitive algorithms for co-evolving both game content and AI a case of study: planet wars. IEEE Transactions on Computational Intelligence and AI in Games 8(4):325–337

  11. Covaci A, Ghinea G, Huang CH, Shih J (2018) Multisensory came-lessons learnt from olfactory enhancement of a digital board game, Springer. Multimed Tools Appl 77:21245–21263

    Article  Google Scholar 

  12. Danny W (2010) Architecture-based design of multi-agent systems. Springer

  13. Ding S, Du W, Zhao X, Wang L, Jia W (2019) A new asynchronous reinforcement learning algorithm based on improved parallel PSO, Springer. Appl Intell 49:4211–4222

  14. Duan Y, Cui BX, Xu XH (2011) A multi-agent reinforcement learning approach to robot soccer, Springer. Artif Intell 36:193–211

  15. Fernando TG, Luis Javier Garcia V, Ana Lucila SO, Kim TH (2019) A comparison of learning methods over raw data: forecasting cab services market share in New York city, Springer. Multimed Tools Appl 78:29783–29804

  16. Guimaraes M, Santos P, Jhala A (2017) Prom week meet Skyrim: developing a social agent architecture in a commercial game. ACM, 17th international conference on autonomous agent and multi-agent system, pp 1562–1564

  17. Hagelbäck J (2016) Hybrid path finding in StarCraft. IEEE, Transactions on Computational Intelligence and AI in Games 38:319–324

  18. Hajuk M, Sukop M, Haun M (2019) Cognitive Multi-agent Systems, Book, Springer

  19. Hübner JF, Bordini RH (2010) Using agent-and organization-oriented programming to develop a team of agents for a competitive game. Springer, Science Business Media 59:351–372

  20. Husseinzadeh Kashan A, Karimi B (2010) A new algorithm for constrained optimization inspired by the sport leagues championships. IEEE. Congress on evolutionary computation. https://ieeexplore.ieee.org/document/5586364

  21. Kamalapurkar R, Walters P, Rosenfeld DW (2018) Reinforcement learning for optimal feedback control. Springer

  22. Kim JH, Vadakkepat P (2000) Multi-agent systems: a survey from the robot-soccer perspective. Intell Autom Soft Comput 6:3–17

    Article  Google Scholar 

  23. Kobti Z, Sharma S (2007) A multi-agent architecture for game playing. Computational intelligence and games. Proceedings of the 2007 IEEE Symposium on Computational Intelligence and Games. https://ieeexplore.ieee.org/document/4219054

  24. Koseler K, Stephan M (2018) Machine learning applications in baseball: a systematic literature review, applied artificial intelligence, applied artificial intelligence. Taylor & Francis, pp 745–763

  25. Laffay HA (2011) Polo in the United States: a history. Book. MC Far Land & Company Inc

  26. Laffay HA (2014) Polo in Argentina: a history. MC Far Land & Company Inc

  27. Horace A Laffaye (2009) The evaluation of polo. McFarlane & Company Inc

  28. Lee JM, Lee BJ, Kim KE (2020) Reinforcement learning for control with multiple frequencies. NeurIPS. 34th conference on neural information processing systems, pp 1–11

  29. Leng J, Lakhmi J (2009) Experimental analysis of eligibility traces strategies in temporal difference learning. Knowl Eng Soft Data Paradigms 1:26–39

    Article  Google Scholar 

  30. Leng J, Lim CP (2011) Reinforcement learning of competitive and cooperative skills in soccer agents. Elsevier. Appl Soft Comput 11:1353–1363

  31. Leng J, Fyfe C, Lakhmi J (2007) Reinforcement learning of competitive skills with soccer agents. Springer. Proc 11th Knowledge-Based Intell Inf Eng Syst 4692:572–579

  32. Marinheiro J, Cardoso HL (2017) A generic agent architecture for cooperative multi-agent games. ICAART. Proceedings of the 9th International Conference on Agents and Artificial Intelligence 2:107-118

  33. Masoumi B, Meybodi MR (2010) Learning automata based multi-agent system algorithms for finding optimal policies in Markov games. J Control 14:137–152

  34. Masoumi B, Meybodi MR (2011) Speeding up learning automata based multi-agent the concept of Stigmergy and entropy. Elsevier. Expert Systems with Applications 38:8105–8118

  35. Masoumi B, Meybodi MR, Abtahi F (2012) Learning automata based algorithms for finding optimal policies in fully cooperative Markov games. PRZEGLĄD ELEKTROTECHNICZNY 8:280–289

  36. Mattiassi ADA (2019) Fighting the game. Command systems and player-avatar interaction in fighting games in a social cognitive neuroscience framework. Springer. Multimedia Tools and Applications 78:13565–13591

  37. Mourao A, Magalhaes J (2013) Competitive affective gaming: winning with smile, proceedings of the 21st ACM international conference on multimedia, ACM, pp 83–91

  38. Nandy A, Biswas M (2018) Reinforcement learning. Springer

  39. Nash A, Koenig S (2013) Any-angle path planning. American Association for Artificial Intelligence. AI Magazine 34:85–107

  40. Parag C, Pendharkar (2012) Game theoretical applications for multi-agent systems. Elsevier. Expert Systems with Applications 39:273–279

  41. Pelechano N, Fuentes C (2016) Hierarchical path-finding for navigation mesh. Elsevier. Computers Graphics 59:68–78

  42. Polk S, Oommen BJ (2018) Novel threat-based AI strategies that incorporate adaptive data structures for multi-player board game. Springer. Applied Intelligence 48:1893–1911

  43. Polo Federation of Iran site: (n.d.) http://iranPolo.org/?page_id=1521#lightbox[gallery-1]/1/

  44. Rabin S (2002) AI programming WISDOM. Charles River media. April 3

  45. Scheepers C, Engelbrecht AP (2014) Competitive co-evolutionary training of simple soccer agents from zero knowledge. IEEE. Congress on Evolutionary Computation. https://ieeexplore.ieee.org/document/6900236

  46. Scheepers C, Engelbrecht AP (2014) Training multi-agent teams from zero knowledge with the competitive co-evolutionary team-based particle swarm optimizer. Springer. Soft Compute 20:607–620

  47. Scheepers C, Engelbrecht AP (2014) Analysis of stagnation behavior of competitive co-evolutionary trained neuro-controller. IEEE. Symposium on Swarm Intelligence. https://ieeexplore.ieee.org/document/7011795

  48. Sewak M (2019) Deep reinforcement learning. Springer

  49. Stone P, Veloso M (1998) Layered approach to learning client behaviors in the Robocup. Taylor & Francis. Applied Artificial Intelligence 12:165–188

  50. Sun P, Hu Y, Lan J, Tian L, Chen M (2019) TIDE: time-relevant deep reinforcement learning for routing optimization. Elsevier. Future Generation Computer Systems 99:401–409

  51. Tomaz L.B.P, Julia R.M.S, Duarte V.A (2017) A multi-agent player system composed by expert agents in specific game stages operating in high performance environment. Springer. Applied Intelligence 48:1–22

  52. Weyns D, Mascarsdi V, Ricci A (2019) Engineering multi-agent systems. Springer

  53. Wooldridge M (2002) An introduction to multi-agent systems. John Wiley& Sons. August

  54. Wooldridge M, Mller J, Tambe M (1997) Intelligent Agent II –Agent Theories. Architecture and Languages. Springer

  55. Yu FR, He Y (2019) Deep reinforcement learning for wireless networks. Springer

  56. Yuan Y, Yu Z.L, Gu Z, Deng X, Li X (2019) A novel multi-step reinforcement learning method for solving reward hacking. Springer. Applied Intelligence 49:2878–2888

Download references

Author information

Authors and Affiliations

  1. Faculty of Computer and Information Technology Engineering, Qazvin Branch, Islamic Azad University, Qazvin, Iran

    Zahra Movahedi

  2. Department of Computer Engineering, Karaj Branch, Islamic Azad University, Karaj, Iran

    Azam Bastanfard

Authors
  1. Zahra Movahedi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Azam Bastanfard
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Azam Bastanfard.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Movahedi, Z., Bastanfard, A. Toward competitive multi-agents in Polo game based on reinforcement learning. Multimed Tools Appl 80, 26773–26793 (2021). https://doi.org/10.1007/s11042-021-10968-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-10968-z

Keywords

Search

Navigation