Skip to main content
SpringerLink
Log in

Respecializing swarms by forgetting reinforced thresholds

  • Published:
Swarm Intelligence Aims and scope Submit manuscript

Abstract

Response threshold reinforcement is a powerful model for decentralized task allocation and specialization in multiagent swarms. In dynamic environments, initial task assignments and specializations must be updated over time to meet changing system needs. The very nature of threshold reinforcement-based behavior can, however, hinder respecialization, limiting its usability in real-world applications. We propose a decentralized forgetting-based extension to response threshold reinforcement and show that it can improve the efficiency and stability of the resulting task assignments under changing system demands.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  • Agassounon, W., & Martinoli, A. (2002). Efficiency and robustness of threshold-based distributed allocation algorithms in multi-agent systems. In Proceedings of the first international joint conference on Autonomous agents and multiagent systems: Part 3 (pp. 1090–1097). ACM.

  • Agassounon, W., Martinoli, A., & Goodman, R. (2001). A scalable, distributed algorithm for allocating workers in embedded systems. In 2001 IEEE international conference on systems, man, and cybernetics (Vol. 5, pp. 3367–3373).

  • Agmon, N., Urieli, D., & Stone, P. (2011). Multiagent patrol generalized to complex environmental conditions. In Proceedings of the twenty-fifth conference on artificial intelligence (AAAI’11).

  • Almeida, A., Ramalho, G., Santana, H., Tedesco, P., Menezes, T., Corruble, V., et al. (2004). Recent advances on multi-agent patrolling. In A. L. C. Bazzan & S. Labidi (Eds.), Advances in artificial intelligence: SBIA 2004 (pp. 474–483). Berlin: Springer.

    Chapter  Google Scholar 

  • Berman, S., Halasz, A., Kumar, V., & Pratt, S. (2007). Bio-inspired group behaviors for the deployment of a swarm of robots to multiple destinations. In Proceedings 2007 IEEE international conference on robotics and automation (pp. 2318–2323).

  • Brutschy, A., Pini, G., Pinciroli, C., Birattari, M., & Dorigo, M. (2014). Self-organized task allocation to sequentially interdependent tasks in swarm robotics. Autonomous Agents and Multi-agent Systems, 28(1), 101–125.

    Article  Google Scholar 

  • Campbell, A., & Wu, A. S. (2011). Multi-agent role allocation: Issues, approaches, and multiple perspectives. Autonomous Agents and Multi-agent Systems, 22(2), 317–355.

    Article  Google Scholar 

  • Campos, M., Bonabeau, E., Theraulaz, G., & Deneubourg, J. L. (2000). Dynamic scheduling and division of labor in social insects. Adaptive Behavior, 8(2), 83–95.

    Article  Google Scholar 

  • Chu, H. N., Glad, A., Simonin, O., Sempe, F., Drogoul, A., & Charpillet, F. (2007). Swarm approaches for the patrolling problem, information propagation vs. pheromone evaporation. In 19th IEEE international conference on tools with artificial intelligence, 2007. ICTAI 2007 (Vol. 1, pp. 442–449). IEEE.

  • Cicirello, V. A., & Smith, S. F. (2004). Wasp-like agents for distributed factory coordination. Autonomous Agents and Multi-Agent Systems, 8(3), 237–266.

    Article  Google Scholar 

  • de Lope, J., Maravall, D., & Quiñonez, Y. (2012). Decentralized multi-tasks distribution in heterogeneous robot teams by means of ant colony optimization and learning automata. In International conference on hybrid artificial intelligence systems (pp. 103–114). Springer.

  • de Lope, J., Maravall, D., & Quiñonez, Y. (2015). Self-organizing techniques to improve the decentralized multi-task distribution in multi-robot systems. Neurocomputing, 163, 47–55.

    Article  Google Scholar 

  • Dias, M. B. (2004). Traderbots: A new paradigm for robust and efficient multirobot coordination in dynamic environments (p. 153). Robotics Institute: Pittsburgh.

    Google Scholar 

  • Dos Santos, D. S., & Bazzan, A. L. (2012). Distributed clustering for group formation and task allocation in multiagent systems: A swarm intelligence approach. Applied Soft Computing, 12(8), 2123–2131.

    Article  Google Scholar 

  • Dos Santos, F., & Bazzan, A. L. (2009). An ant based algorithm for task allocation in large-scale and dynamic multiagent scenarios. In Proceedings of the 11th annual conference on genetic and evolutionary computation (pp. 73–80). ACM.

  • Dos Santos, F., & Bazzan, A. L. (2011). Towards efficient multiagent task allocation in the robocup rescue: A biologically-inspired approach. Autonomous Agents and Multi-agent Systems, 22(3), 465–486.

    Article  Google Scholar 

  • Ducatelle, F., Förster, A., Di Caro, G. A., & Gambardella, L. M. (2009). New task allocation methods for robotic swarms. In 9th IEEE/RAS conference on autonomous robot systems and competitions.

  • Farinelli, A., Iocchi, L., Nardi, D., & Ziparo, V. A. (2006). Assignment of dynamically perceived tasks by token passing in multirobot systems. Proceedings of the IEEE, 94(7), 1271–1288.

    Article  Google Scholar 

  • Ferrante, E., Turgut, A. E., Duéñez-Guzmán, E., Dorigo, M., & Wenseleers, T. (2015). Evolution of self-organized task specialization in robot swarms. PLoS Computational Biology, 11(8), e1004273.

    Article  Google Scholar 

  • Ferreira, P., & Bazzan, A. L. (2006). Swarm-gap: A swarm based approximation algorithm for e-gap. In First international workshop on agent technology for disaster management (pp. 49–55).

  • Ferreira, P. R., Boffo, F. S., & Bazzan, A. L. (2007). Using swarm-gap for distributed task allocation in complex scenarios. In International conference on autonomous agents and multiagent systems (pp. 107–121). Springer.

  • Ferreira, P. R., Dos Santos, F., Bazzan, A. L., Epstein, D., & Waskow, S. J. (2010). Robocup rescue as multiagent task allocation among teams: experiments with task interdependencies. Autonomous Agents and Multi-agent Systems, 20(3), 421–443.

    Article  Google Scholar 

  • Frison, M., Tran, N. L., Baiboun, N., Brutschy, A., Pini, G., Roli, A., Dorigo, M., & Birattari, M. (2010). Self-organized task partitioning in a swarm of robots. In International conference on swarm intelligence (pp. 287–298). Springer.

  • Garnier, S., Gautrais, J., & Theraulaz, G. (2007). The biological principles of swarm intelligence. Swarm Intelligence, 1(1), 3–31.

    Article  Google Scholar 

  • Ghizzioli, R., Nouyan, S., Birattari, M., & Dorigo, M. (2005). An ant-based algorithm for the heterogeneous dynamic task allocation problem. Technical Report, TR/IRIDIA/2005-005.

  • Golfarelli, M., Maio, D., & Rizzi, S. (1997). Multi-agent path planning based on task-swap negotiation. In Proceedings of the 16th UK planning and scheduling SIG workshop (p. 69).

  • Halász, A., Hsieh, M. A., Berman, S., & Kumar, V. (2007). Dynamic redistribution of a swarm of robots among multiple sites. In IEEE/RSJ international conference on intelligent robots and systems, 2007. IROS 2007 (pp. 2320–2325). IEEE.

  • Hsieh, M. A., Halász, Á., Berman, S., & Kumar, V. (2008). Biologically inspired redistribution of a swarm of robots among multiple sites. Swarm Intelligence, 2(2–4), 121–141.

    Article  Google Scholar 

  • Hsieh, M. A., Halász, Á., Cubuk, E. D., Schoenholz, S., & Martinoli, A. (2009). Specialization as an optimal strategy under varying external conditions. In IEEE international conference on robotics and automation, ICRA’09 (pp. 1941–1946).

  • Jones, C., & Mataric, M. (2003). Adaptive division of labor in large-scale minimalist multi-robot systems. In 2003 IEEE/RSJ international conference on intelligent robots and systems, 2003 (IROS 2003). Proceedings (Vol. 2, pp. 1969–1974). IEEE.

  • Kalra, N., & Martinoli, A. (2006). Comparative study of market-based and threshold-based task allocation. In Distributed autonomous robotic systems 7 (pp. 91–101). Springer.

  • Kanakia, A., Touri, B., & Correll, N. (2016). Modeling multi-robot task allocation with limited information as global game. Swarm Intelligence, 10(2), 147–160.

    Article  Google Scholar 

  • Kazakova, V. A., & Wu, A. S. (2018). Specialization vs. re-specialization: Effects of Hebbian learning in a dynamic environment. In Florida artificial intelligence research society conference FLAIRS-31.

  • Kira, Z., & Arkin, R. C. (2004). Forgetting bad behavior: Memory for case-based navigation. In 2004 IEEE/RSJ international conference on intelligent robots and systems (IROS). Proceedings (Vol. 4, pp. 3145–3152).

  • Kittithreerapronchai, O., & Anderson, C. (2003). Do ants paint trucks better than chickens? Markets versus response thresholds for distributed dynamic scheduling. In The 2003 congress on evolutionary computation, 2003. CEC’03 (Vol. 2, pp. 1431–1439). IEEE.

  • Krieger, M. J., & Billeter, J. B. (2000). The call of duty: Self-organised task allocation in a population of up to twelve mobile robots. Robotics and Autonomous Systems, 30(1–2), 65–84.

    Article  Google Scholar 

  • Labella, T. H., Dorigo, M., & Deneubourg, J. L. (2006). Division of labor in a group of robots inspired by ants’ foraging behavior. ACM Transactions on Autonomous and Adaptive Systems (TAAS), 1(1), 4–25.

    Article  Google Scholar 

  • Lee, W., & Kim, D. (2016). Local interaction of agents for division of labor in multi-agent systems. In International conference on simulation of adaptive behavior (pp. 46–54). Springer.

  • Lee, W., & Kim, D. (2017). History-based response threshold model for division of labor in multi-agent systems. Sensors, 17(6), 1232.

    Article  Google Scholar 

  • Levinthal, D. A., & March, J. G. (1993). The myopia of learning. Strategic Management Journal, 14(S2), 95–112.

    Article  Google Scholar 

  • Li, L., Martinoli, A., & Abu-Mostafa, Y. S. (2002). Emergent specialization in swarm systems. In International conference on intelligent data engineering and automated learning (pp. 261–266). Springer.

  • Liu, W., Winfield, A. F., Sa, J., Chen, J., & Dou, L. (2007). Towards energy optimization: Emergent task allocation in a swarm of foraging robots. Adaptive Behavior, 15(3), 289–305.

    Article  Google Scholar 

  • Ma, H., Li, J., Kumar, T., & Koenig, S. (2017). Lifelong multi-agent path finding for online pickup and delivery tasks. In Proceedings of the 16th conference on autonomous agents and multiagent systems, international foundation for autonomous agents and multiagent systems (pp. 837–845).

  • Mavrovouniotis, M., Li, C., & Yang, S. (2017). A survey of swarm intelligence for dynamic optimization: Algorithms and applications. Swarm and Evolutionary Computation, 33, 1–17.

    Article  Google Scholar 

  • McIntire, M., Nunes, E., & Gini, M. (2016). Iterated multi-robot auctions for precedence-constrained task scheduling. In Proceedings of the 2016 international conference on autonomous agents & multiagent systems, international foundation for autonomous agents and multiagent systems (pp. 1078–1086).

  • Murciano, A., Millán, J. D. R., & Zamora, J. (1997). Specialization in multi-agent systems through learning. Biological Cybernetics, 76(5), 375–382.

    Article  Google Scholar 

  • Nitschke, G., Schut, M., & Eiben, A. (2008). Emergent specialization in biologically inspired collective behavior systems. In Intelligent complex adaptive systems (pp. 215–253). IGI Global.

  • Nouyan, S. (2002). Agent-based approach to dynamic task allocation. In International workshop on ant algorithms (pp. 28–39). Springer.

  • Nouyan, S., Ghizzioli, R., Birattari, M., & Dorigo, M. (2005). An insect-based algorithm for the dynamic task allocation problem. KI, 19(4), 25–31.

    Google Scholar 

  • Nunes, E., & Gini, M. L. (2015). Multi-robot auctions for allocation of tasks with temporal constraints. In AAAI (pp. 2110–2116).

  • Nunes, E., McIntire, M., & Gini, M. (2016). Decentralized allocation of tasks with temporal and precedence constraints to a team of robots. In IEEE international conference on simulation, modeling, and programming for autonomous robots (SIMPAR) (pp. 197–202). IEEE.

  • Ono, N., & Fukumoto, K. (1996). Multi-agent reinforcement learning: A modular approach. In Second international conference on multiagent systems (pp. 252–258).

  • Pini, G., Brutschy, A., Frison, M., Roli, A., Dorigo, M., & Birattari, M. (2011). Task partitioning in swarms of robots: An adaptive method for strategy selection. Swarm Intelligence, 5(3–4), 283–304.

    Article  Google Scholar 

  • Portugal, D., & Rocha, R. (2011). A survey on multi-robot patrolling algorithms. In Doctoral conference on computing, electrical and industrial systems (pp. 139–146). Springer.

  • Price, R., & Tiňo, P. (2004). Evaluation of adaptive nature inspired task allocation against alternate decentralised multiagent strategies. In International conference on parallel problem solving from nature (pp. 982–990). Springer.

  • Quiñonez, Y., Maravall, D., & de Lope, J. (2011). Stochastic learning automata for self-coordination in heterogeneous multi-tasks selection in multi-robot systems. In Mexican international conference on artificial intelligence (pp. 443–453). Springer.

  • Román, J. A, Rodríguez, S., & Corchado, J. M. (2014). Improving intelligent systems: Specialization. In International conference on practical applications of agents and multi-agent systems (pp. 378–385). Springer.

  • Schwarzrock, J., Zacarias, I., Bazzan, A. L., de Araujo Fernandes, R. Q., Moreira, L. H., & de Freitas, E. P. (2018). Solving task allocation problem in multi unmanned aerial vehicles systems using swarm intelligence. Engineering Applications of Artificial Intelligence, 72, 10–20.

    Article  Google Scholar 

  • Tavares, A. R., Azpúrua, H., & Chaimowicz, L. (2014). Evolving swarm intelligence for task allocation in a real time strategy game. In 2014 Brazilian symposium on computer games and digital entertainment (SBGAMES) (pp. 99–108). IEEE.

  • Tavares, A. R., Zuin, G. L., Azp, H., Chaimowicz, L., et al. (2017). Combining genetic algorithm and swarm intelligence for task allocation in a real time strategy game. SBC Journal on Interactive Systems, 8(1), 4–19.

    Google Scholar 

  • Theraulaz, G., & Bonabeau, E. (1999). A brief history of stigmergy. Artificial Life, 5(2), 97–116. https://doi.org/10.1162/106454699568700.

    Article  Google Scholar 

  • Theraulaz, G., Bonabeau, E., & Deneubourg, J. L. (1998). Response threshold reinforcement and division of labour in insect societies. Proceedings of the Royal Society of London B, 265, 327–332.

    Article  Google Scholar 

  • van Lon, R. R., & Holvoet, T. (2017). When do agents outperform centralized algorithms? Autonomous Agents and Multi-agent Systems, 31(6), 1578–1609.

    Article  Google Scholar 

  • Villacorta, P. J., Pelta, D. A., & Lamata, M. T. (2013). Forgetting as a way to avoid deception in a repeated imitation game. Autonomous Agents and Multi-agent Systems, 27(3), 329–354.

    Article  Google Scholar 

  • Wawerla, J., Vaughan, R. T. (2010). A fast and frugal method for team-task allocation in a multi-robot transportation system. In ICRA (pp. 1432–1437).

  • Westhus, C., Kleineidam, C., Roces, F., & Weidenmller, A. (2013). Behavioural plasticity in the fanning response of bumblebee workers: Impact of experience and rate of temperature change. Animal Behaviour, 85(1), 27–34.

    Article  Google Scholar 

  • Wu, A. S., & Kazakova, V. A. (2017). Effects of task consideration order on decentralized task allocation using time-variant response thresholds. In Florida artificial intelligence research society conference FLAIRS-30 (pp. 466–471).

  • Zheng, X., & Koenig, S. (2011). Generalized reaction functions for solving complex-task allocation problems. In IJCAI proceedings-international joint conference on artificial intelligence (Vol. 22, p. 478).

Download references

Acknowledgements

This research was supported in part by ONR Grant N000140911043 and NSF Grant IIS1816777.

Author information

Authors and Affiliations

  1. University of Central Florida, Orlando, FL, 32816-2362, USA

    Vera A. Kazakova, Annie S. Wu & Gita R. Sukthankar

Authors
  1. Vera A. Kazakova
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Annie S. Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gita R. Sukthankar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vera A. Kazakova.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 200 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kazakova, V.A., Wu, A.S. & Sukthankar, G.R. Respecializing swarms by forgetting reinforced thresholds. Swarm Intell 14, 171–204 (2020). https://doi.org/10.1007/s11721-020-00181-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11721-020-00181-3

Keywords

Search

Navigation