Skip to main content
SpringerLink
Log in

Cooperative Queuing Policies for Effective Scheduling of Operator Intervention

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

We consider multi-robot applications, where a team of robots can ask for the intervention of a human operator to handle difficult situations. As the number of requests grows, team members will have to wait for the operator attention, hence the operator becomes a bottleneck for the system. Our aim in this context is to make the robots learn cooperative strategies to decrease the idle time of the system by modeling the operator as a shared resource. In particular, we consider a balking queuing model where robots decide whether or not to join the queue and use multi-robot learning to estimate the best cooperative policy. In more detail, we formalize the problem as Decentralized Markov Decision Process and provide a suitable state representation, so to apply an independent learners approach. We evaluate the proposed method in a robotic water monitoring simulation and empirically show that our approach can significantly improve the team performance, while being computationally tractable.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. Some part of this work appears in Raeissi and Farinelli (2018). That work describes basic ideas and preliminary results, here we provide a more detailed description of the methodologies, and more extensive empirical analysis.

  2. While this may be a significant challenge in some domains, this is not the focus of our work.

References

  • Bernstein, D. S., Givan, R., Immerman, N., & Zilberstein, S. (2002). The complexity of decentralized control of Markov decision processes. Mathematics of Operations Research, 27(4), 819–840.

    Article  MathSciNet  Google Scholar 

  • Bevacqua, G., Cacace, J., Finzi, A., & Lippiello, V. (2015). Mixed-initiative planning and execution for multiple drones in search and rescue missions. In Proceedings of the twenty-fifth international conference on automated planning and scheduling, ICAPS’15 (pp. 315–323). AAAI Press

  • Cacace, J., Caccavale, R., Finzi, A., & Lippiello, V. (2016). Attentional multimodal interface for multidrone search in the alps. In 2016 IEEE international conference on systems, man, and cybernetics (SMC) (pp. 001178–001183). https://doi.org/10.1109/SMC.2016.7844401.

  • Chernova, S., & Veloso, M. (2009). Interactive policy learning through confidence-based autonomy. J. Artif. Int. Res., 34(1), 1–25. http://dl.acm.org/citation.cfm?id=1622716.1622717.

  • Chien, S. Y., Lewis, M., Mehrotra, S., Brooks, N., & Sycara, K. P. (2012). Scheduling operator attention for multi-robot control. In IEEE/RSJ international conference on intelligent robots and systems, IROS, Vilamoura, Algarve, Portugal, October 7–12 (pp. 473–479)

  • Collins, J., Bilot, C., Gini, M., & Mobasher, B. (2000). Mixed-initiative decision support in agent-based automated contracting. In Proceedings of the fourth international conference on autonomous agents, AGENTS ’00 (pp. 247–254). ACM, New York, NY, USA

  • Dorais, G. A., Bonasso, R. P., Kortenkamp, D., Pell, B., & Schreckenghost, D. (1998). Adjustable autonomy for human-centered autonomous systems. In: on Mars in First International Conference of the Mars Society.

  • Farinelli, A., Raeissi, M. M., Marchi, N., Brooks, N., & Scerri, P. (2017). Interacting with team oriented plans in multi-robot systems. Autonomous Agents and Multi-Agent Systems, 31(2), 332–361.

    Article  Google Scholar 

  • Goldman, C. V., & Zilberstein, S. (2003). Optimizing information exchange in cooperative multi-agent systems. In Proceedings of the second international joint conference on autonomous agents and multiagent systems, AAMAS ’03 (pp. 137–144). ACM, New York, NY, USA

  • Gromov, B., M. Gambardella, L., & Di Caro, G. (2016). Wearable multi-modal interface for human multi–robot interaction. In IEEE international symposium on safety, security, and rescue robotics, SSRR2016 (pp. 240–245).

  • Gunderson, J. P., & Martin, W. N. (1999). Effects of uncertainty on variable autonomy in maintenance robots. In Workshop on autonomy control software (pp. 26–34).

  • Horvitz, E., Jacobs, A., & Hovel, D. (1999). Attention-sensitive alerting. In Proceedings of the fifteenth conference on uncertainty in artificial intelligence, UAI’99 (pp. 305–313). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA . http://dl.acm.org/citation.cfm?id=2073796.2073831.

  • Hsieh, M. A., Cowley, A., Keller, J. F., Chaimowicz, L., Grocholsky, B., Kumar, V., et al. (2007). Adaptive teams of autonomous aerial and ground robots for situational awareness. Journal of Field Robotics, 24(11–12), 991–1014.

    Article  Google Scholar 

  • Kaminka, G. A., & Frenkel, I. (2005). Flexible teamwork in behavior-based robots. In Proceedings of the 20th national conference on artificial intelligence (Vol. 1, pp. 108–113), AAAI’05. AAAI Press

  • Lewis, M., Chien, S. Y., Mehortra, S., Chakraborty, N., & Sycara, K. (2014). Task switching and single vs. multiple alarms for supervisory control of multiple robots. In International conference on engineering psychology and cognitive ergonomics (pp. 499–510). Springer.

  • Naor, P. (1969). The regulation of queue size by levying tolls. Econometrica, 37(1), 15–24. http://www.jstor.org/stable/1909200.

  • Panait, L., & Luke, S. (2005). Cooperative multi-agent learning: The state of the art. Autonomous Agents and Multi-agent Systems, 11(3), 387–434.

    Article  Google Scholar 

  • Raeissi, M. M., & Farinelli, A. (2018). Learning queuing strategies in human–multi-robot interaction. In Proceedings of the 17th international conference on autonomous agents and multiagent systems, AAMAS ’18 (pp. 2207–2209). International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC.

  • Rosenfeld, A., Agmon, N., Maksimov, O., Azaria, A. & Kraus, S. (2015). Intelligent agent supporting human–multi-robot team collaboration. In Proceedings of the 24th international conference on artificial intelligence, IJCAI’15 (pp. 1902–1908).

  • Rosenthal, S., & Veloso, M. (2010). Using symbiotic relationships with humans to help robots overcome limitations. In Workshop for collaborative human/ai control for interactive experiences.

  • Scheutz, M. & Kramer, J. (2007). Reflection and reasoning mechanisms for failure detection and recovery in a distributed robotic architecture for complex robots. In 2007 IEEE international conference on robotics and automation (pp. 3699–3704). IEEE.

  • Stoica, A., Theodoridis, T., Hu, H., McDonald-Maier, K., & Barrero, D. (2013). Towards human-friendly efficient control of multi-robot teams. In Proceedings of the 2013 international conference on collaboration technologies and systems, CTS 2013 (pp. 226–231).

  • Sutton, R. S., McAllester, D. A., Singh, S. P., & Mansour, Y. (2000). Policy gradient methods for reinforcement learning with function approximation. In Advances in neural information processing systems (pp. 1057–1063).

  • Suzanne Barber, K., Goel, A., & Martin, C. E. (2000). Dynamic adaptive autonomy in multi-agent systems. Journal of Experimental & Theoretical Artificial Intelligence, 12(2), 129–147.

    Article  Google Scholar 

  • Tambe, M. (1997). Towards flexible teamwork. J. Artif. Int. Res., 7(1), 83–124.

    Google Scholar 

  • Valada, A., Velagapudi, P., Kannan, B., Tomaszewski, C., Kantor, G., & Scerri, P. (2014). Development of a Low Cost Multi-robot Autonomous Marine Surface Platform. Berlin Heidelberg: Springer.

    Book  Google Scholar 

  • Wang, J., & Lewis, M. (2007). Human control for cooperating robot teams. In Proceedings of the ACM/IEEE international conference on human–robot interaction, HRI ’07 (pp. 9–16). ACM.

  • Xuan, P., & Lesser, V. (2002). Multi-agent policies: From centralized ones to decentralized ones. In Proceedings of the first international joint conference on autonomous agents and multiagent systems: Part 3, AAMAS ’02 (pp. 1098–1105). ACM.

  • Zhang, Y., Narayanan, V., Chakraborti, T. & Kambhampati, S. (2015). A human factors analysis of proactive support in human–robot teaming. In 2015 IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 3586–3593). https://doi.org/10.1109/IROS.2015.7353878.

Download references

Acknowledgements

This work is partially funded by the European Union’s Horizon 2020 research and innovation programme under grant agreement No 689341. This work reflects only the authors’ view and the EASME is not responsible for any use that may be made of the information it contains.

Author information

Authors and Affiliations

  1. University of Verona, Verona, Italy

    Masoume M. Raeissi & Alessandro Farinelli

  2. Centrum Wiskunde & Informatica, Amsterdam, The Netherlands

    Masoume M. Raeissi

Authors
  1. Masoume M. Raeissi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alessandro Farinelli
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alessandro Farinelli.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This is one of the several papers published in Autonomous Robots comprising the Special Issue on Multi-Robot and Multi-Agent Systems.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Raeissi, M.M., Farinelli, A. Cooperative Queuing Policies for Effective Scheduling of Operator Intervention. Auton Robot 44, 617–626 (2020). https://doi.org/10.1007/s10514-019-09877-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-019-09877-w

Keywords

Search

Navigation