Skip to main content
SpringerLink
Log in

Heterogeneous graph attention networks for scalable multi-robot scheduling with temporospatial constraints

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

Robot teams are increasingly being deployed in environments, such as manufacturing facilities and warehouses, to save cost and improve productivity. To efficiently coordinate multi-robot teams, fast, high-quality scheduling algorithms are essential to satisfy the temporal and spatial constraints imposed by dynamic task specification and part and robot availability. Traditional solutions include exact methods, which are intractable for large-scale problems, or application-specific heuristics, which require expert domain knowledge to develop. In this paper, we propose a novel heterogeneous graph attention network model, called ScheduleNet, to learn scheduling policies that overcome the limitations of conventional approaches. By introducing robot- and proximity-specific nodes into the simple temporal network encoding temporal constraints, we obtain a heterogeneous graph structure that is nonparametric in the number of tasks, robots and task resources or locations. We show that our model is end-to-end trainable via imitation learning on small-scale problems, and generalizes to large, unseen problems. Empirically, our method outperforms the existing state-of-the-art methods in a variety of testing scenarios involving both homogeneous and heterogeneous robot teams.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  • Benders, J. F. (1962). Partitioning procedures for solving mixed-variables programming problems. Numerische Mathematik, 4(1), 238–252.

    Article  MathSciNet  Google Scholar 

  • Bengio, Y., Lodi, A., & Prouvost, A. (2021). Machine learning for combinatorial optimization: A methodological tour d’horizon. European Journal of Operational Research, 290(2), 405–421. https://doi.org/10.1016/j.ejor.2020.07.063.

  • Bogner, K., Pferschy, U., Unterberger, R., & Zeiner, H. (2018). Optimised scheduling in human–robot collaboration—A use case in the assembly of printed circuit boards. International Journal of Production Research, 56(16), 5522–5540.

    Article  Google Scholar 

  • Casalino, A., Zanchettin, A. M., Piroddi, L., & Rocco, P. (2021). Optimal scheduling of human–robot collaborative assembly operations with time petri nets. IEEE Transactions on Automation Science and Engineering, 18(1), 70–84. https://doi.org/10.1109/TASE.2019.2932150.

  • Castro, E., & Petrovic, S. (2012). Combined mathematical programming and heuristics for a radiotherapy pre-treatment scheduling problem. Journal of Scheduling, 15(3), 333–346.

    Article  MathSciNet  Google Scholar 

  • Chen, J., & Askin, R. G. (2009). Project selection, scheduling and resource allocation with time dependent returns. European Journal of Operational Research, 193(1), 23–34.

    Article  Google Scholar 

  • Choudhury, S., Gupta, J., Kochenderfer, M.J., Sadigh, D., & Bohg, J. (2020). Dynamic multi-robot task allocation under uncertainty and temporal constraints. In Proceedings of Robotics: Science and Systems (RSS). https://doi.org/10.15607/rss.2020.xvi.068.

  • Dechter, R., Meiri, I., & Pearl, J. (1991). Temporal constraint networks. Artificial Intelligence, 49(1–3), 61–95.

    Article  MathSciNet  Google Scholar 

  • Essafi, I., Mati, Y., & Dauzère-Pérès, S. (2008). A genetic local search algorithm for minimizing total weighted tardiness in the job-shop scheduling problem. Computers & Operations Research, 35(8), 2599–2616.

    Article  MathSciNet  Google Scholar 

  • Flushing, E. F., Gambardella, L. M., & Di Caro, G. A. (2017). Simultaneous task allocation, data routing, and transmission scheduling in mobile multi-robot teams. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 1861–1868). IEEE.

  • Fout, A., Byrd, J., Shariat, B., & Ben-Hur, A. (2017) Protein interface prediction using graph convolutional networks. In Advances in neural information processing systems, pp. 6530–6539.

  • Gombolay, M., Wilcox, R., & Shah, J. (2013) Fast scheduling of multi-robot teams with temporospatial constraints. In Robotics: Science and system, pp. 49–56.

  • Gombolay, M. C., Wilcox, R. J., & Shah, J. A. (2018). Fast scheduling of robot teams performing tasks with temporospatial constraints. IEEE Transactions on Robotics, 34(1), 220–239.

    Article  Google Scholar 

  • Hamaguchi, T., Oiwa, H., Shimbo, M., & Matsumoto, Y. (2017). Knowledge transfer for out-of-knowledge-base entities: A graph neural network approach. arXiv preprint arXiv:1706.05674.

  • Hamilton, W. L., Ying, R., & Leskovec, J. (2018). Inductive representation learning on large graphs. arXiv:1706.02216.

  • Hari, S. K. K., Nayak, A., & Rathinam, S. (2020). An approximation algorithm for a task allocation, sequencing and scheduling problem involving a human–robot team. IEEE Robotics and Automation Letters, 5(2), 2146–2153.

    Article  Google Scholar 

  • Johnson, D. B. (1977). Efficient algorithms for shortest paths in sparse networks. Journal of the ACM (JACM), 24(1), 1–13.

    Article  MathSciNet  Google Scholar 

  • Kartal, B., Nunes, E., Godoy, J., & Gini, M. (2016). Monte Carlo tree search with branch and bound for multi-robot task allocation. In The IJCAI-16 workshop on autonomous mobile service robots, Vol. 33.

  • Khalil, E., Dai, H., Zhang, Y., Dilkina, B., & Song, L. (2017). Learning combinatorial optimization algorithms over graphs. In Advances in neural information processing systems, pp. 6348–6358.

  • Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.

  • Kool, W., van Hoof, H., & Welling, M. (2019). Attention, learn to solve routing problems! In International conference on learning representations.

  • Korsah, G. A., Stentz, A., & Dias, M. B. (2013). A comprehensive taxonomy for multi-robot task allocation. The International Journal of Robotics Research, 32(12), 1495–1512.

    Article  Google Scholar 

  • LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444.

    Article  Google Scholar 

  • Nikou, A., Boskos, D., Tumova, J., & Dimarogonas, D. V. (2017) Cooperative planning for coupled multi-agent systems under timed temporal specifications. In 2017 American Control Conference (ACC) (pp. 1847–1852). IEEE.

  • Nunes, E., Manner, M., Mitiche, H., & Gini, M. (2017). A taxonomy for task allocation problems with temporal and ordering constraints. Robotics and Autonomous Systems, 90, 55–70.

    Article  Google Scholar 

  • Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., et al. (2019). Pytorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems, 32, 8024–8035.

    Google Scholar 

  • Pinedo, M. (2012). Scheduling, Vol. 29. Springer.

  • Piot, B., Geist, M., & Pietquin, O. (2014). Boosted bellman residual minimization handling expert demonstrations. In Joint European Conference on machine learning and knowledge discovery in databases (pp. 549–564). Springer.

  • Raghavan, H., Madani, O., & Jones, R. (2006). Active learning with feedback on features and instances. Journal of Machine Learning Research, 7, 1655–1686.

    MathSciNet  MATH  Google Scholar 

  • Ren, H., & Tang, L. (2009). An improved hybrid milp/cp algorithm framework for the job-shop scheduling. In 2009 IEEE international conference on automation and logistics (pp. 890–894). IEEE.

  • Shiue, Y. R., Lee, K. C., & Su, C. T. (2018). Real-time scheduling for a smart factory using a reinforcement learning approach. Computers & Industrial Engineering, 125, 604–614.

    Article  Google Scholar 

  • Solomon, M. M. (1986). On the worst-case performance of some heuristics for the vehicle routing and scheduling problem with time window constraints. Networks, 16(2), 161–174.

    Article  MathSciNet  Google Scholar 

  • Solovey, K., Bandyopadhyay, S., Rossi, F., Wolf, M. T., & Pavone, M. (2020). Fast near-optimal heterogeneous task allocation via flow decomposition. arXiv preprint arXiv:201103603.

  • Tang, G., & Webb, P. (2019). Human–robot shared workspace in aerospace factories. In Human–robot interaction: Safety, standardization, and benchmarking, pp. 71–80.

  • Tsamardinos, I. (2000). Reformulating temporal plans for efficient execution. Master’s thesis, University of Pittsburgh.

  • Tsamardinos, I., & Pollack, M. E. (2003). Efficient solution techniques for disjunctive temporal reasoning problems. Artificial Intelligence, 151(1–2), 43–89.

    Article  MathSciNet  Google Scholar 

  • Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., & Bengio, Y. (2017). Graph attention networks. arXiv preprint arXiv:1710.10903.

  • Wang, H., Chen, W., & Wang, J. (2020). Coupled task scheduling for heterogeneous multi-robot system of two robot types performing complex-schedule order fulfillment tasks. Robotics and Autonomous Systems, 131, 103560.

    Article  Google Scholar 

  • Wang, M., Yu, L., Zheng, D., Gan, Q., Gai, Y., Ye, Z., Li, M., Zhou, J., Huang, Q., Ma, C., Huang, Z., Guo, Q., Zhang, H., Lin, H., Zhao, J., Li, J., Smola, A. J., Zhang, Z. (2019a). Deep graph library: Towards efficient and scalable deep learning on graphs. ICLR workshop on representation learning on graphs and manifolds.

  • Wang, X., Ji, H., Shi, C., Wang, B., Ye, Y., Cui, P., Yu, P. S. (2019b). Heterogeneous graph attention network. In The World Wide Web Conference (pp. 2022–2032). ACM.

  • Wang, Y., Sun, Y., Liu, Z., Sarma, S. E., Bronstein, M. M., & Solomon, J. M. (2019c). Dynamic graph CNN for learning on point clouds. ACM Transactions on Graphics (TOG), 38(5), 1–12.

    Article  Google Scholar 

  • Wang, Y. C., & Usher, J. M. (2005). Application of reinforcement learning for agent-based production scheduling. Engineering Applications of Artificial Intelligence, 18(1), 73–82.

    Article  Google Scholar 

  • Wang, Z., & Gombolay, M. (2019). Learning to dynamically coordinate multi-robot teams in graph attention networks. arXiv preprint arXiv:1912.02059.

  • Wang, Z., & Gombolay, M. (2020). Heterogeneous graph attention networks for scalable multi-robot scheduling with temporospatial constraints. In Robotics: Science and System XVI.

  • Wilson, S., Glotfelter, P., Wang, L., Mayya, S., Notomista, G., Mote, M., et al. (2020). The robotarium: Globally impactful opportunities, challenges, and lessons learned in remote-access, distributed control of multirobot systems. IEEE Control Systems Magazine, 40(1), 26–44.

    Article  MathSciNet  Google Scholar 

  • Wu, J., Xu, X., Zhang, P., & Liu, C. (2011). A novel multi-agent reinforcement learning approach for job scheduling in grid computing. Future Generation Computer Systems, 27(5), 430–439.

    Article  Google Scholar 

  • Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., & Philip, S. Y. (2021). A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems, 32(1), 4–24. https://doi.org/10.1109/TNNLS.2020.2978386..

  • Xu, K., Hu, W., Leskovec, J., & Jegelka, S. (2019). How powerful are graph neural networks? In International conference on learning representations.

  • Yan, S., Xiong, Y., & Lin, D. (2018). Spatial temporal graph convolutional networks for skeleton-based action recognition. In Proceedings of the AAAI conference on artificial intelligence, Vol. 32, pp. 7444–7452.

  • Yan, Z., Jouandeau, N., & Cherif, A. A. (2013). A survey and analysis of multi-robot coordination. International Journal of Advanced Robotic Systems, 10(12), 399.

    Article  Google Scholar 

  • Yang, X., Deng, C., Liu, T., & Tao, D. (2020). Heterogeneous graph attention network for unsupervised multiple-target domain adaptation. IEEE Transactions on Pattern Analysis and Machine Intelligence. https://doi.org/10.1109/TPAMI.2020.3026079.

  • Zhang, S., Chen, Y., Zhang, J., & Jia, Y. (2020). Real-time adaptive assembly scheduling in human-multi-robot collaboration according to human capability. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 3860–3866. IEEE.

  • Zhou, J., Cui, G., Zhang, Z., Yang, C., Liu, Z., Wang, L., Li, C., & Sun, M. (2018). Graph neural networks: A review of methods and applications. arXiv preprint arXiv:1812.08434.

Download references

Author information

Authors and Affiliations

  1. Georgia Institute of Technology, Atlanta, GA, USA

    Zheyuan Wang, Chen Liu & Matthew Gombolay

Authors
  1. Zheyuan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chen Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Matthew Gombolay
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zheyuan Wang.

Ethics declarations

Funding

This work was supported in part by the Office of Naval Research under grant GR10006659 and Lockheed Martin Corporation under grant GR00000509.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This is one of the several papers published in Autonomous Robots comprising the Special Issue on Robotics: Science and Systems 2020.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary material 1 (mp4 66720 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Z., Liu, C. & Gombolay, M. Heterogeneous graph attention networks for scalable multi-robot scheduling with temporospatial constraints. Auton Robot 46, 249–268 (2022). https://doi.org/10.1007/s10514-021-09997-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-021-09997-2

Keywords

Search

Navigation