Skip to main content

Advertisement

SpringerLink
Log in

Deployment and evaluation of a flexible human–robot collaboration model based on AND/OR graphs in a manufacturing environment

  • Original Research Paper
  • Published:
Intelligent Service Robotics Aims and scope Submit manuscript

Abstract

The Industry 4.0 paradigm promises shorter development times, increased ergonomy, higher flexibility and resource efficiency in manufacturing environments. Collaborative robots are an important tangible technology for implementing such a paradigm. A major bottleneck to effectively deploy collaborative robots to manufacturing industries is developing task planning algorithms that enable them to recognize and naturally adapt to varying and even unpredictable human actions while simultaneously ensuring an overall efficiency in terms of production cycle time. In this context, an architecture encompassing task representation, task planning, sensing and robot control has been designed, developed and evaluated in a real industrial environment. A pick-and-place palletization task, which requires the collaboration between humans and robots, is investigated. The architecture uses AND/OR graphs for representing and reasoning upon human–robot collaboration models online. Furthermore, objective measures of the overall computational performance and subjective measures of naturalness in human–robot collaboration have been evaluated by performing experiments with production-line operators. The results of this user study demonstrate how human–robot collaboration models like the one we propose can leverage the flexibility and the comfort of operators in the workplace. In this regard, an extensive comparison study among recent models has been carried out.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. https://github.com/EmaroLab/industrialRobot_task_planning

  2. Please refer to: https://youtu.be/mF22PPmwHN0.

References

  1. International Federation of Robotics. Executive summary world robotics 2018 industrial robots. https://ifr.org/downloads/press2018/Executive_Summary_WR_2018_Industrial_Robots.pdf. 2018. [Accessed 2019-10-10]

  2. Colgate JE, Edward J, Peshkin MA, Wannasuphoprasit W (1996 November) Cobots: Robots for collaboration with human operators. In: Proceedings of the 1996 ASME international mechanical engineering congress and exhibition (imece). atlanta, usa

  3. Akella P, Peshkin M, Colgate E, Wannasuphoprasit W, Nagesh N, Wells J, Holland S, Pearson T, Peacock B (1999) Cobots for the automobile assembly line. In: Proceedings of the 1999 IEEE international conference on robotics and automation (ICRA). Detroit, USA

  4. Bicchi A, Peshkin MA, Colgate JE (2008) Safety for physical human–robot interaction. Springer handbook of robotics, Springer, Berlin, pp 1335–1348

  5. Cherubini A, Passama R, Crosnier A, Lasnier A, Fraisse P (2016) Collaborative manufacturing with physical human–robot interaction. Robot Comput Integr Manuf 40(C):1–13

    Article  Google Scholar 

  6. Johannsmeier L, Haddadin S (2017) A hierarchical human–robot interaction-planning framework for task allocation in collaborative industrial assembly processes. IEEE Robot Autom Lett 2(1):41–48

    Article  Google Scholar 

  7. Munzer T, Mollard Y, Lopes M (2017) Impact of robot initiative on human–robot collaboration. In: Proceedings of the 2017 ACM/IEEE international conference on human–robot interaction (HRI). Vienna, Austria

  8. Arai T, Kato R, Fujita M (2010) Assessment of operator stress induced by robot collaboration in assembly. CIRP Ann 59(1):5–8

    Article  Google Scholar 

  9. Capitanelli A, Maratea M, Mastrogiovanni F, Vallati M (2018) On the manipulation of articulated objects in human–robot cooperation scenarios. Robot Auton Syst 109:139–155

    Article  Google Scholar 

  10. Cannata G, Denei S, Mastrogiovanni F (2010) Tactile sensing: steps to artificial somatosensory maps. In: Proceedings of the 2010 IEEE symposium on robot and human interactive communication (RO-MAN). Viareggio, Italy

  11. Dolan P, Adams M (1998) Repetitive lifting tasks fatigue the back muscles and increase the bending moment acting on the lumbar spine. J Biomech 31(8):713–721

    Article  Google Scholar 

  12. Ranz F, Hummel V, Sihn W (2017) Capability-based task allocation in human–robot collaboration. Procedia Manuf 9:182–189

    Article  Google Scholar 

  13. Pinto C, Amorim P, Veiga G, Moreira A (2017) A review on task planning in human-robot teams. In: Proceedings of the 2017 robotics science and systems conference: workshop on mathematical models, algorithms, and human–robot interaction (RSS). Cambridge, USA

  14. Gombolay MC, Huang C, Shah JA (2015) Coordination of human–robot teaming with human task preferences. In: Proceedings of the 2015 AAAI artificial intelligence for human–robot interaction (AI-HRI). Arlington, USA

  15. Chen F, Sekiyama K, Sasaki H, Huang J, Sun B, Fukuda T (2011) Assembly strategy modeling and selection for human and robot coordinated cell assembly. In: Proceedings of the 2011 IEEE/RSJ international conference on intelligent robots and systems (IROS). San Francisco, USA

  16. Tsarouchi P, Michalos G, Makris S, Athanasatos T, Dimoulas K, Chryssolouris G (2017) On a human–robot workplace design and task allocation system. Int J Comput Integr Manuf 30(12):1272–1279

    Article  Google Scholar 

  17. Takata S, Hirano T (2011) Human and robot allocation method for hybrid assembly systems. CIRP Ann-ManufTechnol 60(1):9–12

    Article  Google Scholar 

  18. Darvish K, Bruno B, Simetti E, Mastrogiovanni F, Casalino G (2018) Interleaved online task planning, simulation, task allocation and motion control for flexible human–robot cooperation. In: Proceedings of the 2018 IEEE international symposium on robot and human interactive communication (RO-MAN). Nanjing, China

  19. Gerkey BP, Matarić MJ (2004) A formal analysis and taxonomy of task allocation in multi-robot systems. Int J Robot Res 23(9):939–954

    Article  Google Scholar 

  20. Gombolay M, Jensen R, Stigile J, Son SH, Shah J (2016) Apprenticeship scheduling: learning to schedule from human experts. In: Proceedings of the 2016 AAAI international joint conferences on artificial intelligence (IJCAI). New York, USA

  21. Darvish K, Wanderlingh F, Bruno B, Simetti E, Mastrogiovanni F, Casalino G (2018) Flexible human–robot cooperation models for assisted shop-floor tasks. Mechatronics 51:97–114

    Article  Google Scholar 

  22. Chen F, Sekiyama K, Cannella F, Fukuda T (2014) Optimal subtask allocation for human and robot collaboration within hybrid assembly system. IEEE Trans Autom Sci Eng 11(4):1065–1075

    Article  Google Scholar 

  23. Bortot D, Born M, Bengler K (2013) Directly or on detours: how should industrial robots approximate humans. In: Proceedings of the 2013 ACM/IEEE international conference on human–robot interaction (HRI). Tokyo, Japan

  24. Levine SJ, Williams BC (2014) Concurrent plan recognition and execution for human–robot teams. In: Proceedings of the 2014 international conference on automated planning and scheduling (ICAPS). Portsmouth, USA

  25. Crandall JW, Oudah M, Ishowo-Oloko F, Abdallah S, Bonnefon JF, Cebrian M, Shariff A, Goodrich MA, Rahwan I (2018) Cooperating with machines. Nat Commun 9(1):233

    Article  Google Scholar 

  26. Toussaint M, Munzer T, Mollard Y, Wu LY, Vien NA, Lopes M (2016) Relational activity processes for modeling concurrent cooperation. In: Proceedings of the 2016 IEEE international conference on robotics and automation (ICRA). Stockholm, Sweden

  27. Lamon E, De Franco A, Peternel L, Ajoudani A (2019) A capability-aware role allocation approach to industrial assembly tasks. IEEE Robot Autom Lett 4(4):3378–3385

    Article  Google Scholar 

  28. Colledanchise M, Ögren P (2018) Behavior trees in robotics and Al: an introduction. CRC Press, Boca Ratan

    Book  Google Scholar 

  29. Coronado LE, Mastrogiovanni F, Venture G (2018) Design of a human-centered robot framework for end-user programming and applications. In: Proceedings of the 2018 CISM IFToMM symposium on robot design, dynamics and control (ROMANSY). Rennes, France

  30. Coronado LE, Mastrogiovanni F, Venture G (2018) Development of intelligent behaviours for social robots via user-friendly and modular programming tools. In: Proceedings of the 2018 IEEE workshop on advanced robotics and its social impact (ARSO). Genoa, Italy

  31. Mastrogiovanni F, Sgorbissa A (2013) A behaviour sequencing and composition architecture based on ontologies for entertainment humanoid robot. Robot AutonSyst 2(61):170–183

    Google Scholar 

  32. Paxton C, Hundt A, Jonathan F, Guerin K, Hager GD (2017) CoSTAR: instructing collaborative robots with behavior trees and vision. In: Proceedings of the 2017 IEEE international conference on robotics and automation (ICRA). Singapore, Singapore

  33. Colledanchise M (2017) Behavior trees in robotics. Ph.D. thesis, KTH Royal Institute of Technology

  34. Korsah GA, Stentz A, Dias MB (2013) A comprehensive taxonomy for multi-robot task allocation. Int J Robot Res 32(12):1495–1512

    Article  Google Scholar 

  35. Hawkins KP, Bansal S, Vo NN, Bobick AF (2014) Anticipating human actions for collaboration in the presence of task and sensor uncertainty. In: Proceedings of the 2014 IEEE international conference on robotics and automation (ICRA). Hong Kong, China

  36. Rozo L, Silvério J, Calinon S, Caldwell DG (2016) Exploiting interaction dynamics for learning collaborative robot behaviors. In: Proceedings of the 2016 AAAI international joint conference on artificial intelligence: Interactive machine learning workshop (IJCAI). New York, USA

  37. Koppula HS, Saxena A (2016) Anticipating human activities using object affordances for reactive robotic response. IEEE Trans Pattern Anal Mach Intell 38(1):14–29

    Article  Google Scholar 

  38. Caccavale R, Finzi A (2017) Flexible task execution and attentional regulations in human–robot interaction. IEEE Trans Cogn Dev Syst 9(1):68–79

    Article  Google Scholar 

  39. Haigh KZ, Veloso MM (1998) Planning, execution and learning in a robotic agent. In: Proceedings of the 1998 AAAI international conference on artificial intelligence planning systems (ICAPS). Pittsburgh, USA

  40. Wilcox R, Nikolaidis S, Shah J (2012) Optimization of temporal dynamics for adaptive human-robot interaction in assembly manufacturing. In: Proceedings of the 2012 robotics: science and systems (RSS). Sydney, Australia

  41. Agostini A, Torras C, Wörgötter F (2011) Integrating task planning and interactive learning for robots to work in human environments. In: Proceedings of the 2011 AAAI international joint conference on artificial intelligence (IJCAI). Barcelona, Spain

  42. Sanderson A, Peshkin M, de Mello LH (1988) Task planning for robotic manipulation in space applications. IEEE Trans Aerosp Electron Syst 24(5):619–629

    Article  Google Scholar 

  43. Fikes RE, Nilsson NJ (1971) Strips: a new approach to the application of theorem proving to problem solving. Artif Intell 2(3–4):189–208

    Article  Google Scholar 

  44. Bruno B, Mastrogiovanni F, Sgorbissa A (2014) A public domain dataset for adl recognition using wrist-placed accelerometers. In: Proceedings of the 2014 IEEE symposium on robot and human interactive communication (RO-MAN). Edinburgh, Scotland

  45. Carfì A, Motolese C, Bruno B, Mastrogiovanni F (2018) Online human gesture recognition using recurrent neural networks and wearable sensors. In: Proceedings of the 2018 IEEE symposium on robot and human interactive communication (RO-MAN). Nanjing, China

  46. Krause KW, DeMotte DD, Dinsmoor CA, Evans JA, Nowak GF, Ross GA, Rutledge GJ, Slabe CF (2003) Robotic system with teach pendant. May 6. US Patent 6,560,513

  47. Billard A, Siegwart R (2004) Robot learning from demonstration. Robot Auton Syst 2(47):65–67

    Article  Google Scholar 

  48. Hoener S, Mellert FT (1985) Offline programming of industrial robots. In: Toward the factory of the future. Springer, pp 597–602

  49. Akgun B, Cakmak M, Yoo JW, Thomaz AL (2012) Trajectories and keyframes for kinesthetic teaching: a human-robot interaction perspective. In: Proceedings of the 2012 ACM/IEEE international conference on human–robot interaction (HRI). Boston, USA

  50. De Mello LH, Sanderson AC (1990) AND/OR graph representation of assembly plans. IEEE Trans Robot Autom 6(2):188–199

    Article  Google Scholar 

  51. Quigley M, Conley K, Gerkey B, Faust J, Foote T, Leibs J, Wheeler R, Ng AY (2009) ROS: an open-source robot operating system. In: Proceedings of the 2009 international conference on robotics and automation (ICRA): workshop on open source software. Kobe, Japan

  52. Latombe JC (2012) Robot motion planning, vol 124. Springer, Berlin

    Google Scholar 

  53. Andersen TT (2015) Optimizing the universal robots ROS driver. Technical University of Denmark, Department of Electrical Engineering. Tech Rep. http://orbit.dtu.dk/en/publications/optimizing-the-universal-robots-ros-driver.html

  54. User Manual: UR10/CB3 version 3.4.5. Universal Robots A/S. 2017. https://s3-eu-west-1.amazonaws.com/ur-support-site/27484/UR10_User_Manual_en_E67ON_Global-3.4.5.pdf

  55. Goodrich MA, Schultz AC (2008) Human–robot interaction: a survey. Found Trends® Human-Comput Interact 1(3):203–275

    Article  Google Scholar 

  56. Tsai RY, Lenz RK (1989) A new technique for fully autonomous and efficient 3D robotics hand/eye calibration. IEEE Trans Robot Autom 5(3):345–358

    Article  Google Scholar 

  57. Silvio Traversaro AS (2016) Multibody dynamics notation. Technische Universiteit Eindhoven, Tech Rep. 4;https://pure.tue.nl/ws/files/25753352/Traversaro_en_Saccon_DC_2016.064.pdf

  58. Kareem SY, Buoncompagni L, Mastrogiovanni F (2018) Arianna+: scalable human activity recognition by reasoning with a network of ontologies. In: Proceedings of the 17th International conference of the Italian association for artificial intelligence (AIxIA). Trento, Italy

  59. Luger GF (2005) Artificial intelligence: structures and strategies for complex problem solving. Pearson education, London

    Google Scholar 

  60. Russell SJ, Norvig P (2016) Artificial intelligence: a modern approach. Pearson Education Limited, Malaysia

    MATH  Google Scholar 

  61. Carfì A, Foglino F, Bruno B, Mastrogiovanni F (2019) A multi-sensor dataset of human–human handover. Data Brief 22:109–117

    Article  Google Scholar 

  62. Sebastia L, Onaindia E, Marzal E (2001) Stella: an optimal sequential and parallel planner. In: Proceedings of the 2002 Portuguese conference on artificial intelligence (EPIA). Porto, Portugal

  63. Koenig N, Howard A (2004) Design and use paradigms for gazebo, an open-source multi-robot simulator. In: Proceedings of the 2004 IEEE/RSJ international conference on intelligent robots and systems (IROS). Sendai, Japan

  64. Shah JA (2011) Fluid coordination of human–robot teams. Ph.D. thesis, Massachusetts Institute of Technology

  65. Robla-Gómez S, Becerra VM, Llata JR, Gonzalez-Sarabia E, Torre-Ferrero C, Perez-Oria J (2017) Working together: a review on safe human–robot collaboration in industrial environments. IEEE Access 5:26754–26773

    Article  Google Scholar 

  66. Mastrogiovanni F, Paikan A, Sgorbissa A (2013) Semantic-aware real-time scheduling in robotics. IEEE Trans Rob 1(29):118–135

    Article  Google Scholar 

  67. Espiau B, Chaumette F, Rives P (1992) A new approach to visual servoing in robotics. IEEE Trans Robot Autom 8(3):313–326

    Article  Google Scholar 

  68. Michalos G, Makris S, Tsarouchi P, Guasch T, Kontovrakis D, Chryssolouris G (2015) Design considerations for safe human–robot collaborative workplaces. Procedia CIrP. 37:248–253

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the Schaeffler Group, Slovakia, for the opportunity to perform research in their manufacturing plant and solve real-world problems in human–robot collaboration. The work has been supported by the European Master on Advanced Robotics Plus (EMARO+) programme.

Author information

Authors and Affiliations

  1. Department of Informatics, Bioengineering, Robotics, and Systems Engineering, University of Genoa, Via Opera Pia 13, 16145, Genoa, Italy

    Prajval Kumar Murali, Kourosh Darvish & Fulvio Mastrogiovanni

Authors
  1. Prajval Kumar Murali
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kourosh Darvish
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fulvio Mastrogiovanni
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kourosh Darvish.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Murali, P.K., Darvish, K. & Mastrogiovanni, F. Deployment and evaluation of a flexible human–robot collaboration model based on AND/OR graphs in a manufacturing environment. Intel Serv Robotics 13, 439–457 (2020). https://doi.org/10.1007/s11370-020-00332-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11370-020-00332-9

Keywords

Search

Navigation