Skip to main content
SpringerLink
Log in

A Bayesian tracker for synthesizing mobile robot behaviour from demonstration

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

Programming robots often involves expert knowledge in both the robot itself and the task to execute. An alternative to direct programming is for a human to show examples of the task execution and have the robot perform the task based on these examples, in a scheme known as learning or programming from demonstration. We propose and study a generic and simple learning-from-demonstration framework. Our approach is to combine the demonstrated commands according to the similarity between the demonstrated sensory trajectories and the current replay trajectory. This tracking is solely performed based on sensor values and time and completely dispenses with the usually expensive step of precomputing an internal model of the task. We analyse the behaviour of the proposed model in several simulated conditions and test it on two different robotic platforms. We show that it can reproduce different capabilities with a limited number of meta parameters.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

Notes

  1. https://github.com/bayesian-trajectory-replay.

  2. The video is available at http://stephane.magnenat.net/videos/artor_entering_trailer.mp4.

References

  • Abbeel, P., & Ng, A. Y. (2004). Apprenticeship learning via inverse reinforcement learning. In Proceedings of the twenty-first international conference on machine learning (p. 1).

  • Abbeel, P., Coates, A., & Ng, A. Y. (2010). Autonomous helicopter aerobatics through apprenticeship learning. International Journal of Robotics Research, 29(13), 1608–1639. https://doi.org/10.1177/0278364910371999.

    Article  Google Scholar 

  • Argall, B. D., Chernova, S., Veloso, M., & Browning, B. (2009). A survey of robot learning from demonstration. Robotics and Autonomous Systems, 57, 469–483.

    Article  Google Scholar 

  • Beeson, P., Modayil, J., & Kuipers, B. (2010). Factoring the mapping problem: Mobile robot map-building in the hybrid spatial semantic hierarchy. The International Journal of Robotics Research, 29(4), 428–459. https://doi.org/10.1177/0278364909100586.

    Article  Google Scholar 

  • Berlati, A., Scheel, O., Stefano, L. D., & Tombari, F. (2020). Ambiguity in sequential data: Predicting uncertain futures with recurrent models. IEEE Robotics and Automation Letters, 5(2), 2935–2942. https://doi.org/10.1109/LRA.2020.2974716.

    Article  Google Scholar 

  • Bonani, M., Longchamp, V., Magnenat, S., Rétornaz, P., Burnier, D., Roulet, G., Vaussard, F., Bleuler, H, & Mondada, F. (2010). The marxbot, a miniature mobile robot opening new perspectives for the collective-robotic research. In Proceedings of the IEEE/RSJ international conference intelligent robots and systems (IROS) (pp. 4187-4193). IEEE.

  • Calinon, S. (2016). A tutorial on task-parameterized movement learning and retrieval. Intelligent Service Robotics, 9(1), 1–29. https://doi.org/10.1007/s11370-015-0187-9.

    Article  Google Scholar 

  • Calinon, S., & Lee, D. (2016). Learning control. In A. Goswami & P. Vadakkepat (Eds.), Humanoid robotics: A reference (pp. 1–52). Springer. https://doi.org/10.1007/978-94-007-7194-9_68-1.

  • Calinon, S., Guenter, F., & Billard, A. G. (2007). On learning, representing, and generalizing a task in a humanoid robot. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 37(2), 286–298. https://doi.org/10.1109/TSMCB.2006.886952.

    Article  Google Scholar 

  • Calinon, S., D’Halluin, F., Sauser, E. L., Caldwell, D. G., & Billard, A. G. (2010). Learning and reproduction of gestures by imitation. IEEE Robotics & Automation Magazine, 17(2), 44–54. https://doi.org/10.1109/MRA.2010.936947.

  • Chatzis, S. P., Korkinof, D., & Demiris, Y. (2012). A nonparametric Bayesian approach toward robot learning by demonstration. Robotics and Autonomous Systems, 60(6), 789–802. https://doi.org/10.1016/j.robot.2012.02.005.

    Article  Google Scholar 

  • Chen, N., Karl, M., & van der Smagt, P. (2016). Dynamic movement primitives in latent space of time-dependent variational autoencoders. In 2016 IEEE-RAS 16th international conference on humanoid robots (humanoids) (pp. 629-636). IEEE.

  • Daniel, C., Neumann, G., & Peters, J. (2012). Learning concurrent motor skills in versatile solution spaces. In 2012 IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 3591-3597). IEEE.

  • DelPreto, J., Lipton, J. I., Sanneman, L., Fay, A. J., Fourie, C., Choi, C., & Rus, D. (2020). Helping robots learn: A human–robot master–apprentice model using demonstrations via virtual reality teleoperation. In Proceedings of the IEEE international conference on robotics and automation (ICRA) (pp. 10226–10233). https://doi.org/10.1109/ICRA40945.2020.9196754.

  • Dong, S., & Williams, B. (2012). Learning and recognition of hybrid manipulation motions in variable environments using probabilistic flow tubes. International Journal of Social Robotics, 1–12,. https://doi.org/10.1007/s12369-012-0155-x.

  • Furgale, P., & Barfoot, T. D. (2010). Visual teach and repeat for long-range rover autonomy. Journal of Field Robotics, 27(5), 534–560. https://doi.org/10.1002/rob.20342.

    Article  Google Scholar 

  • Grollman, D. H., & Jenkins, O. C. (2010). Can we learn finite state machine robot controllers from interactive demonstration? In O. Sigaud & J. Peters (Eds.), From motor learning to interaction learning in robots. Studies in computational intelligence (Vol. 264, pp. 407–430). Springer. https://doi.org/10.1007/978-3-642-05181-4_17.

  • Havoutis, I., & Calinon, S. (2019). Learning from demonstration for semi-autonomous teleoperation. Autonomous Robots, 43, 713–726.

    Article  Google Scholar 

  • Infantes, G., Ghallab, M., & Ingrand, F. (2011). Learning the behavior model of a robot. Autonomous Robots, 30, 157–177. https://doi.org/10.1007/s10514-010-9212-1.

    Article  Google Scholar 

  • Karl, M., Soelch, M., Becker-Ehmck, Djamel P., Benbouzid, van der Smagt, P., & Bayer, J. (2017). Deep variational Bayes filters: Unsupervised learning of state space models from raw data. In 5th international conference on learning representations (ICLR), Toulon.

  • Keogh, E., Chu, S., Hart, D., & Pazzani, M. (2004). Segmenting time series: A survey and novel approach. In Data mining in time series databases. World Scientific.

  • Khansari-Zadeh, S. M., & Billard, A. (2011). Learning stable nonlinear dynamical systems with Gaussian Mixture Models. IEEE Transactions on Robotics, 27, 943–957.

    Article  Google Scholar 

  • Koenig, N., & Matarić, M. J. (2017). Robot life-long task learning from human demonstrations: A Bayesian approach. Autonomous Robots, 41, 1173–1188.

    Article  Google Scholar 

  • Konidaris, G., Kuindersma, S., Grupen, R., & Barto, A. (2012). Robot learning from demonstration by constructing skill trees. The International Journal of Robotics Research, 31(3), 360–375.

    Article  Google Scholar 

  • Loula, J., Allen, K., Silver, T., & Tenenbaum, J. (2020). Learning constraint-based planning models from demonstrations. In IEEE/RSJ international conference on intelligent robots and systems (IROS).

  • MacTavish, K., Paton, M., & Barfoot, T. D. (2018). Selective memory: Recalling relevant experience for long-term visual localization. Journal of Field Robotics, 35(8), 1265–1292. https://doi.org/10.1002/rob.21838.

    Article  Google Scholar 

  • Magnenat, S., Waibel, M., & Beyeler, A. (2009). Enki—An open source fast 2d robot simulator. https://github.com/enki-community/enk.i

  • Magnenat, S., Philippsen, R., & Mondada, F. (2012a). Autonomous construction using scarce resources in unknown environments. Autonomous Robots, 33, 467–485. https://doi.org/10.1007/s10514-012-9296-x.

    Article  Google Scholar 

  • Magnenat, S., Pradalier, C., & Colas, F. (2012b). Towards non-parametric Bayesian learning of robot behaviors from demonstration. In Bayesian nonparametric models for reliable planning and decision-making under uncertainty, NIPS 2012.

  • Maye, J., Triebel, R., Spinello, L., & Siegwart, R. (2011). Bayesian on-line learning of driving behaviors. In Proceedings of the IEEE international conference on robotics and automation (ICRA) (pp. 4341–4346). IEEE. https://doi.org/10.1109/ICRA.2011.5980414.

  • Mondada, F., Bonani, M., Raemy, X., Pugh, J., Cianci, C., Klaptocz, A., Magnenat, S., Zufferey, J.-C., Floreano, D., & Martinoli, A. (2009). The e-puck, a robot designed for education in engineering. In Proceedings of the 9th conference on autonomous robot systems and competitions (pp. 59–65).

  • Mühlig, M., Gienger, M., & Steil, J. J. (2012). Interactive imitation learning of object movement skills. Autonomous Robots, 32(2), 97–114. https://doi.org/10.1007/s10514-011-9261-0.

    Article  Google Scholar 

  • Ng, A. Y., & Russell, S. J. (2000). Algorithms for inverse reinforcement learning. In ICML (Vol. 1, p. 2).

  • Niekum, S., Osentoski, S., Konidaris, G., Chitta, S., Marthi, B., & Barto, A. G. (2015). Learning grounded finite-state representations from unstructured demonstrations. The International Journal of Robotics Research, 34(2), 131–157. https://doi.org/10.1177/0278364914554471.

    Article  Google Scholar 

  • Pastor, P., Kalakrishnan, M., Meier, F., Stulp, F., Buchli, J., Theodorou, E., & Schaal, S. (2013). From dynamic movement primitives to associative skill memories. Robotics and Autonomous Systems, 61(4), 351–361. https://doi.org/10.1016/j.robot.2012.09.017.

    Article  Google Scholar 

  • Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., et al. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830.

    MathSciNet  MATH  Google Scholar 

  • Pradalier, C., & Bessière, P. Perceptual navigation around a sensori-motor trajectory. In Proceedings of the IEEE international conference on robotics and automation (ICRA) (vol. 4, pp. 3831–3836). IEEE.

  • Rasmussen, C. E., & Williams, C. K. I. (2006). Gaussian processes for machine learning. MIT Press.

  • Rochat, F., Schoeneich, P., Bonani, M., Magnenat, S., Mondada, F., Bleuler, H., & Hürzeler, C. (2010). Design of magnetic switchable device (MSD) and applications in climbing robot. In Proceedings of the 13th international conference on climbing and walking robots (pp. 375–382). World Scientific.

  • Russell, S. (1998). Learning agents for uncertain environments (extended abstract). In Proceedings of the eleventh annual conference on computational learning theory—COLT’98 (pp. 101–103). https://doi.org/10.1145/279943.279964.

  • Schulman, J., Duan, Y., Ho, J., Lee, A., Awwal, I., Bradlow, H., et al. (2014). Motion planning with sequential convex optimization and convex collision checking. The International Journal of Robotics Research, 33, 1251–1270.

    Article  Google Scholar 

  • Siciliano, B., & Khatib, O. (Eds.). (2016). Springer handbook of robotics. Springer.

  • Soans, N., Asali, E., Hong, Y., & Doshi, P. (2020). Sa-net: Robust state-action recognition for learning from observations. In Proceedings of the IEEE international conference on robotics and automation (ICRA) (pp. 2153–2159). https://doi.org/10.1109/ICRA40945.2020.9197393.

  • Vasquez, D., Okal, B., & Arras, K. (2013). Inverse Reinforcement Learning algorithms and features for robot navigation in crowds: An experimental comparison. In IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 1341–1346). https://doi.org/10.1109/IROS.2014.6942731.

  • Wang, Y., Yao, H., & Zhao, S. (2016). Auto-encoder based dimensionality reduction. Neurocomputing, 184, 232–242. https://doi.org/10.1016/j.neucom.2015.08.104.

    Article  Google Scholar 

  • Williams, C. K. I. & Rasmussen, C E. Gaussian processes for regression. In Advances in neural information processing systems (pp. 514–520).

  • Wu, X., & Kofman, J. (2008). Human-inspired robot task learning from human teaching. In Proceedings of the IEEE international conference on robotics and automation (ICRA) (pp. 3334–3339). IEEE. https://doi.org/10.1109/ROBOT.2008.4543719.

  • Xie, L., Markham, A., & Trigoni, N. (2020). Snapnav: Learning mapless visual navigation with sparse directional guidance and visual reference. In Proceedings of the IEEE international conference on robotics and automation (ICRA) (pp. 1682–1688). https://doi.org/10.1109/ICRA40945.2020.9197523.

  • Yang, W., Strokina, N., Serbenyuk, N., Ghabcheloo, R., & Kämäräinen, J. (2020). Learning a pile loading controller from demonstrations (pp. 4427–4433). https://doi.org/10.1109/ICRA40945.2020.9196907.

Download references

Acknowledgements

This work was mostly conducted at the Autonomous Systems Laboratory of ETH Zürich. We thank Philipp Krüsi for his help conducting the experiments, and Cédric Pradalier for his insightful comments on the algorithm. We thank the Mobots group of Francesco Mondada at EPFL for the access to the marXbot robot. This work was supported by the NIFTi (FP7-247870), myCopter (FP7-266470) and Noptilus (FP7-270180) European projects and by Willow Garage through a 3-month visiting researcher grant.

Author information

Authors and Affiliations

  1. Game Technology Center, ETH Zürich, Zürich, Switzerland

    Stéphane Magnenat

  2. Université de Lorraine, CNRS, Inria, LORIA, F-54000, Nancy, France

    Francis Colas

Authors
  1. Stéphane Magnenat
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Francis Colas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Francis Colas.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Magnenat, S., Colas, F. A Bayesian tracker for synthesizing mobile robot behaviour from demonstration. Auton Robot 45, 1077–1096 (2021). https://doi.org/10.1007/s10514-021-10019-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-021-10019-4

Keywords

Search

Navigation