Skip to main content
SpringerLink
Log in

Quantitative analysis of robot gesticulation behavior

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

Social robot capabilities, such as talking gestures, are best produced using data driven approaches to avoid being repetitive and to show trustworthiness. However, there is a lack of robust quantitative methods that allow to compare such methods beyond visual evaluation. In this paper a quantitative analysis is performed that compares two Generative Adversarial Networks based gesture generation approaches. The aim is to measure characteristics such as fidelity to the original training data, but at the same time keep track of the degree of originality of the produced gestures. Principal Coordinate Analysis and procrustes statistics are performed and a new Fréchet Gesture Distance is proposed by adapting the Fréchet Inception Distance to gestures. These three techniques are taken together to asses the fidelity/originality of the generated gestures.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. https://www.softbankrobotics.com/us/pepper.

  2. http://doc.aldebaran.com/2-5/software/choregraphe/index.html.

  3. https://www.youtube.com/watch?v=h9wpMEH8JQc.

References

  • Alibeigi, M., Rabiee, S., & Ahmadabadi, M. N. (2017). Inverse kinematics based human mimicking system using skeletal tracking technology. Journal of Intelligent & Robotic Systems, 85(1), 27–45.

    Article  Google Scholar 

  • Barratt, S., & Sharma, R. (2018). A note on the inception score. arXiv:1801.01973.

  • Beck, A., Yumak, Z., & Magnenat-Thalmann, N. (2017). Body movements generation for virtual characters and social robots. In Social signal processing, chap. 20, pp. 273–286. Cambridge University Press.

  • Becker-Asano, C., & Ishiguro, H. (2011). Evaluating facial displays of emotion for the android robot Geminoid F. In 2011 IEEE Workshop on Affective Computational Intelligence (WACI), pp. 1–8. https://doi.org/10.1109/WACI.2011.5953147.

  • Borji, A. (2019). Pros and cons of GAN evaluation measures. Computer Vision and Image Understanding, 179, 41–65.

    Article  Google Scholar 

  • Breuleux, O., Bengio, Y., & Vincent, P. (2010). Unlearning for better mixing. Montreal: Universite de Montreal/DIRO.

    Google Scholar 

  • Calinon, S., D’halluin, F., Sauser, E. L., Cakdwell, D. G., & Billard, A. G. (2004). Learning and reproduction of gestures by imitation. In International Conference on Intelligent Robots and Systems, pp. 2769–2774.

  • Cao, Z., Hidalgo, G., Simon, T., Wei, S. E., & Sheikh, Y. (2018). OpenPose: Realtime multi-person 2D pose estimation using Part Affinity Fields. In arXiv:1812.08008.

  • Carpinella, C., Wyman, A., Perez, M., & Stroessner, S. (2017). The robotic social attributes scale (RoSAS): Development and validation. In 17th Human Robot Interaction, pp. 254–262. https://doi.org/10.1145/2909824.3020208.

  • Cerrato, L., & Campbell, N. (2017). Engagement in dialogue with social robots. In K. Jokinen & G. Wilcock (Eds.), Dialogues with social robots; Enablements, analyses, and evalution (pp. 313–319). Singapore: Springer Singapore. https://doi.org/10.1007/978-981-10-2585-3_25.

  • Eielts, C., Pouw, W., Ouwehand, K., van Gog, T., Zwaan, R. A., & Paas, F. (2020). Co-thought gesturing supports more complex problem solving in subjects with lower visual working-memory capacity. Psychological Research, 84(2), 502–513. https://doi.org/10.1007/s00426-018-1065-9.

    Article  Google Scholar 

  • Gao, X., Yun, C., Jin, H., & Gao, Y. (2016). Calibration method of robot base frame using procrustes analysis. In 2016 Asia-Pacific Conference on Intelligent Robot Systems (ACIRS), pp. 16–20. IEEE.

  • Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., et al. (2014). Generative adversarial nets. In Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, & K. Q. Weinberger (Eds.), Advances in neural information processing systems (Vol. 27, pp. 2672–2680). Curran Associates, Inc.

  • Gower, J. (1985). Encyclopedia of statistical sciences, chap. Measures of similarity, dissimilarity and distance (Vol. 5). New York: Wiley.

    Google Scholar 

  • Gower, J. C. (1966). Some distance properties of latent root and vector methods used in multivariate analysis. Biometrika, 53(3–4), 325–338.

    Article  MathSciNet  Google Scholar 

  • Gower, J. C., Dijksterhuis, G. B., et al. (2004). Procrustes problems (Vol. 30). Oxford: Oxford University Press on Demand.

    Book  Google Scholar 

  • Hasegawa, D., Kaneko, N., Shirakawa, S., Sakuta, H., & Sumi, K. (2018). Evaluation of speech-to-gesture generation using bi-directional LSTM network. In 18th International Conference on Intelligent Virtual Agents, pp. 79–86. https://doi.org/10.1145/3267851.3267878.

  • Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., & Hochreiter, S. (2017). GANs trained by a two time-scale update rule converge to a local Nash equilibrium. In Advances in neural information processing systems, pp. 6626–6637.

  • Hotelling, H. (1993). Analysis of a complex of statistical variables into principal components. Journal of Educational Psychology, 24(6), 417–441. https://doi.org/10.1037/h0071325.

    Article  Google Scholar 

  • Jarque-Bou, N. J., Scano, A., Atzori, M., & Müller, H. (2019). Kinematic synergies of hand grasps: A comprehensive study on a large publicly available dataset. Journal of Neuroengineering and Rehabilitation, 16(1), 63.

    Article  Google Scholar 

  • Kofinas, N., Orfanoudakis, E., & Lagoudakis, M. G. (2015). Complete analytical forward and inverse kinematics for the nao humanoid robot. Journal of Intelligent & Robotic Systems, 77(2), 251–264. https://doi.org/10.1007/s10846-013-0015-4.

    Article  Google Scholar 

  • Kucherenko, T., Hasegawa, D., Kaneko, N., Henter, G., & Kjellström, H. (2019). On the importance of representations for speech-driven gesture generation. In 18th International Conference on Autonomous Agents and MultiAgent Systems (AAMAS), pp. 2072–2074.

  • Kucherenko, T., Jonell, P., van Waveren, S., Eje Henter, G., Alexanderson, S., Leite, I., & Kjellström, H. (2020). Gesticulator: A framework for semantically-aware speech-driven gesture generation. arXiv:2001.09326.

  • Kullback, S. (1997). Information theory and statistics. North Chelmsford: Courier Corporation.

    MATH  Google Scholar 

  • Lhommet, M., & Marsella, S. (2015). The oxford handbook of affective computing, chap. Expressing emotion through posture and gesture (pp. 273–285). Oxford: Oxford University Press.

    Google Scholar 

  • Makondo, N., Rosman, B., & Hasegawa, O. (2015). Knowledge transfer for learning robot models via local procrustes analysis. In 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids), pp. 1075–1082. IEEE.

  • Maset, E., Scalera, L., Zonta, D., Alba, I., Crosilla, F., & Fusiello, A. (2020). Procrustes analysis for the virtual trial assembly of large-size elements. Robotics and Computer-Integrated Manufacturing, 62, 101885.

    Article  Google Scholar 

  • McNeill, D. (1992). Hand and mind: What gestures reveal about thought. Chicago: University of Chicago press.

    Google Scholar 

  • Mukherjee, S., Paramkusam, D., & Dwivedy, S. K. (2015). Inverse kinematics of a NAO humanoid robot using Kinect to track and imitate human motion. In International Conference on Robotics, Automation, Control and Embedded Systems (RACE). IEEE.

  • Nazeri, K., Ng, E., Joseph, T., Qureshi, F. Z., & Ebrahimi, M. (2019). Edgeconnect: Generative image inpainting with adversarial edge learning. arXiv:1901.00212.

  • Pan, M., Croft, E., & Niemeyer, G. (2018). Evaluating social perception of human-to-robot handovers using the robot social attributes scale (rosas). In ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 443–451. https://doi.org/10.1145/3171221.3171257.

  • Park, G., & Konno, A. (2015). Imitation learning framework based on principal component analysis. Advanced Robotics, 29(9), 639–656. https://doi.org/10.1080/01691864.2015.1007084.

    Article  Google Scholar 

  • Park, T., Liu, M. Y., Wang, T. C., & Zhu, J. Y. (2019). Semantic image synthesis with spatially-adaptive normalization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2337–2346.

  • Poubel, L. P. (2013). Whole-body online human motion imitation by a humanoid robot using task specification. Master’s thesis, Ecole Centrale de Nantes–Warsaw University of Technology.

  • Rodriguez, I., Astigarraga, A., Jauregi, E., Ruiz, T., & Lazkano, E. (2014). Humanizing NAO robot teleoperation using ROS. In International Conference on Humanoid Robots (Humanoids).

  • Rodriguez, I., Martínez-Otzeta, J. M., Irigoien, I., & Lazkano, E. (2019). Spontaneous talking gestures using generative adversarial networks. Robotics and Autonomous Systems, 114, 57–65.

    Article  Google Scholar 

  • Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., & Chen, X. (2016). Improved techniques for training GANs. In Advances in neural information processing systems, pp. 2234–2242.

  • Suguitan, M., Gomez, R., & Hoffman, G. (2020). MoveAE: Moditying affective robot movements using classifying variational autoencoders. In ACM/IEEE International Conference on Human Robot Interaction (HRI), pp. 481–489. https://doi.org/10.1145/3267851.3267878.

  • Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015). Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1–9.

  • Theis, L., & Bethge, M. (2015). Generative image modeling using spatial lstms. In Advances in Neural Information Processing Systems, pp. 1927–1935.

  • Theis, L., van den Oord, A., & Bethge, M. (2015). A note on the evaluation of generative models. CoRR arXiv:1511.01844.

  • Velner, E., Boersma, P. P., & de Graaf, M. M. (2020). Intonation in robot speech: Does it work the same as with people? In ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 569–578.

  • Wolfert, P., Kucherenko, T., Kjelström, H., & Belpaeme, T. (2019). Should beat gestures be learned or designed? A benchmarking user study. In ICDL-EPIROB 2019 Workshop on Naturalistic Non-Verbal and Affective Human-Robot Interactions, p. 4.

  • Wood, M., Simmatis, L., Boyd, J. G., Scott, S., & Jacobson, J. (2018). Using principal component analysis to reduce complex datasets produced by robotic technology in healthy participants. Journal of NeuroEngineering and Rehabilitation, 15, https://doi.org/10.1186/s12984-018-0416-5.

  • Wu, Y., Donahue, J., Balduzzi, D., Simonyan, K., & Lillicrap, T. (2019). LOGAN: Latent optimisation for generative adversarial networks. arXiv:1912.00953.

  • Zabala, U., Rodriguez, I., Martínez-Otzeta, J. M., & Lazkano, E. (2019). Learning to gesticulate by observation using a deep generative approach. In 11th International Conference on Social Robotics (ICSR) (2019 (Accepted)). arXiv:1909.01768.

  • Zhang, H., Xu, T., Li, H., Zhang, S., Wang, X., Huang, X., et al. (2018). Stackgan++: Realistic image synthesis with stacked generative adversarial networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(8), 1947–1962.

    Article  Google Scholar 

  • Zhang, Z., Niu, Y., Kong, L. D., Lin, S., & Wang, H. (2019). A real-time upper-body robot imitation system. International Journal of Robotics and Control, 2, 49–56. https://doi.org/10.5430/ijrc.v2n1p49.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Artificial Intelligence, Faculty of Informatics, University of Basque Country (UPV/EHU), 20018, Donostia, Spain

    Unai Zabala, Igor Rodriguez, José María Martínez-Otzeta, Itziar Irigoien & Elena Lazkano

Authors
  1. Unai Zabala
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Igor Rodriguez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. José María Martínez-Otzeta
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Itziar Irigoien
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Elena Lazkano
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Elena Lazkano.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work has been partially supported by the Basque Government (IT900-16 and Elkartek 2018/00114), the Spanish Ministry of Economy and Competitiveness (RTI 2018-093337-B-100, MINECO/FEDER,EU).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zabala, U., Rodriguez, I., Martínez-Otzeta, J.M. et al. Quantitative analysis of robot gesticulation behavior. Auton Robot 45, 175–189 (2021). https://doi.org/10.1007/s10514-020-09958-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-020-09958-1

Keywords

Search

Navigation