Skip to main content

Advertisement

SpringerLink
Log in

Cognitive System Framework for Brain-Training Exercise Based on Human-Robot Interaction

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

Every 3 seconds, someone develops dementia worldwide. Brain-training exercises, preferably involving also physical activity, have shown their potential to monitor and improve the brain function of people affected by Alzheimer disease (AD) or mild cognitive impairment (MCI). This paper presents a cognitive robotic system designed to assist mild dementia patients during brain-training sessions of sorting tokens, an exercise inspired by the Syndrom KurzTest neuropsychological test (SKT). The system is able to perceive, learn and adapt to the user’s behaviour and is composed of two main modules. The adaptive module based on representing the human-robot interaction as a planning problem, that can adapt to the user performance offering different encouragement and recommendation actions using both verbal and gesture communication in order to minimize the time spent to solve the exercise. As safety is a very important issue, the cognitive system is enriched with a safety module that monitors the possibility of physical contact and reacts accordingly. The cognitive system is presented as well as its embodiment in a real robot. Simulated experiments are performed to (i) evaluate the adaptability of the system to different patient use-cases and (ii) validate the coherence of the proposed safety module. A real experiment in the lab, with able users, is used as preliminary evaluation to validate the overall approach. Results in laboratory conditions show that the two presented modules effectively provide additional and essential functionalities to the system, although further work is necessary to guarantee robustness and timely response of the robot before testing it with patients.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. http://www.fundacioace.org/en

  2. https://www.barrett.com/wam-arm/

  3. https://youtu.be/xtS1yDIyrmQ

  4. https://youtu.be/pQ-RM_l1YkI

References

  1. World Alzheimer Report The state of the art of dementia research: new frontiers. Alzheimer’s Disease International (ADI). 2018: 1–48. https://www.alz.co.uk/research/WorldAlzheimerReport2018.pdf.

  2. Abdelnour C, Tantyna N, Hernandez J, Giakoumis D, Ribes JC, Gerlowska J, Skrobas U, Korchut A, Grabowska K, Szklener S, Hernandez I, Rosende-Roca M, Mauleon A, Vargas L, Alegret M, Espinosa A, Ortega G, Sanchez D, Rodriguez-Gomez O, Sanabria A, Perez A, Canabate P, Moreno M, Preckler S, Ruiz A, Rejdak K, Tzovaras D, Tarraga L, Boada M. Are there differences in the opinion of patients with Alzheimer’s disease and their caregivers about having support from a service robot at home?. Alzheimer’s Dementia: J Alzheimer’s Assoc 2017;13(7):P1412–P1413.

    Article  Google Scholar 

  3. Alami R, Clodic A, Montreuil V, Sisbot E, Chatila R. Task planning for human-robot interaction. Proceedings of the 2005 Joint Conference on Smart Objects and Ambient Intelligence: innovative context-aware services: usages and technologies (october). 2005. p. 81–85.

  4. Andriella A, Alenyȧ G, Hernȧndez-Farigola J, Torras C. Deciding the different robot roles for patient cognitive training. Int J Hum Comput Stud 2018;117:20–29.

    Article  Google Scholar 

  5. Bahar-Fuchs A, Webb S, Bartsch L, Clare L, Rebok G, Cherbuin N, Anstey KJ. Tailored and adaptive computerized cognitive training in older adults at risk for dementia: a randomized controlled trial. J Alzheimers Dis 2017;60(3):889–911.

    Article  Google Scholar 

  6. Beetz M, Bartels G, Albu-Schaffer A, Balint-Benczedi F, Belder R, Bebler D, Haddadin S, Maldonado A, Mansfeld N, Wiedemeyer T, Weitschat R, Worch JH. Robotic agents capable of natural and safe physical interaction with human co-workers. IEEE International Conference on Intelligent Robots and Systems; 2015. p. 6528–6535.

  7. Bhat AA, Mohan V. Goal-Directed reasoning and cooperation in robots in shared workspaces: an internal simulation based neural framework. Cogn Comput 2018;10(4):558–576.

    Article  Google Scholar 

  8. Broquėre X, Sidobre D, Herrera-aguilar I. 2009. Soft motion trajectory planner for service manipulator robot. ArXiv e-prints.

  9. Cambon S, Alami R, Gravot F. A hybrid approach to intricate motion, manipulation and task planning. Int J Robot Res 2009;28 (1):104–126.

    Article  Google Scholar 

  10. Carbone A, Finzi A, Orlandini A, Pirri F, Ugazio G. Augmenting situation awareness via model-based control in rescue robots. In: 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS. IEEE; 2005. p. 1549–1555.

  11. Colome A, Pardo D, Alenya G, Torras C. External force estimation during compliant robot manipulation. Proceedings - IEEE International Conference on Robotics and Automation. IEEE; 2013. p. 3535–3540.

  12. Colome A, Planells A, Torras C. A friction-model-based framework for reinforcement learning of robotic tasks in non-rigid environments. 2015 IEEE International Conference on Robotics and Automation (ICRA) (1); 2015. p. 5649–5654.

  13. De Boer C, Echlin HV, Rogojin A, Baltaretu BR, Sergio LE. Thinking-while-moving exercises may improve cognition in elderly with mild cognitive deficits: a proof-of-principle study. Dement Geriatric Cogn Disord Extra 2018;8(2):248– 258.

    Article  Google Scholar 

  14. De Giacomo G, Iocchi L, Nardi D, Rosati R. Moving a robot: the KR&r approach at work. In: Proceedings of the Fifth International Conference on the Principles of Knowledge Representation and Reasoning (KR’96); 1996. p. 198–209.

  15. De Giacomo G, Iocchi L, Nardi D, Rosati R. Planning with sensing for a mobile robot. Proceedings of ECP-97: Fourth European Conference on Planning; 1997. p. 158–170.

  16. Devin S, Clodic A, Alami R. About decisions during human-robot shared plan achievement: who should act and how? Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 10652 LNAI; 2017. p. 453–463.

  17. Dominey PF, Metta G, Nori F, Natale L. Anticipation and initiative in human-humanoid interaction. In: 2008 8Th IEEE-RAS International Conference on Humanoid Robots, Humanoids 2008. IEEE; 2008. p. 693–699.

  18. Erzigkeit H. SKT: a short cognitive performance test for assessing deficits of memory and attention. User’s Manual. Int Psychoger 2001;9(S1):115–121.

    Google Scholar 

  19. Fan J, Bian D, Zheng Z, Beuscher L, Newhouse PA, Mion LC, Sarkar N. A robotic coach architecture for elder care (ROCARE) based on multi-user engagement models. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2016;PP(99).

  20. Ferrein A, Fritz C, Lakemeyer G. On-line decision-theoretic Golog for unpredictable domains. KI2004 Advances in Artificial Intelligence 2004;3238:322–336.

    Google Scholar 

  21. Gates N, Valenzuela M. Cognitive exercise and its role in cognitive function in older adults. Current Psych Rep 2010;12(1):20–27.

    Article  Google Scholar 

  22. Gaugler J, James B, Johnson T, Scholz K, Weuve J. 2016 Alzheimer’s disease facts and figures. Alzheimer’s Dementia 2016;12(4):459–509.

    Article  Google Scholar 

  23. Gnjatoviċ M. Therapist-centered design of a robot’s dialogue behavior. Cogn Comput 2014;6(4):775–788.

    Article  Google Scholar 

  24. Golz S, Osendorfer C, Haddadin S. Using tactile sensation for learning contact knowledge: discriminate collision from physical interaction. 2015 IEEE International Conference on Robotics and Automation (ICRA); 2015. p. 3788–3794.

  25. Haddadin S, Albu-Schȧffer A, De Luca A, Hirzinger G. Collision detection and reaction: a contribution to safe physical human-robot interaction. In: 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS. IEEE; 2008. p. 3356–3363.

  26. Heinzmann J, Zelinsky A. Quantitative safety guarantees for physical human-robot interaction. Int J Robot Res 2003;22(7-8):479–504.

    Article  Google Scholar 

  27. Hoffman G, Breazeal C. Cost-Based anticipatory action selection for human–robot fluency. IEEE Trans Robot 2007;23(5):952–961.

    Article  Google Scholar 

  28. Husain F, Colome A, Dellen B, Alenya G, Torras C. Realtime tracking and grasping of a moving object from range video. Proceedings - IEEE International Conference on Robotics and Automation; 2014. p. 2617–2622.

  29. Kuliċ D, Croft E. Pre-collision safety strategies for human-robot interaction. Auton Robot 2007;22(2): 149–164.

    Article  Google Scholar 

  30. Laffranchi M, Tsagarakis NG, Caldwell DG. Safe human robot interaction via energy regulation control. IEEE/RSJ International Conference on Intelligent Robots and Systems, 2009. IROS 2009; 2009. p. 35–41.

  31. Lampit A, Valenzuela M, Gates N. Computerized cognitive training is beneficial for older adults. J Am Geriatr Soc 2015;63(12):2610–2612.

    Article  Google Scholar 

  32. Lasota P, Fong T, Shah JA. A survey of methods for safe human-robot interaction. Found Trends Robot 2017;5(3):261–349.

    Article  Google Scholar 

  33. Lasota P, Rossano G, Shah JA. Toward safe close-proximity human-robot interaction with standard industrial robots. In: IEEE International Conference on Automation Science and Engineering; 2014. p. 339–344.

  34. Lemaignan S, Warnier M, Sisbot EA, Clodic A, Alami R. Artificial cognition for social human–robot interaction: an implementation. Artif Intell 2017;247:45–69.

    Article  MathSciNet  Google Scholar 

  35. Levesque H, Lakemeyer G. Cognitive robotics. Found Artif Intell 2008;3(07):869–886.

    Article  Google Scholar 

  36. Li K, Fu Y. Prediction of human activity by discovering temporal sequence patterns. IEEE Trans Pattern Anal Mach Intell 2014;36(8):1644–1657.

    Article  Google Scholar 

  37. Mainprice J, Sisbot EA, Jaillet L, Cortės J, Alami R, Simėon T. Planning human-aware motions using a sampling-based costmap planner. In: Proceedings - IEEE International Conference on Robotics and Automation; 2011. p. 5012–5017.

  38. Marti̇n F, Agu̇ero C, Caṅas JM, Abella G, Beni̇tez R, Rivero S, Valenti M, Marti̇nez-marti̇n P. Robots in therapy for dementia patients. Journal of Physical Agents 2013;7(1):49–56.

    Google Scholar 

  39. Matthias B, Reisinger T. example application of ISO/TS 15066 to a collaborative assembly scenario. In: Inproceedings of the 47th International Symposium on Robotics; 2016. p 1–5.

  40. McCallum S, Boletsis C. Dementia games: a literature review of dementia-related serious games. In: Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics) 2013, vol. 8101 LNCS; 2013. p 15–27.

  41. McColl D, Nejat G. Meal-time with a socially assistive robot and older adults at a long-term care facility. J Human-Robot Interact 2013;2(1):152–171.

    Article  Google Scholar 

  42. Mohan V, Morasso P, Sandini G, Kasderidis S. Inference through embodied simulation in cognitive robots. Cogn Comput 2013;5(3):355–382.

    Article  Google Scholar 

  43. Nikolaidis S, Lasota P, Rossano G, Martinez C, Fuhlbrigge T, Shah J. Human-robot collaboration in manufacturing: quantitative evaluation of predictable, convergent joint action. In: 2013 44th International Symposium on Robotics, ISR 2013. IEEE; 2013. p 1–6.

  44. Pérula-Martínez R, Castro-González Á, Malfaz M, Alonso-Martín F, Salichs MA. Bioinspired decision-making for a socially interactive robot. Cogn Syst Res 2019;54:287–301.

    Article  Google Scholar 

  45. Ragaglia M, Bascetta L, Rocco P, Zanchettin AM. Integration of perception, control and injury knowledge for safe human-robot interaction. In: 2014 IEEE International Conference on Robotics and Automation (ICRA). IEEE; 2014. p 1196–1202.

  46. Salichs E, Fernández-Rodicio E, Castillo J, Castro-González Á, Malfaz M, Salichs MÁ. A social robot assisting in cognitive stimulation therapy. In: Demazeau Y, An B, Bajo J, and Fernández-Caballero A, editors. Advances in Practical Applications of Agents, Multi-Agent Systems, and Complexity: The PAAMS Collection. Cham: Springer International Publishing; 2018. p. 344–347.

  47. Sheikh ZC, Simon T, Wei SE. 2017. Yaser: Realtime multi-person 2D pose estimation using part affinity fields CVPR.

  48. Sisbot EA, Alami R. A human-aware manipulation planner. IEEE Trans Robot 2012;28(5):1045–1057.

    Article  Google Scholar 

  49. Sisbot EA, Marin-Urias LF, Broquėre X, Sidobre D, Alami R. Synthesizing robot motions adapted to human presence: a planning and control framework for safe and socially acceptable robot motions. Int J Soc Robot 2010;2(3):329–343.

    Article  Google Scholar 

  50. Tapus A, Mataric M. Socially assistive robotic music therapist for maintaining attention of older adults with cognitive impairments. Proceedings of AAAI Fall Symposium AI in Eldercare; 2008. p. 124–127.

  51. Tapus A, Mataric MJ, Scassellati B. Socially assistive robotics [Grand challenges of robotics]. IEEE Robot Autom Mag 2007;14(1):35–42.

    Article  Google Scholar 

Download references

Acknowledgements

We would like to thank Carla Abdelnour, Joan Hernandez and Natalia Tantinya from Fundació ACE for the fruitful discussions and the help in the design of the sorting tokens exercise.

Funding

This project has been partially funded by the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement SOCRATES (No 721619), by the Spanish Ministry of Science and Innovation HuMoUR (TIN2017-90086-R), and by the Spanish State Research Agency through the María de Maeztu Seal of Excellence to IRI (MDM-2016-0656).

Author information

Authors and Affiliations

  1. Institut de Robòtica i Informàtica Industrial, CSIC-UPC, C/Llorens i Artigas 4-6, 08028, Barcelona, Spain

    Antonio Andriella, Carme Torras & Guillem Alenyà

Authors
  1. Antonio Andriella
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carme Torras
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guillem Alenyà
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Antonio Andriella.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Ethical Approval

All participants were able adult students at the Institut de Robòtica i Informàtica Industrial in Barcelona (Spain) and were informed that if at any point they did not wish to continue with the study, they could withdraw from the experiment. The study was approved by the Ethical Committee of the Consejo Superior de Investigaciones Científicas (reference code 048/2018).

Informed Consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Andriella, A., Torras, C. & Alenyà, G. Cognitive System Framework for Brain-Training Exercise Based on Human-Robot Interaction. Cogn Comput 12, 793–810 (2020). https://doi.org/10.1007/s12559-019-09696-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-019-09696-2

Keywords

Search

Navigation