Skip to main content
SpringerLink
Log in

Von der Fremdbeurteilung des Schmerzes zur automatisierten multimodalen Messung der Schmerzintensität

Narrativer Review zum Stand der Forschung und zur klinischen Perspektive

From external assessment of pain to automated multimodal measurement of pain intensity

Narrative review of state of research and clinical perspectives

  • Übersichten
  • Published:
Der Schmerz Aims and scope Submit manuscript

Zusammenfassung

Hintergrund

Bei Patienten mit eingeschränkten Kommunikationsfähigkeiten ist der Einsatz von konventionellen Skalen bzw. Fremdbeurteilung nur bedingt oder nicht möglich, eine multimodale Schmerzerkennung basierend auf Algorithmen der künstlichen Intelligenz (KI) könnte eine Lösung darstellen.

Fragestellung

Überblick über Methoden der automatisierten multimodalen Schmerzmessung und deren Erkennungsraten, welche mit Algorithmen der KI berechnet wurden.

Methoden

Zur Darstellung des aktuellen Forschungsstands wurden mittels selektiver Literaturrecherche im April 2018 in der Datenbank Web of Science 101 Studien zur automatisierten Schmerzerkennung identifiziert. 14 dieser Studien berichten speziell Erkennungsraten der automatisierten multimodalen Schmerzmessung, diese werden als Fokus des vorliegenden narrativen Reviews dargestellt.

Ergebnisse

Die Varianz der Erkennungsraten liegt bei 52,9–55,0 % (Schmerzschwelle) bzw. 66,8–85,7 % (Schmerztoleranz), bei 9 Studien ist die Erkennungsrate ≥80 % (Schmerztoleranz), eine Arbeit berichtet hohe Erkennungsraten von 79,3 % (Schmerzschwelle) und 90,9 % (Schmerztoleranz).

Schlussfolgerungen

Schmerzen werden, basierend auf Fremdbeobachtungsskalen, grundsätzlich multimodal erfasst. In Bezug auf die automatisierte Schmerzerkennung ist auf Basis der 14 ausgewählten Studien noch nicht abschließend beantwortet, ob eine multimodale Schmerzerkennung einer unimodalen überlegen ist. Im klinischen Kontext könnte eine multimodale Schmerzerkennung von Vorteil sein, da dieser Ansatz flexibler einsetzbar ist. Der Algorithmus könnte im Fall der fehlenden Verfügbarkeit einer Modalität, z. B. der elektrodermalen Aktivität bei Handverbrennungen, auf andere Modalitäten (Video) zurückgreifen und somit fehlende Informationen kompensieren.

Abstract

Background

In patients with limited communication skills, the use of conventional scales or external assessment is only possible to a limited extent or not at all. Multimodal pain recognition based on artificial intelligence (AI) algorithms could be a solution.

Objective

Overview of the methods of automated multimodal pain measurement and their recognition rates that were calculated with AI algorithms.

Methods

In April 2018, 101 studies on automated pain recognition were found in the Web of Science database to illustrate the current state of research. A selective literature review with special consideration of recognition rates of automated multimodal pain measurement yielded 14 studies, which are the focus of this review.

Results

The variance in recognition rates was 52.9–55.0% (pain threshold) and 66.8–85.7%; in nine studies the recognition rate was ≥80% (pain tolerance), while one study reported recognition rates of 79.3% (pain threshold) and 90.9% (pain tolerance).

Conclusion

Pain is generally recorded multimodally, based on external observation scales. With regard to automated pain recognition and on the basis of the 14 selected studies, there is to date no conclusive evidence that multimodal automated pain recognition is superior to unimodal pain recognition. In the clinical context, multimodal pain recognition could be advantageous, because this approach is more flexible. In the case of one modality not being available, e.g., electrodermal activity in hand burns, the algorithm could use other modalities (video) and thus compensate for missing information.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Literatur

  1. Turk DC, Melzack R (2011) Preface. In: Handbook of pain assessment. Guilford, New York

    Google Scholar 

  2. Melzack R, Casey K (1968) Sensory, motivational and central control determinants of pain: a new conceptual model. In: Kenshalo D (Hrsg) The skin senses: Proceedings. Charles C. Thomas, Springfield, S 423–443

    Google Scholar 

  3. Magerl W, Treede R‑D (2017) Physiologie von Nozizeption und Schmerz. In: Schmerzpsychotherapie. Springer, Berlin, Heidelberg, S 31–72

    Google Scholar 

  4. Benarroch EE (2001) Pain-autonomic interactions: a selective review. Clin Auton Res 11(6):343–349

    CAS  PubMed  Google Scholar 

  5. Stewart G, Panickar A (2013) Role of the sympathetic nervous system in pain. Anaesth Intensive Care Med 14(12):524–527

    Google Scholar 

  6. Williams ACDC (2002) Facial expression of pain: an evolutionary account. Behav Brain Sci 25(4):439–455

    PubMed  Google Scholar 

  7. Craig KD, Prkachin KM, Grunau RE (2011) The facial expression of pain. In: Turk DC, Melzack R (Hrsg) Handbook of pain assessment. Guilford, New York

    Google Scholar 

  8. Prkachin KM, Solomon PE (2008) The structure, reliability and validity of pain expression: evidence from patients with shoulder pain. Pain 139(2):267–274

    PubMed  Google Scholar 

  9. Prkachin KM (1992) The consistency of facial expressions of pain: a comparison across modalities. Pain 51(3):297–306

    CAS  PubMed  Google Scholar 

  10. Kunz M, Scharmann S, Hemmeter U, Schepelmann K, Lautenbacher S (2007) The facial expression of pain in patients with dementia. Pain 133(1):221–228

    PubMed  Google Scholar 

  11. Ekman P, Friesen W, Hager J (2002) Facial action coding system: Manual and Investigator`s Guide. Research Nexus, Salt Lake City

  12. Ekman P (1992) An argument for basic emotions. Cogn Emot 6:169

    Google Scholar 

  13. Simon D, Craig KD, Gosselin F, Belin P, Rainville P (2008) Recognition and discrimination of prototypical dynamic expressions of pain and emotions. Pain 135(1):55–64

    PubMed  Google Scholar 

  14. Kunz M, Lautenbacher S (2019) Schmerz hat viele Gesichter. Schmerzpatient 2(4):158–163

    Google Scholar 

  15. Lopez-Martinez D, Rudovic O, Picard R (2017) Personalized automatic estimation of self-reported pain intensity from facial expressions. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).. IEEE, Hawaii, S 2318–2327

  16. Werner P, Al-Hamadi A, Walter S (2017) Analysis of facial expressiveness during experimentally induced heat pain. In: International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW), S 176–180

    Google Scholar 

  17. Prkachin KM, Craig KD (1995) Expressing pain: the communication and interpretation of facial pain signals. J Nonverbal Behav 19(4):191–205

    Google Scholar 

  18. Aung MSH, Kaltwang S, Romera-Paredes B, Martinez B, Singh A, Cella M et al (2016) The automatic detection of chronic pain-related expression: requirements, challenges and the multimodal emopain dataset. IEEE Trans Affect Comput 7(4):435–451

    PubMed  Google Scholar 

  19. Walsh J, Eccleston C, Keogh E (2014) Pain communication through body posture: The development and validation of a stimulus set. Pain 155(11):2282–2290

    PubMed  Google Scholar 

  20. Werner P, Al-Hamadi A, Limbrecht-Ecklundt K, Walter S, Traue HC (2018) Head movements and postures as pain behavior. PLoS ONE 13(2):e192767

    PubMed  PubMed Central  Google Scholar 

  21. Handel E (2010) Praxishandbuch ZOPA©. Schmerzeinschätzung bei Patienten mit kognitiven und/oder Bewusstseinsbeeinträchtigungen. Huber, Bern

    Google Scholar 

  22. Nilges P (2013) Klinische Schmerzmessung. In: Baron R, Koppert W, Strumpf M, Willweber-Strumpf A (Hrsg) Praktische Schmerzmedizin. Springer, Berlin Heidelberg, S 79–85

    Google Scholar 

  23. Pagé MG, Katz J, Stinson J, Isaac L, Martin-Pichora AL, Campbell F (2012) Validation of the numerical rating scale for pain intensity and unpleasantness in pediatric acute postoperative pain: Sensitivity to change over time. J Pain 13(4):359–369

    PubMed  Google Scholar 

  24. Thong ISK, Jensen MP, Miró J, Tan G (2018) The validity of pain intensity measures: What do the NRS, VAS, VRS, and FPS‑R measure? Scand J Pain 18(1):99–107

    PubMed  Google Scholar 

  25. Duncan GH, Bushnell CM, Lavigne GJ (1989) Comparison of verbal and visual analogue scales for measuring the intensity and unpleasantness of experimental pain. Pain 37(3):295–303

    CAS  PubMed  Google Scholar 

  26. Herr K, Coyne PJ, McCaffery M, Manworren R, Merkel S (2011) Pain assessment in the patient unable to self-report: position statement with clinical practice recommendations. Pain Manag Nurs 12(4):230–250

    PubMed  Google Scholar 

  27. Lautenbacher S, Kunz M (2019) Schmerzerfassung bei Patienten mit Demenz. Schmerz 33(6):563–575

    PubMed  Google Scholar 

  28. Basler HD, Hüger D, Kunz R, Luckmann J, Lukas A, Nikolaus T et al (2006) Beurteilung von schmerz bei demenz (BESD). Untersuchung zur Validität eines Verfahrens zur Beobachtung des Schmerzverhaltens. Schmerz 20(6):519–526

    CAS  PubMed  Google Scholar 

  29. Payen JF, Bru O, Bosson JL, Lagrasta A, Novel E, Deschaux I et al (2001) Assessing pain in critically ill sedated patients by using a behavioral pain scale. Crit Care Med 29(12):2258–2263

    CAS  PubMed  Google Scholar 

  30. Chanques G, Payen J‑F, Mercier G, de Lattre S, Viel E, Jung B et al (2009) Assessing pain in non-intubated critically ill patients unable to self report: an adaptation of the Behavioral Pain Scale. Intensive Care Med 35(12):2060

    PubMed  Google Scholar 

  31. Craig KD (2009) The social communication model of pain. Can Psychol 50(1):22

    Google Scholar 

  32. Reddy KSK, Naidu MUR, Rani PU, Rao TRK (2012) Human experimental pain models: A review of standardized methods in drug development. J Res Med Sci 17(6):587–595

    PubMed  PubMed Central  Google Scholar 

  33. Kranjec J, Beguš S, Geršak G, Drnovšek J (2014) Non-contact heart rate and heart rate variability measurements: a review. Biomed Signal Process Control 13:102–112

    Google Scholar 

  34. Werner P, Al-Hamadi A, Walter S, Gruss S, Traue HC (2014) Automatic heart rate estimation from painful faces. In: International Conference on Image Processing (ICIP)

    Google Scholar 

  35. Hernandez J, McDuff D, Picard R (2015) Biowatch: estimation of heart and breathing rates from wrist motions. https://doi.org/10.4108/icst.pervasivehealth.2015.259064

  36. Limbrecht-Ecklundt K, Werner P, Traue HC, Al-Hamadi A, Walter S (2016) Mimische Aktivität differenzierter Schmerzintensitäten: Korrelation der Merkmale von Facial Action Coding System und Elektromyographie. Schmerz 30(3):248–256

    CAS  PubMed  Google Scholar 

  37. Ashraf AB, Lucey S, Cohn JF, Chen T, Ambadar Z, Prkachin KM et al (2009) The painful face - pain expression recognition using active appearance models. Image Vis Comput 27(12):1788–1796

    PubMed  PubMed Central  Google Scholar 

  38. Niese R, Al-Hamadi A, Panning A, Brammen D, Ebmeyer U, Michaelis B (2009) Towards pain recognition in post-operative phases using 3D-based features from video and support vector machines. Int J Digit Content Technol Its Appl 3(4):21–33

    Google Scholar 

  39. Lucey P, Cohn JF, Prkachin KM, Solomon PE, Chew S, Matthews I (2012) Painful monitoring: automatic pain monitoring using the UNBC-mcmaster shoulder pain expression archive database. Image Vis Comput 30(3):197–205

    Google Scholar 

  40. Kaltwang S, Rudovic O, Pantic M (2012) Continuous pain intensity estimation from facial expressions. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Lecture Notes in Computer Science. Springer, Berlin Heidelberg, S 368–377 (7432 LNCS)

    Google Scholar 

  41. Werner P, Al-Hamadi A, Niese R, Walter S, Gruss S, Traue HC (2013) Towards pain monitoring: Facial expression, head pose, a new database, an automatic system and remaining challenges. In: Proc British machine vision conf. BMVA Press, Essex, S 111–119

    Google Scholar 

  42. Bartlett MS, Littlewort GC, Frank MG, Lee K (2014) Automatic decoding of facial movements reveals deceptive pain expressions. Curr Biol 24(7):738–743

    CAS  PubMed  PubMed Central  Google Scholar 

  43. Sikka K, Ahmed AA, Diaz D, Goodwin MS, Craig KD, Bartlett MS et al (2015) Automated assessment of children’s postoperative pain using computer vision. Pediatrics 136(1):e124–e131

    PubMed  PubMed Central  Google Scholar 

  44. Zhou J, Hong X, Su F, Zhao G (2016) Recurrent convolutional neural network regression for continuous pain intensity estimation in video. In: Computer vision and pattern recognition workshops (CVPRW), S 1535–1543

    Google Scholar 

  45. Wang F, Xiang X, Liu C, Tran TD, Reiter A, Hager GD et al (2017) Regularizing face verification nets for pain intensity regression. In: International Conference on Image Processing (ICIP). IEEE, Washington DC, S 1087–1091

    Google Scholar 

  46. Werner P, Al-Hamadi A, Limbrecht-Ecklundt K, Walter S, Gruss S, Traue HC (2017) Automatic pain assessment with facial activity descriptors. IEEE Trans Affect Comput 8(3):286–299

    Google Scholar 

  47. Walter S, Gruss S, Limbrecht-Ecklundt K, Traue HC, Werner P, Al-Hamadi A et al (2014) Automatic pain quantification using autonomic parameters. Psychol Neurosci 7(3):363

    Google Scholar 

  48. Gruss S, Treister R, Werner P, Traue HC, Crawcour S, Andrade A et al (2015) Pain intensity recognition rates via biopotential feature patterns with support vector machines. PLoS ONE 10(10):e140330

    PubMed  PubMed Central  Google Scholar 

  49. Lopez-Martinez D, Picard R (2018) Continuous pain intensity estimation from autonomic signals with recurrent neural networks. In: 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, Washington DC, S 5624–5627

    Google Scholar 

  50. Thiam P, Schwenker F (2017) Multi-modal data fusion for pain intensity assessment and classification. In: International Conference on Image Processing Theory, Tools and Applications (IPTA). IEEE, Wasshington DC, S 1–6

    Google Scholar 

  51. Tsai F‑S, Weng Y‑M, Ng C‑J, Lee C‑C (2017) Embedding stacked bottleneck vocal features in a LSTM architecture for automatic pain level classification during emergency triage. In: International Conference on Affective Computing and Intelligent Interaction (ACII). IEEE, Washington DC, S 313–318

    Google Scholar 

  52. Werner P, Lopez-Martinez D, Walter S, Al-Hamadi A, Gruss S, Picard R (2019) Automatic recognition methods supporting pain assessment: a survey. IEEE Trans Affect Comput. https://doi.org/10.1109/TAFFC.2019.2946774

    Article  Google Scholar 

  53. Lucey P, Cohn JF, Prkachin KM, Solomon PE, Matthews I (2011) Painful data: The UNBC-McMaster shoulder pain expression archive database. In: 2011 IEEE International Conference on Automatic Face and Gesture Recognition and Workshops, FG 2011, S 57–64

    Google Scholar 

  54. Walter S, Gruss S, Ehleiter H, Tan J, Traue HC, Crawcour S et al (2013) The biovid heat pain database: Data for the advancement and systematic validation of an automated pain recognition. In: IEEE International Conference on Cybernetics, CYBCONF

    Google Scholar 

  55. Velana M, Gruss S, Layher G, Thiam P, Zhang Y, Schork D et al (2016) The senseemotion database: a Multimodal database for the development and systematic validation of an automatic pain- and emotion-recognition system. In: IAPR workshop on multimodal pattern recognition of social signals in human-computer interaction. Springer, Cham, S 127–139

    Google Scholar 

  56. Werner P, Al-Hamadi A, Niese R, Walter S, Gruss S, Traue HC (2014) Automatic pain recognition from video and biomedical signals. In: Proceedings—International Conference on Pattern Recognition

    Google Scholar 

  57. Kächele M, Thiam P, Amirian M, Werner P, Walter S, Schwenker F et al (2015) Multimodal data fusion for person-independent, continuous estimation of pain intensity. In: Communications in computer and information science, S 275–285

    Google Scholar 

  58. Kächele M, Thiam P, Amirian M, Schwenker F, Palm G (2016) Methods for person-centered continuous pain intensity assessment from bio-physiological channels. IEEE J Sel Top Signal Process 10(5):854–864

    Google Scholar 

  59. Kächele M, Amirian M, Thiam P, Werner P, Walter S, Palm G et al (2017) Adaptive confidence learning for the personalization of pain intensity estimation systems. Evol Syst 8(1):71–83

    Google Scholar 

  60. Gruss S, Geiger M, Werner P, Wilhelm O, Traue HC, Al-Hamadi A et al (2019) Multi-modal signals for analyzing pain responses to thermal and electrical stimuli. J Vis Exp. https://doi.org/10.3791/59057

    Article  PubMed  Google Scholar 

  61. Hassan T, Seuß D, Wollenberg J, Weitz K, Kunz M, Garbas J et al (2019) Automatic detection of pain from facial expressions : a survey. IEEE Trans Softw Engeneer, im Druck

  62. Zhang X, Yin L, Cohn JF, Canavan S, Reale M, Horowitz A et al (2014) BP4D-Spontaneous: a high-resolution spontaneous 3D dynamic facial expression database. Image Vis Comput 32(10):692–706

    Google Scholar 

  63. Zhang Z, Girard JM, Wu Y, Zhang X, Liu P, Ciftci U et al (2016) Multimodal spontaneous emotion corpus for human behavior analysis. In: Proceedings of the IEEE conference on computer vision and pattern recognition, S 3438–3446

    Google Scholar 

  64. Haque MA, Bautista RB, Noroozi F, Kulkarni K, Laursen CB, Irani R et al (2018) Deep Multimodal pain recognition: a database and comparison of Spatio-temporal visual modalities. In: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), S 250–257

    Google Scholar 

  65. Lopez-Martinez D, Picard R (2017) Multi-task neural networks for personalized pain recognition from physiological signals. In: International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW). IEEE, Washington DC, S 181–184

    Google Scholar 

  66. Walter S, Gruss S, Traue H, Werner P, Al-Hamadi A et al (2015) Data fusion for automated pain recognition. In: International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth), S 261–264

    Google Scholar 

  67. Kächele M, Werner P, Al-Hamadi A, Palm G, Walter S, Schwenker F (2015) Bio-visual fusion for person-independent recognition of pain intensity. In: Multiple classifier systems (MCS), S 220–230

    Google Scholar 

  68. Thiam P, Kessler V, Walter S, Palm G, Schwenker F (2016) Audio-visual recognition of pain intensity. In: Multimodal pattern recognition of social signals in human-computer-interaction workshop, S 110–126

    Google Scholar 

  69. Thiam P, Kessler V, Amirian M, Bellmann P, Layher G, Zhang Y et al (2019) Multi-modal pain intensity recognition based on the senseemotion database. IEEE Trans Affect Comput. https://doi.org/10.1109/TAFFC.2019.2892090

  70. Kächele M, Amirian M, Thiam P, Werner P, Walter S, Palm G et al (2017) Adaptive confidence learning for the personalization of pain intensity estimation systems. Evol Syst 8(1):71–83

    Google Scholar 

  71. Amirian M, Kächele M, Schwenker F (2016) Using radial basis function neural networks for continuous and discrete pain estimation from bio-physiological signals. In: Artificial neural networks in pattern recognition, S 269–284

    Google Scholar 

  72. Kessler V, Thiam P, Amirian M, Schwenker F (2017) Multimodal fusion including camera photoplethysmography for pain recognition. Int Conf Companion Technol. https://doi.org/10.1109/COMPANION.2017.8287083

  73. Gruss S, Treister R, Werner P, Traue HC, Crawcour S, Andrade A et al (2015) Pain intensity recognition rates via biopotential feature patterns with support vector machines. PLoS ONE 10(10):e140330

    PubMed  PubMed Central  Google Scholar 

  74. Hua-Mei C, Varshney PK, Arora MK (2003) Performance of mutual information similarity measure for registration of multitemporal remote sensing images. Geosci Remote Sens IEEE Trans 41(11):2445–2454

    Google Scholar 

  75. PMD-200TM | Medasense Biometrics Ltd.

  76. Edry R, Recea V, Dikust Y, Sessler DI (2016) Preliminary intraoperative validation of the nociception level index. Anesthesiology 125:193–203

    PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Sektion Medizinische Psychologie, Klinik für Psychosomatische Medizin und Psychotherapie, Universitätsklinikum Ulm, Frauensteige 6, 89075, Ulm, Deutschland

    S. Frisch, H. C. Traue, S. Gruss &  S. Walter

  2. Praxis für Neurologie und Psychiatrie Leutkirch, Leutkirch, Deutschland

    S. Frisch

  3. Neuro-Informationstechnik, Institut für Informations- und Kommunikationstechnik, Otto-von-Guericke-Universität Magdeburg, Magdeburg, Deutschland

    P. Werner & A. Al-Hamadi

Authors
  1. S. Frisch
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. P. Werner
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. A. Al-Hamadi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. H. C. Traue
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. S. Gruss
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. S. Walter
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to S. Walter.

Ethics declarations

Interessenkonflikt

S. Frisch, P. Werner, A. Al-Hamadi, H.C. Traue, S. Gruss und S. Walter geben an, dass kein Interessenkonflikt besteht.

Für diesen Beitrag wurden von den Autoren keine Studien an Menschen oder Tieren durchgeführt. Für die aufgeführten Studien gelten die jeweils dort angegebenen ethischen Richtlinien.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Frisch, S., Werner, P., Al-Hamadi, A. et al. Von der Fremdbeurteilung des Schmerzes zur automatisierten multimodalen Messung der Schmerzintensität. Schmerz 34, 376–387 (2020). https://doi.org/10.1007/s00482-020-00473-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00482-020-00473-x

Schlüsselwörter

Keywords

Search

Navigation