Skip to main content

Advertisement

SpringerLink
Log in

Inter-rater reliability in performance status assessment among healthcare professionals: an updated systematic review and meta-analysis

  • Review Article
  • Published:
Supportive Care in Cancer Aims and scope Submit manuscript

Abstract

Introduction

Survival prediction for patients with incurable malignancies is invaluable information during end-of-life discussions, as it helps the healthcare team to appropriately recommend treatment options and consider hospice enrolment. Assessment of performance status may differ between different healthcare professionals (HCPs), which could have implications in predicting prognosis. The aim of this systematic review and meta-analysis is to update a prior systematic review with recent articles, as well as conduct a meta-analysis to quantitatively compare performance status scores.

Methods

A literature search was carried out in Ovid MEDLINE, Embase, and Cochrane Central Register of Controlled Trials, from the earliest date until the first week of August 2019. Studies were included if they reported on (1) Karnofsky Performance Status (KPS), Eastern Cooperative Oncology Group (ECOG) Performance Status, and/or Palliative Performance Scale (PPS) and (2) assessment of performance status by multiple HCPs for the same patient sets. The concordance statistics (Kappa, Krippendorff’s alpha, Kendall correlation, Spearman rank correlation, Pearson correlation) were extracted into a summary table for narrative review, and Pearson correlation coefficients were calculated for each study and meta-analyzed with a random effects analysis model. Analyses were conducted using Comprehensive Meta-Analysis (Version 3) by Biostat.

Results

Fourteen articles were included, with a cumulative sample size of 2808 patients. The Pearson correlation coefficient was 0.787 (95% CI: 0.661, 0.870) for KPS, 0.749 (95% CI: 0.716, 0.779) for PPS, and 0.705 (95% CI: 0.536, 0.819) for ECOG. Four studies compared different tools head-to-head; KPS was favored in three studies. The quality of evidence was moderate, as determined by the GRADE tool.

Conclusions

The meta-analysis’s Pearson correlation coefficient ranged from 0.705 to 0.787; there is notable correlation of performance status scores, with no one tool statistically superior to others. KPS is, however, descriptively better and favored in head-to-head trials. Future studies could now examine the accuracy of KPS assessment in prognostication and focus on model-building around KPS.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Chow R, Chiu N, Burera E et al (2016) Inter-rater reliability in performance status assessment among health care professionals: a systematic review. Ann Palliat Med 5:83–92

    Article  Google Scholar 

  2. Temel JS, Greer JA, Muzikansky A, Gallagher ER, Admane S, Jackson VA, Dahlin CM, Blinderman CD, Jacobsen J, Pirl WF, Billings JA, Lynch TJ (2010) Early palliative care for patients with metastatic non-small-cell lung cancer. New Engl J Med 363:733–742

    Article  CAS  Google Scholar 

  3. Wright AA, Zhang B, Ray A, Mack JW, Trice E, Balboni T, Mitchell SL, Jackson VA, Block SD, Maciejewski PK, Prigerson HG (2008) Associations between end-of-life discussions, patient mental health, medical care near death, and caregiver bereavement adjustment. JAMA 300:1665–1673

    Article  CAS  Google Scholar 

  4. Zhang B, Wright AA, Huskamp HA, Nilsson ME, Maciejewski ML, Earle CC, Block SD, Maciejewski PK, Prigerson HG (2009) Health care costs in the last week of life: associations with end-of-life conversations. Arch Intern Med 169:480–488

    Article  Google Scholar 

  5. Evans C, McCarthy M (1985) Prognostic uncertainty in terminal care: can the Karnofsky index help? Lancet 1:1204–1206

    Article  CAS  Google Scholar 

  6. Pirovano M, Maltoni M, Nanni O, Marinari M, Indelli M, Zaninetta G, Petrella V, Barni S, Zecca E, Scarpi E, Labianca R, Amadori D, Luporini G (1999) A new palliative prognostic score: a first step for the staging of terminally ill cancer patients. Italian multicenter and study group on palliative care. J Pain Symptom Manag 17:231–239

    Article  CAS  Google Scholar 

  7. Bachelot T, Ray-Coquard I, Catimel G et al (2000) Multivariable analysis of prognostic factors for toxicity and survival for patients enrolled in phase I clinical trials. Ann Oncol 11:115–116

    Article  Google Scholar 

  8. Maltoni M, Caraceni A, Brunelli C, Broeckaert B, Christakis N, Eychmueller S, Glare P, Nabal M, Viganò A, Larkin P, de Conno F, Hanks G, Kaasa S, Steering Committee of the European Association for Palliative Care (2005) Prognostic factors in advanced cancer patients: evidence-based clinical recommendations--a study by the steering Committee of the European Association for palliative care. J Clin Oncol 23:6240–6248

    Article  Google Scholar 

  9. Lingjun Z, Jing C, Jian L, Wee B, Jijun Z (2009) Prediction of survival time in advanced cancer: a prognostic scale for Chinese patients. J Pain Symptom Manag 38:578–586

    Article  Google Scholar 

  10. Chan EY, Wu HY, Chan YH (2013) Revisiting the palliative performance scale: changes in scores during disease trajectory predicts survival. Palliat Med 27:367–374

    Article  Google Scholar 

  11. Krishnan M, Temel JS, Wright AA, Bernacki R, Selvaggi K, Balboni T (2013) Predicting life expectancy in patients with advanced incurable cancer: a review. J Support Oncol 11:68–74

    Article  Google Scholar 

  12. Kelly CM, Shahrokni (2016) Moving beyond Karnofsky and ECOG performance status assessments with new technologies. J Oncol:6186543

  13. Fleiss JL, Levin B, Paik MC (eds) (2003) Statistical methods for rates and proportions, 3rd edn. John wiley & Sons, Inc., Hoboken

    Google Scholar 

  14. De Swert K (2002) Calculating inter-coder reliability in media content analysis using Krippendorff’s alpha. Available online: http://www.polcomm.org/wp-content/uploads/ICR01022012.pdf. Accessed 20 July 2015

  15. Nunally JC (ed) (1978) Psychometric theory. McGraw-Hill, New York

    Google Scholar 

  16. Landis JR, Koch GC (1977) The measurement of observer agreement for categorical data. Biometrics 33:159–174

    Article  CAS  Google Scholar 

  17. Daniela S, Jantschi L. Pearson versus spearman, Kendall’s tau correlation analysis on structure-activity relationships of biological active compounds. Available online: http://ljs.academicdirectorg/A09/179-200.htm.

  18. Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, Schünemann HJ, GRADE Working Group (2008) GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 336:924–926

    Article  Google Scholar 

  19. Hutchinson TA, Boyd NF, Feinstein AR (1979) Scientific problems in clinical scales, as demonstrated in the Karnofsky index of performance status. J Chronic Dis 32:661–666

    Article  CAS  Google Scholar 

  20. Schag CC, Heinrich RL, Ganz PA (1984) Karnofsky performance status revisited: reliability, validity and guidelines. J Clin Oncol 2:187–193

    Article  CAS  Google Scholar 

  21. Conill C, Verge E, Salamero M (1990) Performance status assessment in cancer patients. Cancer 65:1864–1866

    Article  CAS  Google Scholar 

  22. Roila F, Lupattelli M, Sassi M, Basurto C, Bracarda S, Picciafuoco M, Boschetti E, Milella G, Ballatori E, Tonato M (1991) Intra- and inter-observer variability in cancer patients’ performance status assessed according to Karnofsky and ECO scales. Ann Oncol 2:437–439

    Article  CAS  Google Scholar 

  23. Sorenson JB, Klee M, Palshof T et al (1993) Performance status assessment in cancer patients. An inter-observer variability study. Br J Cancer 159:1988–1992

    Google Scholar 

  24. Fantoni M, Izzi I, Del Borgo C et al (1999) Inter-rater reliability of a modified Karnofsky scale of performance status for HIV-infected individuals. AIDS Patient Care STDs 13:23–28

    Article  CAS  Google Scholar 

  25. Taylor AE, Olver IN, Sivanthan T, Chi M, Purnell C (1999) Observer error in grading performance status in cancer patients. Support Care Cancer 7:332–335

    Article  CAS  Google Scholar 

  26. Ando M, Ando Y, Hasegawa Y, Shimokata K, Minami H, Wakai K, Ohno Y, Sakai S (2001) Prognostic value of performance status assessed by patients themselves, nurses, and oncologists in advanced non-small cell lung cancer. Br J Cancer 85:1634–1639

    Article  CAS  Google Scholar 

  27. Liem BJ, Holland JM, Kang MY et al (2002) Karnofsky performance status assessment: resident versus attending. J Cancer Educ 17:138–141

    PubMed  Google Scholar 

  28. de Borja MT, Chow E, Bovett G et al (2004) The correlation among patients and health care professionals in assessing functional status using the Karnofsky and eastern cooperative oncology group performance status scales. Support Cancer Ther 2:59–63

    Article  Google Scholar 

  29. Campos S, Zhang L, Sinclair E, Tsao M, Barnes EA, Danjoux C, Sahgal A, Goh P, Culleton S, Mitera G, Chow E (2009) The palliative performance scale: examining its inter-rater reliability in an outpatient palliative radiation oncology clinic. Support Care Cancer 17:685–690

    Article  Google Scholar 

  30. Zimmermann C, Burman D, Bandukwala S, Seccareccia D, Kaya E, Bryson J, Rodin G, Lo C (2010) Nurse and physician inter-rater agreement of three performance status measures in palliative care outpatients. Support Care Cancer 18:609–616

    Article  Google Scholar 

  31. Kim YJ, Hui D, Zhang Y et al. Difference in performance status assessments between palliative care specialists, nurses and oncologists. J Pain Symptom Manag 2015; 49: 1050–1058.e2

  32. Neeman E, Gresham G, Ovasapians N et al (2019) Comparing physician and nurse eastern cooperative oncology group performance status (ECOG-PS) ratings as predictors of clinical outcomes in patients with Cancer. Oncologist 24:1–7

    Article  Google Scholar 

  33. May CH, Lester JF, Lee S (2012) PS score discordance and why it matters. Lung Cancer 75:S18

    Article  Google Scholar 

  34. Addy C, Sephton M, Suntharalingam J et al (2012) Assessment of performance status in long cancer: do oncologists and respiratory physicians agree? Lung Cancer 75:S17–S18

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Epidemiology and Biostatistics, Schulich School of Medicine & Dentistry, University of Western Ontario, London, Ontario, Canada

    Ronald Chow & James Im

  2. Division of Radiation Oncology, Department of Oncology, London Regional Cancer Program, University of Western Ontario, London, Ontario, Canada

    Ronald Chow & Michael Lock

  3. Department of Palliative, Rehabilitation and Integrative Medicine, Division of Cancer Medicine, The University of Texas MD Anderson Cancer Center, Houston, TX, USA

    Eduardo Bruera

  4. Division of Hematology and Oncology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA

    Jennifer S. Temel

  5. Department of Radiation Oncology, Dana-Farber Cancer Institute, Harvard Medical School, Boston, Harvard University, Boston, MA, USA

    Monica Krishnan

Authors
  1. Ronald Chow
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eduardo Bruera
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jennifer S. Temel
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Monica Krishnan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. James Im
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Michael Lock
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ronald Chow.

Ethics declarations

Conflict of interest

None.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix 1

Appendix 1

Search Strategy

Database: Ovid MEDLINE(R) and Epub Ahead of Print, In-Process & Other Non-Indexed.

Citations, Daily and Versions(R) <1946 to August 05, 2019 > Search Strategy:

1 exp. Karnofsky Performance Status/ or Karnofsky performance status.mp. (4669).

2 Eastern Cooperative Oncology Group Performance Status.mp. (2204).

3 (KPS or ECOGPS or ECOG or ECOG or PPS).mp. (16049).

4 ((Karnofsky or Eastern Cooperative Oncology Group or palliative) adj4 (scale or status or score)).mp. (11320).

5 (performance status or performance scale or performance score).mp. (27973).

6 or/1–5 (40324).

7 ((evaluat* or assess* or compar* or choose or choice or select or pick or prefer* or inter-rater or interrater or rate or rating or difference*) adj5 (“performance status” or “performance score” or “performance scale” or KPS or Karnofsky or ECOG* or prognostic tool* or prognostic.

instrument*)).mp. (3265).

8 (physician* or doctor* or nurse* or oncologist* or research assistant* or clinician* or practitioner* or specialist*).mp. (1250590).

9 ((physician* or doctor* or nurse* or oncologist* or research assistant* or clinician* or practitioner* or specialist*) adj5 (“performance status” or KPS or Karnofsky or ECOG* or prognostic tool* or prognostic instrument*)).mp. (218).

10 6 and (9 or (7 and 8)) (357).

11 limit 10 to English language (346).

Database: Embase Classic+Embase <1947 to 2019 Week 31 > Search Strategy:

1 exp. Karnofsky Performance Status/ or Karnofsky performance status.mp. (11358).

2 Eastern Cooperative Oncology Group Performance Status.mp. (3254).

3 (KPS or ECOGPS or ECOG or ECOG or PPS).mp. (40720).

4 ((Karnofsky or Eastern Cooperative Oncology Group or palliative) adj4 (scale or status or.

score)).mp. (19654).

5 (performance status or performance scale or performance score).mp. (54278).

6 or/1–5 (82709).

7 ((evaluat* or assess* or compar* or choose or choice or select or pick or prefer* or inter-.

rater or interrater or rate or rating or difference*) adj5 (“performance status” or “performance score” or “performance scale” or KPS or Karnofsky or ECOG* or prognostic tool* or prognostic.

instrument*)).mp. (6119).

8 (physician* or doctor* or nurse* or oncologist* or research assistant* or clinician* orpractitioner* or specialist*).mp. (1768578).

9 ((physician* or doctor* or nurse* or oncologist* or research assistant* or clinician* orpractitioner* or specialist*) adj5 (“performance status” or KPS or Karnofsky or ECOG* or.

prognostic tool* or prognostic instrument*)).mp. (450).

10 6 and (9 or (7 and 8)) (845).

11 limit 10 to english language (832).

Database: EBM Reviews - Cochrane Central Register of Controlled Trials <June 2019 > Search.

Strategy:

1 exp. Karnofsky Performance Status/ or Karnofsky performance status.mp. (1445).

2 Eastern Cooperative Oncology Group Performance Status.mp. (1102).

3 (KPS or ECOGPS or ECOG or ECOG or PPS).mp. (12098).

4 ((Karnofsky or Eastern Cooperative Oncology Group or palliative) adj4 (scale or status or.

score)).mp. (4804).

5 (performance status or performance scale or performance score).mp. (13980).

6 or/1–5 (20035).

7 ((evaluat* or assess* or compar* or choose or choice or select or pick or prefer* or inter-.

rater or interrater or rate or rating or difference*) adj5 (“performance status” or “performance.

score“ or “performance scale” or KPS or Karnofsky or ECOG* or prognostic tool* or prognostic.

instrument*)).mp. (1526) (physician* or doctor* or nurse* or oncologist* or research assistant* or clinician* or practitioner* or specialist*).mp. (97075)((physician* or doctor* or nurse* or oncologist* or research assistant* or clinician* or practitioner* or specialist*) adj5 (“performance status” or KPS or Karnofsky or ECOG* or prognostic tool* or prognostic instrument*)).mp. (59) 6 and (9 or (7 and 8)) (156)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chow, R., Bruera, E., Temel, J.S. et al. Inter-rater reliability in performance status assessment among healthcare professionals: an updated systematic review and meta-analysis. Support Care Cancer 28, 2071–2078 (2020). https://doi.org/10.1007/s00520-019-05261-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00520-019-05261-7

Keywords

Search

Navigation