Skip to main content
SpringerLink
Log in

Fairness in human judgement in assessment: a hermeneutic literature review and conceptual framework

  • Review
  • Published:
Advances in Health Sciences Education Aims and scope Submit manuscript

Abstract

Human judgement is widely used in workplace-based assessment despite criticism that it does not meet standards of objectivity. There is an ongoing push within the literature to better embrace subjective human judgement in assessment not as a ‘problem’ to be corrected psychometrically but as legitimate perceptions of performance. Taking a step back and changing perspectives to focus on the fundamental underlying value of fairness in assessment may help re-set the traditional objective approach and provide a more relevant way to determine the appropriateness of subjective human judgements. Changing focus to look at what is ‘fair’ human judgement in assessment, rather than what is ‘objective’ human judgement in assessment allows for the embracing of many different perspectives, and the legitimising of human judgement in assessment. However, this requires addressing the question: what makes human judgements fair in health professions assessment? This is not a straightforward question with a single unambiguously ‘correct’ answer. In this hermeneutic literature review we aimed to produce a scholarly knowledge synthesis and understanding of the factors, definitions and key questions associated with fairness in human judgement in assessment and a resulting conceptual framework, with a view to informing ongoing further research. The complex construct of fair human judgement could be conceptualised through values (credibility, fitness for purpose, transparency and defensibility) which are upheld at an individual level by characteristics of fair human judgement (narrative, boundaries, expertise, agility and evidence) and at a systems level by procedures (procedural fairness, documentation, multiple opportunities, multiple assessors, validity evidence) which help translate fairness in human judgement from concepts into practical components.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  • American Educational Research Association, American Psychological Association, National Council on Measurement in Education & Joint Committee on Standards for Educational Psychological Testing. (1999). Standards for educational and psychological testing. Washington: American Educational Research Association.

    Google Scholar 

  • Bacon, R., Holmes, K., & Palermo, C. (2017). Exploring subjectivity in competency-based assessment judgements of assessors. Nutrition & Dietetics, 74(4), 357–364.

    Article  Google Scholar 

  • Bacon, R., Williams, L., Grealish, L., & Jamieson, M. (2015). Credible and defensible assessment of entry-level clinical competence: Insights from a modified Delphi study. Focus on Health Professional Education: A Multi-Disciplinary Journal, 16(3), 57.

    Article  Google Scholar 

  • Beckett, D. (2008). Holistic competence: Putting judgements first. Asia Pacific Education Review, 9(1), 21–30.

    Article  Google Scholar 

  • Berendonk, C., Stalmeijer, R. E., & Schuwirth, L. W. (2013). Expertise in performance assessment: Assessors’ perspectives. Advances Health Sciences Education: Theory and Practice, 18(4), 559–571.

    Article  Google Scholar 

  • Boell, S., & Cecez-Kecmanovic, D. (2010). Literature reviews and the hermeneutic circle. Australian Academic & Research Libraries, 41(2), 129–144.

    Article  Google Scholar 

  • Boell, S. K., & Cecez-Kecmanovic, D. (2014). A hermeneutic approach for conducting literature reviews and literature searches. Communications for the Association of Information Systems, 34, 12.

    Article  Google Scholar 

  • Boud, D. (1990). Assessment and the promotion of academic values. Studies in Higher Education, 15(1), 101–111.

    Article  Google Scholar 

  • Boulet, J. R., & Durning, S. J. (2019). What we measure… and what we should measure in medical education. Medical Education, 53(1), 86–94.

    Article  Google Scholar 

  • Boursicot, K. (2020). Consensus statement reports: Performance assessment. Paper presented at Ottawa 2020, Kuala Lumpar, Malayasia.

  • Bullock, J. L., Lai, C. J., Lockspeiser, T., O’Sullivan, P. S., Aronowitz, P., et al. (2019). In pursuit of Honors: A multi-institutional study of students’ perceptions of clerkship evaluation and grading. Academic Medicine, 94(11S), S48–S56.

    Article  Google Scholar 

  • Burgess, A., Roberts, C., Clark, T., & Mossman, K. (2014). The social validity of a national assessment centre for selection into general practice training. BMC Medical Education, 14(1), 261.

    Article  Google Scholar 

  • Chory, R. M. (2007). Enhancing student perceptions of fairness: The relationship between instructor credibility and classroom justice. Communication Education, 56(1), 89–105.

    Article  Google Scholar 

  • Cleland, J. A., Knight, L. V., Rees, C. E., Tracey, S., & Bond, C. M. (2008). Is it me or is it them? Factors that influence the passing of underperforming students. Medical Dducation, 42(8), 800–809.

    Google Scholar 

  • Cohen, G. S., Blumberg, P., Ryan, N. C., & Sullivan, P. L. (1993). Do final grades reflect written qualitative evaluations of student performance? Teaching and Learning in Medicine: An International Journal, 5(1), 10–15.

    Article  Google Scholar 

  • Colbert, C. Y., Dannefer, E. F., & French, J. C. (2015). Clinical competency committees and assessment: Changing the conversation in graduate medical education. Journal of Graduate Medical Education, 7(2), 162–165.

    Article  Google Scholar 

  • Colbert, C. Y., French, J. C., Herring, M. E., & Dannefer, E. F. (2017). Fairness: The hidden challenge for competency-based postgraduate medical education programs. Perspectives on Medical Education, 6(5), 347–355.

    Article  Google Scholar 

  • Crossley, J., & Jolly, B. (2012). Making sense of work-based assessment: Ask the right questions, in the right way, about the right things, of the right people. Medical Education, 46(1), 28–37.

    Article  Google Scholar 

  • Daniels, N., & Sabin, J. (1997). Limits to health care: Fair procedures, democratic deliberation, and the legitimacy problem for insurers. Philosophy & Public Affairs, 26(4), 303–350.

    Article  Google Scholar 

  • Dauphinee, W. D. (1995). Assessing clinical performance: Where do we stand and what might we expect? Journal of the American Medical Association, 274(9), 741–743.

    Article  Google Scholar 

  • Dijksterhuis, M. G. K., Voorhuis, M., Teunissen, P. W., Schuwirth, L. W. T., ten Cate, O. T. J., et al. (2009). Assessment of competence and progressive independence in postgraduate clinical training. Medical Education, 43(12), 1156–1165.

    Article  Google Scholar 

  • Downie, R., & Macnaughton, J. (2009). In defence of professional judgement. Advances in Psychiatric Treatment, 15(5), 322–327.

    Article  Google Scholar 

  • Duffield, K., & Spencer, J. (2002). A survey of medical students’ views about the purposes and fairness of assessment. Medical Education, 36(9), 879–886.

    Article  Google Scholar 

  • Durning, S. J., Hanson, J., Gilliland, W., McManigle, J. M., Waechter, D., et al. (2010). Using qualitative data from a program director’s evaluation form as an outcome measurement for medical school. Military Medicine, 175(6), 448–452.

    Article  Google Scholar 

  • Epstein, R. M. (2013). Whole mind and shared mind in clinical decision-making. Patient Education and Counselling, 90(2), 200–206.

    Article  Google Scholar 

  • Eva, K. W. (2008). On the limits of systematicity. Medical Education, 42(9), 852–853.

    Article  Google Scholar 

  • Eva, K. W. (2015). Moving beyond childish notions of fair and equitable. Medical Education, 49(1), 1–3.

    Article  Google Scholar 

  • Flin, R., Youngson, G., & Yule, S. (2007). How do surgeons make intraoperative decisions? Quality & Safety in Health Care, 16(3), 235–239.

    Article  Google Scholar 

  • Gingerich, A., Kogan, J., Yeates, P., Govaerts, M., & Holmboe, E. (2014). Seeing the ‘black box’ differently: Assessor cognition from three research perspectives. Medical Education, 48(11), 1055–1068.

    Article  Google Scholar 

  • Ginsburg, S., Eva, K., & Regehr, G. (2013). Do in-training evaluation reports deserve their bad reputations? A study of the reliability and predictive ability of ITER scores and narrative comments. Academic Medicine, 88(10), 1539–1544.

    Article  Google Scholar 

  • Ginsburg, S., Regehr, G., Lingard, L., & Eva, K. W. (2015). Reading between the lines: Faculty interpretations of narrative evaluation comments. Medical Education, 49(3), 296–306.

    Article  Google Scholar 

  • Ginsburg, S., van der Vleuten, C. P. M., & Eva, K. W. (2017a). The hidden value of narrative comments for assessment: A quantitative reliability analysis of qualitative data. Academic Medicine, 92(11), 1617–1621.

    Article  Google Scholar 

  • Ginsburg, S., van der Vleuten, C., Eva, K. W., & Lingard, L. (2016). Hedging to save face: A linguistic analysis of written comments on in-training evaluation reports. Advances in Health Science Education : Theory and Practice, 21(1), 175–188.

    Article  Google Scholar 

  • Ginsburg, S., van der Vleuten, C. P., Eva, K. W., & Lingard, L. (2017b). Cracking the code: Residents’ interpretations of written assessment comments. Medical Education, 51(4), 401–410.

    Article  Google Scholar 

  • Gipps, C., & Stobart, G. (2009). Fairness in assessment. In Educational assessment in the 21st century (pp. 105–118). Springer.

  • Govaerts, M. J., Schuwirth, L. W., Van der Vleuten, C. P., & Muijtjens, A. M. (2011). Workplace-based assessment: Effects of rater expertise. Advances in Health Science Education: Theory and Practice, 16(2), 151–165.

    Article  Google Scholar 

  • Govaerts, M. J., Van de Wiel, M. W., Schuwirth, L. W., Van der Vleuten, C. P., & Muijtjens, A. M. (2013). Workplace-based assessment: Raters’ performance theories and constructs. Advances in Health Science Education: Theory and Practice, 18(3), 375–396.

    Article  Google Scholar 

  • Govaerts, M., & van der Vleuten, C. P. (2013). Validity in work-based assessment: Expanding our horizons. Medical Education, 47(12), 1164–1174.

    Article  Google Scholar 

  • Govaerts, M. J. B., van der Vleuten, C. P. M., & Holmboe, E. S. (2019). Managing tensions in assessment: Moving beyond either-or thinking. Medical Education, 53(1), 64–75.

    Article  Google Scholar 

  • Govaerts, M. J., van der Vleuten, C. P., Schuwirth, L. W., & Muijtjens, A. M. (2007). Broadening perspectives on clinical performance assessment: Rethinking the nature of in-training assessment. Advances in Health Science Education: Theory and Practice, 12(2), 239–260.

    Article  Google Scholar 

  • Greenhaigh, T., & Hurwitz, B. (1999). Why study narrative? Western Journal of Medicine, 170(6), 367–369.

    Google Scholar 

  • Greenhalgh, T., Howick, J., & Maskrey, N. (2014). Evidence based medicine: A movement in crisis? British Medical Journal, 348, g3725.

    Article  Google Scholar 

  • Greenhalgh, T., & Hurwitz, B. (1999). Narrative based medicine: Why study narrative? British Medical Journal, 318(7175), 48–50.

    Article  Google Scholar 

  • Greenhalgh, T., & Papoutsi, C. (2018). Studying complexity in health services research: Desperately seeking an overdue paradigm shift. BMC Medicine, 16(1), 95.

    Article  Google Scholar 

  • Greenhalgh, T., & Shaw, S. (2017). Understanding heart failure; explaining telehealth—A hermeneutic systematic review. BMC Cardiovascular Disorders, 17(1), 156.

    Article  Google Scholar 

  • Groarke, L. (2019). ‘Informal logic’ Summer 2019. In E. N. Zalta (Ed.), The Standford encyclopedia of philosophy.

  • Ham, C. (1999). Tragic choices in health care: Lessons from the child B case. British Medical Journal, 319(7219), 1258–1261.

    Article  Google Scholar 

  • Harden, R. M., Lilley, P., & Patricio, M. (2015). The Definitive Guide to the OSCE: The Objective Structured Clinical Examination as a performance assessment. Philadelphia: Elsevier Health Sciences.

    Google Scholar 

  • Hauer, K. E., Cate, O. T., Boscardin, C. K., Iobst, W., Holmboe, E. S., et al. (2016). Ensuring resident competence: A narrative review of the literature on group decision making to inform the work of clinical competency committees. Journal of Graduate Medical Education, 8(2), 156–164.

    Article  Google Scholar 

  • Hauer, K. E., Chesluk, B., Iobst, W., Holmboe, E., Baron, R. B., et al. (2015). Reviewing residents’ competence: A qualitative study of the role of clinical competency committees in performance assessment. Academic Medicine, 90(8), 1084–1092.

    Article  Google Scholar 

  • Hays, R. B., Hamlin, G., & Crane, L. (2015). Twelve tips for increasing the defensibility of assessment decisions. Medical Teacher, 37(5), 433–436.

    Article  Google Scholar 

  • Heifetz, R. A., Heifetz, R., Grashow, A., & Linsky, M. (2009). The practice of adaptive leadership: Tools and tactics for changing your organization and the world. Boston: Harvard Business Press.

    Google Scholar 

  • Hilligoss, B., & Young Rich, S. (2008). Developing a unifying framework of credibility assessment: Construct, heuristics and interaction in context. Information Processing and Management, 44, 1467–1484.

    Article  Google Scholar 

  • Hodges, B. (2013). Assessment in the post-psychometric era: Learning to love the subjective and collective. Medical Teacher, 35(7), 564–568.

    Article  Google Scholar 

  • Hodges, B. D., Ginsburg, S., Cruess, R., Cruess, S., Delport, R., et al. (2011). Assessment of professionalism: Recommendations from the Ottawa 2010 Conference. Medical Teacher, 33(5), 354–363.

    Article  Google Scholar 

  • Houston, D. (2002). Quality and the University: Stakeholders, boundary judgements and systems. In Change management: Proceedings of the 7th International Conference on ISO9000 and TQM Melbourne, RMIT University.

  • Hunter, K. (1996). “Don’t think zebras”: Uncertainty, interpretation, and the place of paradox in clinical education (journal article). Theoretical Medicine, 17(3), 225–241.

    Article  Google Scholar 

  • Jones, A. (1999). The place of judgement in competency-based assessment. Journal of Vocational Education and Training, 51(1), 145–160.

    Article  Google Scholar 

  • Kaldjian, L. C. (2010). Teaching practical wisdom in medicine through clinical judgement, goals of care, and ethical reasoning. Journal of Medical Ethics, 36(9), 558–562.

    Article  Google Scholar 

  • Katerndahl, D., Parchman, M., & Wood, R. (2010). Trends in the perceived complexity of primary health care: A secondary analysis. Journal of Evaluation in Clinical Practice, 16(5), 1002–1008.

    Article  Google Scholar 

  • Kirkland, A. (2012). The legitimacy of vaccine critics: What is left after the autism hypothesis? Journal of Health Politics, Policy and Law, 37(1), 69–97.

    Article  Google Scholar 

  • Kogan, J. R., Conforti, L. N., Iobst, W. F., & Holmboe, E. S. (2014). Reconceptualizing variable rater assessments as both an educational and clinical care problem. Academic Medicine, 89(5), 721–727.

    Article  Google Scholar 

  • Krefting, L. (1991). Rigor in qualitative research: The assessment of trustworthiness. American Journal of Occupational Therapy, 45(3), 214–222.

    Article  Google Scholar 

  • Kusnanto, H., Agustian, D., & Hilmanto, D. (2018). Biopsychosocial model of illnesses in primary care: A hermeneutic literature review (Review Article). Journal of Family Medicine and Primary Care, 7(3), 497–500.

    Article  Google Scholar 

  • Lind, E., & Tyler, T. (1988) Critical issues in social justice. In The social psychology of procedural justice. New York: Plenum Press.

  • Lind, E. A., & Van den Bos, K. (2002). When fairness works: Toward a general theory of uncertainty management. Research in Organizational Behavior, 24, 181–223.

    Article  Google Scholar 

  • Lipshitz, H. D., Klein, G., Orasanu, J., & Salas, E. (2001). Taking stock of naturalistic decision making. Journal of Behavioral Decision Making, 14(5), 331–352.

    Article  Google Scholar 

  • Lucey, C., & Souba, W. (2010). Perspective: The problem with the problem of professionalism. Academic Medicine, 85(6), 1018–1024.

    Article  Google Scholar 

  • MacRae, R. G., MacRae, H., Reznick, R. K., et al. (1998). Comparing the psychometric properties of checklists and global rating scales for assessing performance on an OSCE-format examination. Academic Medicine, 73, 993.

    Article  Google Scholar 

  • Marewski, J. N., Gaissmaier, W., & Gigerenzer, G. (2010). Good judgments do not require complex cognition. Cognitive Processing, 11(2), 103–121.

    Article  Google Scholar 

  • McCready, T. (2007). Portfolios and the assessment of competence in nursing: A literature review. International Journal of Nursing Studies, 44(1), 143–151.

    Article  Google Scholar 

  • Moore, T. (2011). Wicked problems, rotten outcomes and clumsy solutions. Children and families in a changing world. In NIFTeY/CCCH Conference 2011. Children’s place on the agenda… past, present and future, pp. 28–29.

  • Patterson, F., Zibarras, L., Carr, V., Irish, B., & Gregory, S. (2011). Evaluating candidate reactions to selection practices using organisational justice theory. Medical Education, 45(3), 289–297.

    Article  Google Scholar 

  • Plsek, P. E., & Greenhalgh, T. (2001). Complexity science: The challenge of complexity in health care. British Medical Journal, 323(7313), 625–628.

    Article  Google Scholar 

  • Ramani, S., Post, S. E., Konings, K., Mann, K., Katz, J. T., et al. (2017). “It’s just not the culture”: A qualitative study exploring residents’ perceptions of the impact of institutional culture on feedback. Teaching and Learning in Medicine, 29(2), 153–161.

    Article  Google Scholar 

  • Rees, C., & Shepherd, M. (2005). The acceptability of 360-degree judgements as a method of assessing undergraduate medical students’ personal and professional behaviours. Medical Education, 39(1), 49–57.

    Article  Google Scholar 

  • Reid, T. (1850). Essays on the intellectual powers of man. Cambridge: J. Bartlett.

    Google Scholar 

  • Rieh, S. Y., & Hilligoss, B. (2008). College students’ credibility judgments in the information-seeking process. Digital media, youth, and credibility (pp. 49–72). Cambridge, MA: The MIT Press.

    Google Scholar 

  • Robinson, J. M. (2002). In search of fairness: An application of multi-reviewer anonymous peer review in a large class. Journal of Further and Higher Education, 26(2), 183–192.

    Article  Google Scholar 

  • Rodabaugh, R. C. (1996). Institutional commitment to fairness in college teaching. New Directions for teaching and learning, 1996(66), 37–45.

    Article  Google Scholar 

  • Rotthoff, T. (2018). Standing up for subjectivity in the assessment of competencies. GMS Journal for Medical Education, 35(3), Doc29.

    Google Scholar 

  • Sadler, D. R. (2009). Indeterminacy in the use of preset criteria for assessment and grading. Assessment & Evaluation in Higher Education, 34(2), 159–179.

    Article  Google Scholar 

  • Schuwirth, L., Southgate, L., Page, G., Paget, N., Lescop, J., et al. (2002). When enough is enough: A conceptual basis for fair and defensible practice performance assessment. Medical Education, 36(10), 925–930.

    Article  Google Scholar 

  • Schuwirth, L. W., & van der Vleuten, C. P. (2006). A plea for new psychometric models in educational assessment. Medical Education, 40(4), 296–300.

    Article  Google Scholar 

  • Southgate, L., Cox, J., David, T., Hatch, D., Howes, A., et al. (2001). The General Medical Council’s Performance Procedures: Peer review of performance in the workplace. Med Education, 35(Suppl 1), 9–19.

    Google Scholar 

  • Ståhl, C., Seing, I., Gerdle, B., & Sandqvist, J. (2019). Fair or square? Experiences of introducing a new method for assessing general work ability in a sickness insurance context. Disability and Rehabilitation, 41(6), 656–665.

    Article  Google Scholar 

  • Stefan, S. (1993). What constitutes departure from professional judgment? Mental and Physical Disability Law Reporter, 17(2), 207–213.

    Google Scholar 

  • Stobart, G. (2005). Fairness in multicultural assessment systems. Assessment in Education: Principles, Policy & Practice, 12(3), 275–287.

    Google Scholar 

  • Tavares, W., & Eva, K. W. (2013). Exploring the impact of mental workload on rater-based assessments. Advances in Health Science Education: Theory and Practice, 18(2), 291–303.

    Article  Google Scholar 

  • Telio, S., Regehr, G., & Ajjawi, R. (2016). Feedback and the educational alliance: Examining credibility judgements and their consequences. Medical Education, 50(9), 933–942.

    Article  Google Scholar 

  • ten Cate, O. (2017). Competency-based postgraduate medical education: Past, present and future. GMS Journal for Medical Education, 34(5), Doc69.

    Google Scholar 

  • ten Cate, O., & Billett, S. (2014). Competency-based medical education: Origins, perspectives and potentialities. Medical Education, 48(3), 325–332.

    Article  Google Scholar 

  • ten Cate, O., & Regehr, G. (2019). The power of subjectivity in the assessment of medical trainees. Academic Medicine, 94(3), 333–337.

    Article  Google Scholar 

  • ten Cate, O., & Scheele, F. (2007). Competency-based postgraduate training: Can we bridge the gap between theory and clinical practice? Acadamic Medicine, 82(6), 542–547.

    Article  Google Scholar 

  • Tierney, R. D. (2012). Fairness in classroom assessment. Thousand Oaks: Sage.

    Google Scholar 

  • Tochel, C., Haig, A., Hesketh, A., Cadzow, A., Beggs, K., et al. (2009). The effectiveness of portfolios for post-graduate assessment and education: BEME Guide No 12. Medical Teacher, 31(4), 299–318.

    Article  Google Scholar 

  • Upshur, R. E., & Colak, E. (2003). Argumentation and evidence. Theoretical Medicine and Bioethics, 24(4), 283–299.

    Article  Google Scholar 

  • Valentine, N., & Schuwirth, L. (2019). Identifying the narrative used by educators in articulating judgement of performance. Perspectives Medical Education, 8(2), 83–89.

    Article  Google Scholar 

  • Van den Bos, K., Lind, E. A., Vermunt, R., & Wilke, H. A. (1997). How do I judge my outcome when I do not know the outcome of others? The psychology of the fair process effect. Journal of Personality and Social Psychology, 72(5), 1034.

    Article  Google Scholar 

  • Van den Bos, K., & Miedema, J. (2000). Toward understanding why fairness matters: The influence of mortality salience on reactions to procedural fairness. Journal of Personality and Social Psychology, 79(3), 355.

    Article  Google Scholar 

  • Van den Bos, K., Wilke, H. A., & Lind, E. A. (1998). When do we need procedural fairness? The role of trust in authority. Journal of Personality and Social Psychology, 75(6), 1449.

    Article  Google Scholar 

  • van der Vleuten, C. P., Norman, G. R., & De Graaff, E. (1991). Pitfalls in the pursuit of objectivity: Issues of reliability. Medical Education, 25(2), 110–118.

    Article  Google Scholar 

  • van der Vleuten, C. P., & Schuwirth, L. W. (2005). Assessing professional competence: From methods to programmes. Medical Education, 39(3), 309–317.

    Article  Google Scholar 

  • van Der Vleuten, C. P. M., Schuwirth, L. W. T., Driessen, E. W., Govaerts, M. J. B., & Heeneman, S. (2015). Twelve tips for programmatic assessment. Medical Teacher, 37(7), 641–646.

    Article  Google Scholar 

  • Viney, R., Rich, A., Needleman, S., Griffin, A., & Woolf, K. (2017). The validity of the Annual Review of Competence Progression: A qualitative interview study of the perceptions of junior doctors and their trainers. Journal of the Royal Society of Medicine, 110(3), 110–117.

    Article  Google Scholar 

  • Watling, C. J. (2014). Unfulfilled promise, untapped potential: Feedback at the crossroads. Medical Teacher, 36(8), 692–697.

    Article  Google Scholar 

  • Watling, C., Driessen, E., van der Vleuten, C. P., & Lingard, L. (2012). Learning from clinical work: The roles of learning cues and credibility judgements. Medical Education, 46(2), 192–200.

    Article  Google Scholar 

  • Watling, C., Driessen, E., van der Vleuten, C. P., Vanstone, M., & Lingard, L. (2013a). Beyond individualism: Professional culture and its influence on feedback. Medical Education, 47(6), 585–594.

    Article  Google Scholar 

  • Watling, C., Driessen, E., van der Vleuten, C. P. M., Vanstone, M., & Lingard, L. (2013b). Music lessons: Revealing medicine’s learning culture through a comparison with that of music. Medical Education, 47(8), 842–850.

    Article  Google Scholar 

  • Watling, C. J., & Ginsburg, S. (2019). Assessment, feedback and the alchemy of learning. Medical Education, 53(1), 76–85. https://doi.org/10.1111/medu.13645.

    Article  Google Scholar 

  • Watling, C. J., Kenyon, C. F., Zibrowski, E. M., Schulz, V., Goldszmidt, M. A., et al. (2008). Rules of engagement: Residents’ perceptions of the in-training evaluation process. Academic Medicine, 83(10 Suppl), S97–S100.

    Article  Google Scholar 

  • Webb, C., Endacott, R., Gray, M. A., Jasper, M. A., McMullan, M., et al. (2003). Evaluating portfolio assessment systems: What are the appropriate criteria? Nurse Education Today, 23(8), 600–609.

    Article  Google Scholar 

  • Weller, J. M., Misur, M., Nicolson, S., Morris, J., Ure, S., et al. (2014). Can I leave the theatre? A key to more reliable workplace-based assessment. British Journal of Anaesthesia, 112(6), 1083–1091.

    Article  Google Scholar 

  • Whitty, C. J. (2015). What makes an academic paper useful for health policy? BioMed Central, 13, 301.

    Google Scholar 

  • Wolf, M. M. (1978). Social validity: The case for subjective measurement or how applied behavior analysis is finding its heart 1. Journal of Applied Behavior Analysis, 11(2), 203–214.

    Article  Google Scholar 

  • Wycliffe-Jones, K., Hecker, K. G., Schipper, S., Topps, M., Robinson, J., et al. (2018). Selection for family medicine residency training in Canada: How consistently are the same students ranked by different programs? Canadian Family Physician, 64(2), 129–134.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Prideaux Health Professions Education, Flinders University, Bedford Park 5042, SA, Australia

    Nyoli Valentine, Ernst Michael Shanahan & Lambert Schuwirth

  2. Center for Health Professions Education, Uniformed Services University of the Health Sciences, Bethesda, MD, USA

    Steven Durning

Authors
  1. Nyoli Valentine
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Steven Durning
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ernst Michael Shanahan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lambert Schuwirth
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nyoli Valentine.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest. The views expressed herein are those of the authors and not necessarily those of the Department of Defense or other federal agencies.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Valentine, N., Durning, S., Shanahan, E.M. et al. Fairness in human judgement in assessment: a hermeneutic literature review and conceptual framework. Adv in Health Sci Educ 26, 713–738 (2021). https://doi.org/10.1007/s10459-020-10002-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10459-020-10002-1

Keywords

Search

Navigation