EAP teachers’ assessment literacy: From theory to practice

https://doi.org/10.1016/j.stueduc.2021.101042Get rights and content

Highlights

  • The study explored the Iranian EAP teachers’ assessment literacy (AL).

  • Beliefs and actual classroom practices on assessment were compared.

  • The findings showed that only a few EAP teachers were assessment literate.

  • Mismatches were found between assessment beliefs and assessment practices.

  • Study field, university degree, and training courses significantly predicted AL.

Abstract

This study explored the extent to which English for Academic Purposes (EAP) teachers were assessment literate, their beliefs about assessment, their actual classroom assessment practices, and the reasons behind the possible mismatches. Data were collected through Assessment Literacy Survey (Mertler, 2003), semi-structured interviews, classroom observation checklists, and the field notes obtained from a sample of 27 language and 23 content teachers of e.g., engineering, basic sciences, medical sciences, etc., out of whom 28 participated in the process of interviews and observations. Data were analyzed both quantitatively and qualitatively in this sequential explanatory mixed-methods study. The findings indicated that only a few EAP teachers were assessment literate and that field of study, university degree, and assessment training courses were statistically significant predictors of EAP teachers’ assessment literacy. Inconsistencies between assessment beliefs and assessment practices were found across both groups of teachers for which numerous reasons were noted by the participants. Implications for EAP teachers themselves and EAP teacher education programs are discussed.

Introduction

English for Academic Purposes (EAP), as a sub-branch of English for Specific Purposes (ESP), aims at helping students to go through their academic careers and research. EAP is divided into two general branches, namely English for General Academic Purposes (EGAP), which concentrates on those parts of language common to various fields and disciplines, and English for Specific Academic Purposes (ESAP), which makes a distinction between different fields and disciplines (Hyland, 2006). The quality of the assessment practices employed by EAP teachers plays a significant role in improving the effectiveness of teaching. As Berry, Sheehan, and Munro (2019) assert, learning can be supported and improved by effective assessment, and the capacity to engage with various teaching, learning, and assessment practices is of paramount importance for teachers. Looney, Cumming, van der Kleij, and Harris (2017, p. 1) assert that “What and how student learning is assessed identify what is valued or important for students to learn”, meaning that assessment influences both the process and the product of learning.

Moreover, as Engelsen and Smith (2014) maintain, teachers’ assessment knowledge and skill determine the quality of assessment practices. Thus, teachers’ assessment literacy (AL), which is defined by Webb (2002, p.1) as “the knowledge of means for assessing what students know and can do, how to interpret the results from these assessments, and how to apply these results to improve student learning and program effectiveness”, is highlighted by the scholars in the field to specify language teachers and instructors’ requirements of language classroom assessment (Stoynoff & Chapelle, 2005). Putting emphasis on the significance of assessment, Popham (2009) maintains that the quality of education can be attenuated by teachers’ insufficient assessment knowledge. Moreover, being aware of new testing and assessment paradigms is necessary for teachers (Marso & Pigge, 1993) because poorly designed tests might result in superficial learning and cause a mismatch between assessment practices in the classrooms and ultimate goals of teaching and learning (Heritage, 2013). However, it is worth mentioning that teachers need a convincing reason to implement various forms of assessment in practice and as Vattøy (2020) maintains, if language teachers regard innovative forms of assessment as additional work for the mere purpose of documentation, their resistance and challenges might arise as a result.

Due to the breadth and depth of the concept of AL (Giraldo, 2021), the field is still surrounded by controversy over what exactly language AL can and should be for the teachers (Stabler-Havener, 2018). Language AL is, in fact, defined in the present study as the teachers' knowledge of the theoretical aspects of assessment and their ability to put that knowledge into practice in actual classroom settings. In other words, an assessment literate teacher is the one who has good command of assessment methods and techniques, test development, scoring, and interpretation, and the factors affecting test results and publicizing them. Assessment literate teachers are thus expected to know about such new assessment methods as alternative and dynamic forms of assessment, validity, reliability, ethicality, washback, impact, test interpretation, and the way a test affects the learning process and the academic lives of students as well as the effects tests might have on the society at large.

English language is taught at Iranian universities for both general and academic purposes. In both cases, the main goal is to prepare language learners to read and write in English which is known as the language of science. These two purposes have been specified by the educational policymakers and there is an expectation that university students reach an acceptable level of functioning in English at least in their Bachelor of Arts (BA) program. However, dissatisfaction with various dimensions of EAP education in Iran can be observed among different stakeholders (Soodmand Afshar & Movassagh, 2016; Tavakoli & Tavakol, 2018). Although EAP teaching has heavily focused on the content of instruction and many researchers (e.g., Atai, 2002, 2013; Atai & Mazlum, 2013; Hayati, 2008; Khany & Tarlani-Aliabadi, 2016; Soodmand Afshar & Movassagh, 2016) have already examined EAP teaching in Iran, its methodology and especially its assessment have not yet received due attention within Applied Linguistics literature (Soodmand Afshar, Tofighi, Asoudeh, & Ranjbar, 2018)

As Zulaiha, Mulyono, and Ambarsari (2020) maintain, a vital element of teachers’ job and an integral part of teaching and learning processes is assessment of learners’ performance. All teachers need to be competent in assessment to be capable of determining learners’ knowledge and ability; otherwise, they cannot decide rationally if they (i.e., the teachers) have been successful in their performance and might, consequently, lack the ability to decide what to do next to assist their learners (Tsagari et al., 2018).

One reason to conduct the present study was the influence EAP teachers’ AL might have on the quality of EAP education because, not only does teachers’ AL provide the essential potentials for meaningful interpretations of the test results and critical discussions of assessment practices to make appropriate decisions like test development, administration, interpretation, task design, etc. (Giraldo, 2021), but, as Kremmel and Harding (2020) maintain, it also plays a significant role in the process of teaching and learning. That is, in our view, dissatisfaction with various dimensions of EAP education can be reduced if teachers become sufficiently assessment literate so that they can identify and assess learners’ problems properly and alleviate them accordingly. Thus, due to the crucial role assessment literacy plays in teaching in general and assessment in particular, this study intends to evaluate EAP teachers’ AL at Iranian universities with its main focus being on the Iranian EAP teachers’ current knowledge in assessment and the extent they implement their knowledge in practice. In other words, the study aims to detect the assessment literacy of Iranian EAP teachers and find out the factors which best account for teachers’ assessment literacy. Additionally, it intends to detect the possible mismatches existing between teachers’ beliefs and their actual classroom performance in assessment.

Section snippets

Assessment literacy and language assessment literacy: definitions

The concept of AL was first introduced into general education and was then integrated into language education. The notion of AL has changed over time, due to an enhanced interest among the researchers to probe the notion in the field. Overall, there is yet no unanimously agreed-upon definition in relation to the notion of AL (Fulcher, 2012; Taylor, 2009; Walters, 2010). Within the general education literature, Stiggins (1991, 1999) was perhaps the first person who invented the concept

Design

The present study enjoyed both qualitative and quantitative research data collection, analysis, and interpretation procedures since mixed methods research is thought to yield a much more comprehensive result (Richards, 2003). According to Creswell (2009), the present study employed a sequential explanatory mixed-methods design as the quantitative data was first collected in the form of a test. Interviews and observations were then used to collect qualitative data.

Participants

The sample of the present study

Technical english language teachers’ AL

The first research question of the study was set out to investigate the extent to which Technical English language teachers were assessment literate. To this end, descriptive statistics were calculated, the results of which are presented in Table 1.

As shown in Table 1, the results of the teachers' responses to CALI showed that only 13 teachers (26 %) out of the whole population (N = 50) passed the cut-off score which was 24.5 according to Plake and Impara (1993) as explained earlier. This

Conclusion and implications

The present study aimed at exploring the extent to which both EAP language teachers and EAP content teachers were assessment literate and its association with such various factors as gender, field of study, university degree, experience, and assessment training courses. The study also focused on what EAP teachers preached and what they practiced while assessing their students. We found that EAP teachers were, overall, not assessment literate enough to utilize innovative forms of assessment in

Hassan Soodmand Afshar is a full professor of Applied Linguistics, Department of English Language, Bu-Ali Sina University, Hamedan, Iran. He has published extensively in such various accredited international journals as System, Journal of English for Academic purposes, Thinking Skills and Creativity, Research Papers in Education,Reflective Practice, Language Learning Journal, Issues in Educational Research, e-FLT, Intercultural Communication Research, etc. and various local journals. His recent

References (92)

  • American Federation of Teachers et al.

    Standards for teacher competence in educational assessment of students

    (1990)
  • M.R. Anani Sarab et al.

    Teachers’ perception of content knowledge and language knowledge in EAP assessment: A case of argument-based approach toward EAP validity

    The Asian ESP Journal

    (2020)
  • M.R. Atai

    ESAP curriculum planning in Iran: An incoherent educational experience

    Special Issue of the Journal of Persian Literature and Human Sciences of Tehran Teacher Training University

    (2002)
  • M.R. Atai

    English for specific purposes: International trends and Middle East concerns

  • M.R. Atai et al.

    English language teaching curriculum in Iran: Planning and practice

    The Curriculum Journal

    (2013)
  • V. Berry et al.

    What does language assessment literacy mean to teachers?

    ELT Journal

    (2019)
  • British Association of Lecturers in English for Academic Purposes

    TEAP working party: The competency framework for teachers of English for academic purposes

    (2008)
  • G.T. Brown et al.

    Assessment for student improvement: Understanding Hong Kong teachers’ conceptions and practices of assessment

    Assessment in Education: Principles, Policy & Practice

    (2009)
  • S. Bruce

    Assessing EAP: Theory and practice in assessment literacy [book review]

    English Australia Journal

    (2017)
  • M. Carr

    Assessment in early childhood settings: Learning stories

    (2001)
  • J. Cohen

    Statistical power analysis for the behavioral sciences

    (1988)
  • J.L. Coiduras et al.

    Initial teacher education in a dual-system: Addressing the observation of teaching performance

    Studies in Educational Evaluation

    (2020)
  • K.M.T. Collins et al.

    A mixed methods investigation of mixed methods sampling designs in social and health science research

    Journal of Mixed Methods Research

    (2007)
  • J.W. Creswell

    Research design: Qualitative, quantitative, and mixed methods approaches

    (2009)
  • F. Davidson et al.

    Testcraft: A teacher’s guide to writing and using language test specifications

    (2002)
  • A. Davies

    Textbook trends in teaching language testing

    Language Testing

    (2008)
  • Z. Dörnyei

    Research methods in applied linguistics: Quantitative, qualitative and mixed methodologies

    (2007)
  • D. Douglas

    Assessing languages for specific purposes

    (2000)
  • D. Douglas

    ESP and assessment

  • T. Dudley-Evans et al.

    Developments in ESP: A multi-disciplinary approach

    (1998)
  • J. Edge et al.

    May I see your warrant, please?: Justifying outcomes in qualitative research

    Applied Linguistics

    (1998)
  • K.S. Engelsen et al.

    Assessment literacy

  • T. Firoozi et al.

    The language assessment literacy needs of Iranian EFL teachers with a focus on reformed assessment policies

    Language Testing in Asia

    (2019)
  • M. Fishbein et al.

    Predicting and changing behavior: The reasoned action approach

    (2010)
  • G. Fulcher

    Assessment in English for academic purposes: Putting content validity in its place

    Applied Linguistics

    (1999)
  • G. Fulcher

    Assessment literacy for the language classroom

    Language Assessment Quarterly

    (2012)
  • F. Giraldo

    A reflection on initiatives for teachers’ professional development through language assessment literacy

    Profile: Issues in Teachers’ Professional Development

    (2021)
  • S.B. Green

    How many subjects does it take to do a regression analysis

    Multivariate Behavioral Research

    (1991)
  • R.J. Harris

    A primer of multivariate statistics

    (1975)
  • A. Hasselgreen et al.

    European survey of language testing and assessment needs. Part 1: General findings

    (2004)
  • M. Hayati

    Teaching English for special purposes in Iran: Problems and suggestions

    Arts and Humanities in Higher Education

    (2008)
  • M. Heritage

    Gathering evidence of student understanding

  • E.K. Horwitz

    Using student beliefs about language learning and teaching in the foreign language methods course

    Foreign Language Annals

    (1985)
  • J. Huang et al.

    Exploring assessment literacy

    Higher Education of Social Science

    (2016)
  • T. Hutchinson et al.

    English for specific purposes

    (1987)
  • K. Hyland

    English for academic purposes: An advanced resource book

    (2006)
  • Cited by (11)

    • Assessment literacy of college EFL teachers in China: Status quo and mediating factors

      2022, Studies in Educational Evaluation
      Citation Excerpt :

      The past two decades have witnessed a large body of literature on conceptualization of the construct of generic AL (Chan & Luk, 2021; DeLuca et al., 2016; Popham, 2009; Xu & Brown, 2016) or discipline-specific AL like language AL (LAL) (Fulcher, 2012; Inbar-Lourie, 2012; Kremmel & Harding, 2020; Levi & Inbar-Lourie, 2020; Malone, 2013; Pill & Harding, 2013; Taylor, 2009, 2013), measures of AL (DeLuca & Klinger, 2010; Kremmel & Harding, 2020; Mertler, 2004; Pastore & Andrade, 2019), status quo of in-service teachers’ AL or needs for AL training (e.g. Crusan et al., 2016; Vogt et al., 2020), approaches to develop or improve teacher AL (e.g. Fan et al., 2011; Koh, 2011; Yan & Fan, 2020). The status quo of in-service teachers' AL has been investigated both globally (Crusan et al., 2016; Kremmel & Harding, 2020) and in individual countries such as the United States (Mertler, 2004; Plake et al., 1993), Canada (DeLuca & Klinger, 2010), European countries (Fulcher, 2012; Tsagari & Vogt, 2017; Vogt & Tsagari, 2014; Vogt et al., 2020), Oman (Alkharusi et al., 2011), Iran (Soodmand Afshar & Ranjbar, 2021; Tajeddin et al., 2022), China (Lan & Fan, 2019; Xu & Brown, 2017), etc. Among these studies, a few of them compared language teachers’ perception of AL level and their perceived needs for AL training and identified big gaps between the two (Lan & Fan, 2019; Vogt & Tsagari, 2014; Vogt et al., 2020).

    • Impact of educating faculty on student assessment: beyond satisfaction level

      2024, Journal of Applied Research in Higher Education
    View all citing articles on Scopus

    Hassan Soodmand Afshar is a full professor of Applied Linguistics, Department of English Language, Bu-Ali Sina University, Hamedan, Iran. He has published extensively in such various accredited international journals as System, Journal of English for Academic purposes, Thinking Skills and Creativity, Research Papers in Education,Reflective Practice, Language Learning Journal, Issues in Educational Research, e-FLT, Intercultural Communication Research, etc. and various local journals. His recent (2020) book chapter in Routledge, “Thinking Skills and Creativity in Second Language Education” (edited by Li Li), has caught the attention of the readers. He is also the vice-president and a member of Board of Directors of TELLSI (Teaching English Language and Literature Society of Iran) since 2012. Additionally, he is the Editor-in-chief of TEL journal belonging to TELLSI.

    Naser Ranjbar is a PhD candidate in TEFL at Bu-Ali Sina University, Hamedan, Iran. He is the author of some articles in TEFL and has presented in International and national conferences. He has taught English at the Ministry of Education for many years and is a lecturer at university. His research interests include language testing and assessment, EAP education, and psychology of education.

    View full text