Elsevier

Assessing Writing

Volume 41, July 2019, Pages 14-24
Assessing Writing

Lower English proficiency means poorer feedback performance? A mixed-methods study

https://doi.org/10.1016/j.asw.2019.05.001Get rights and content

Highlights

  • A mixed-methods design is adopted to examine the relation between English proficiency and peer feedback performance.

  • This study compares the feedback amount, type and quality between lower and higher English proficiency students.

  • The two groups do not differ in feedback amount or feedback quality, but differ in feedback type.

  • Feedback performance is shaped by cognitive, affective, sociocultural, and instructional factors.

Abstract

This study adopts a mixed-methods design and examines the relation between English proficiency and peer feedback performance. Data sources included peer feedback made by 23 lower English proficiency (LEP) students and 23 higher English proficiency (HEP) students, and semi-structured interviews with four LEP and four HEP students from that sample. Quantitative analysis did not find significant difference between the two groups in feedback amount or feedback quality, but significant difference was found in feedback type. LEP students tended to make more clarification requests in content feedback, more suggestions and fewer direct corrections in language feedback. Qualitative analysis of students’ interviews showed that individual factors (i.e. genre knowledge, L1, L2, reference materials, and imagined identities) and contextual factors (i.e. anonymity, feedback separation, and time availability) shaped their feedback processes. Based on these findings, a conceptual framework is suggested to explain how feedback performance is enabled or constrained by cognitive, affective, sociocultural, and instructional factors. The framework can be a useful heuristic for EFL teachers to create facilitative conditions to engage and empower LEP students in feedback activities.

Introduction

Peer feedback is well supported by a number of theories, including process writing theory, interactionist theory, sociocultural theory of learning, and many others (Hyland & Hyland, 2006; Lee, 2017; Yu & Lee, 2016a). Empirically, peer feedback has been found to reduce language errors (Diab, 2010), improve text complexity (Allen & Mills, 2013), increase audience awareness (Chang, 2015), and develop learner autonomy (Yang, Badger, & Yu, 2006). Studies further show that peer feedback is not inferior to teacher feedback (Caulk, 1994), but differs in focal areas. Specifically, peer feedback tends to focus more on content and meaning issues, while teacher feedback focuses on language issues (Tsui & Ng, 2010; Yang et al., 2006). Therefore, peer feedback and teacher feedback can play complementary roles in EFL writing pedagogies (Lee, 2017; Tsui & Ng, 2000).

Despite the theoretical and empirical support of peer feedback, EFL writing instructors are faced with multiple challenges when designing feedback activities. One vexing challenge is students’ English proficiency (Nelson & Murphy, 1993; Tsui & Ng, 2000). When we assign students into feedback groups, should we form heterogeneous or homogenous groups? While ability matching is not a significant predictor of feedback quality in the L1 context (Huisman, Saab, van Driel, & van den Broek, 2017), EFL students hold mixed perceptions of their ability to give feedback. For instance, in Allen’s (2015) survey study, half of the participants believed that English proficiency had strong or some influence on the feedback process, while the other half did not think so. In Zhao’s (2011) study, students regarded English proficiency as a prominent factor impacting their feedback focus, type, and accuracy. Due to their under-developed English competence, lower English proficiency (LEP) students are usually considered not knowledgeable enough to identify or rectify language issues (Nelson & Murphy, 1993). Consequently, they tend to be marginalized and play the role of a feedback receiver (Lee, 2017; Yu & Lee, 2016a). Is this distrust of LEP students justified? Unfortunately, not much is known given the paucity of empirical research on LEP students’ feedback products and processes (Yu & Lee, 2016b).

Existing studies examining the relation between English proficiency and feedback performance can be grouped into two strands. The first strand compares how higher English proficiency (HEP) students and LEP students differ in their feedback products. Studies have tended to employ a quantitative design and yield conflicting results. For instance, Kamimura (2006) found that LEP students made more comments than HEP students, while Allen and Mills (2016) found that HEP students gave more comments than LEP students. The result discrepancy might be caused by the grouping methods and feedback procedures. In Kamimura’s (2006) study, LEP and HEP students swapped the drafts amongst themselves and formed matched proficiency dyads (LEP-LEP; HEP-HEP). They were asked to respond to a feedback worksheet, which contained more content-related questions than language-related questions. As such, LEP students’ attention might be more drawn to their LEP peers’ content issues (which were more prevalent than those in HEP peers’ drafts), thus boosting the total number of peer comments.

The second line of research adopts a qualitative design to describe the dynamics in mixed-proficiency feedback groups. Yu and Lee (2016b) focused on 12 students in three heterogeneous groups and found that LEP students were able to provide a wide range of quality comments, about 80% of which were subsequently incorporated by their peers. The researchers identified four factors that empowered LEP students in peer feedback: congenial group relationship, positive attitude, use of first language (L1), and feedback training. The benefits of heterogeneous grouping is corroborated by Yu and Hu (2017a). HEP students could learn from the feedback given by LEP students, and developed their audience awareness and L2 writing knowledge. Similarly, Allen and Katayama (2016) examined the group dynamics of six dyads with different English proficiency levels. After triangulating multiple data sources, the researchers concluded that language proficiency and perception of relative proficiency had a great impact on the number and the type of feedback provided and incorporated.

Although the aforementioned studies are informative, their insights are potentially limited by the research methods. Quantitative methods have been used to measure the products of peer feedback, such as the number, type, quality, and uptake of feedback made by LEP students (Allen & Mills, 2016; Kamimura, 2006). On the other hand, qualitative methods have been adopted to understand the processes, whereby LEP students negotiate a multitude of individual and contextual factors (Allen & Katayama, 2016; Yu & Hu, 2017a, 2017b; Yu & Lee, 2016a, 2016b). However, feedback product and process are two sides of the same coin—feedback performance. To fully understand how LEP students perform and why they perform the way they do, the connection between product and process needs to be addressed. Therefore, a mixed-methods design is warranted to examine whether quantitative differences exist in the feedback products of HEP and LEP students, and what factors shape these differences or similarities in the processes of giving feedback.

Another potential limitation of previous research is that LEP and HEP students did not work on the same piece of writing. In Kamimura’s (2006) study, as described previously, matched proficiency dyads were created, while in Allen and Mills’ (2016) study, students formed self-initiated dyads, producing four types of writer-reviewer combinations: HEP-HEP, HEP-LEP, LEP-HEP, and LEP-LEP. In all these scenarios, students worked on different essays, which had varying amounts of treatable issues for feedback-givers. A more refined design is to have a LEP student and a HEP student make feedback on the same piece of writing by a third student. In this way, we place LEP and HEP students on a level playing field, and we are more justified to associate the observed different feedback performances with the proficiency gap.

Section snippets

Research questions

To gain a fuller understanding of the relation between English proficiency and peer feedback performance, this study was guided by the following questions: (1) Do HEP and LEP students differ in feedback amount, type, and quality? (2) Why do HEP and LEP students share similarities or differences in feedback performance?

Context and participants

This study involved 69 first-year students who studied English as a foreign language (EFL) at a first-tier university in Guangdong Province, southern China. They came from three

Feedback amount

As shown in Table 3, the two proficiency groups did not statistically differ in the total amount of content or language feedback. This result is different from the observation in Allen and Mills (2016), who found that HEP students made more comments. The inconsistency might be caused by the grouping method and the policy of anonymity. In this study, an HEP student and a LEP student commented on the same text and a double-blind policy was enacted. In Allen and Mills’s (2016) study, students

Discussion

Based on the quantitative and qualitative analysis, a conceptual framework is suggested to understand the feedback performance by LEP and HEP students (see Fig. 2). Inspired by Ellis (2010) and Yu and Hu (2017b), the framework has two overarching domains: individual and contextual. Individual factors include cognition and affection, while contextual factors include sociocultural and instructional affordances and constraints. Each factor has its specific instantiations and impacts on feedback

Pedagogical implications

Some EFL writing instructors might presume that lower English proficiency means poorer feedback performance. As the title of this paper asks, is this presumption a myth or a reality? The research results indicate that the distrust of LEP students’ ability to make effective feedback is misplaced. LEP students do not differ in feedback amount or quality, but differ in feedback type. Therefore, we must reframe our question from whether to how: instead of questioning LEP students’ ability to give

Conclusion

This study contributes to a growing body of research in understanding feedback performance of LEP students. It is found that HEP and LEP students do not differ in feedback amount or feedback quality, but differ in feedback type. LEP students tend to make more clarification requests in content feedback, more suggestions and fewer direct changes in language feedback. Based on students’ interviews, a conceptual framework is proposed to explain how feedback performance is enabled or constrained by

Zhiwei Wu holds a Ph.D. in applied linguistics and is an assistant professor at Hong Kong Polytechnic University. He was a visiting scholar at Lancaster University (2014) and the Pennsylvania State University (2016–2017). His research interests include student motivation, academic writing, and multiliteracies.

References (45)

  • A.B. Tsui et al.

    Do secondary L2 writers benefit from peer comments?

    Journal of Second Language Writing

    (2000)
  • M. Yang et al.

    A comparative study of peer and teacher feedback in a Chinese EFL writing class

    Journal of Second Language Writing

    (2006)
  • S. Yu et al.

    Understanding university students’ peer feedback practices in EFL writing: Insights from a case study

    Assessing Writing

    (2017)
  • S. Yu et al.

    An analysis of Chinese EFL students’ use of first and second language in peer feedback of L2 writing

    System

    (2014)
  • S. Allaei et al.

    Exploring the dynamics of cross-cultural collaboration in writing classrooms

    The Writing Instructor

    (1990)
  • D. Allen

    Personal and procedural factors in peer feedback: A survey study

    Komaba Journal of English Education

    (2015)
  • D. Allen et al.

    Peer feedback in the academic English classroom: A pilot study

    Komaba Journal of English Education

    (2013)
  • D. Allen et al.

    The impact of second language proficiency in dyadic peer feedback

    Language Teaching Research

    (2016)
  • J. Bitchener et al.

    The value of a focused approach to written corrective feedback

    ELT Journal

    (2009)
  • N. Caulk

    Comparing teacher and student responses to written work

    TESOL Quarterly

    (1994)
  • N.M. Diab

    Effects of peer-versus self-editing on students’ revision of language errors in revised drafts

    System

    (2010)
  • R. Ellis

    Epilogue: A framework for investigating oral and written corrective feedback

    Studies in Second Language Acquisition

    (2010)
  • Cited by (0)

    Zhiwei Wu holds a Ph.D. in applied linguistics and is an assistant professor at Hong Kong Polytechnic University. He was a visiting scholar at Lancaster University (2014) and the Pennsylvania State University (2016–2017). His research interests include student motivation, academic writing, and multiliteracies.

    View full text