Introduction

Today, reflective teaching is essential for all teachers in different fields of education. Without regular reflective practice on instructional context, the teacher cannot understand how effective his/her teaching is for encouraging, motivating and evaluating students or how the students’ emotions, lives, and directions are shaped, or how their activities and reading text are related to their learning efforts (Çimer et al., 2013). Actually, reflective teaching is a kind of teaching approach which can encourage teachers to improve their teaching skills by engaging in critical reflection on their teaching learning process (Ashraf & Zolfaghari, 2018). In the field of teaching English reading comprehension, the teacher’s responsibility is to help students express their own thoughts by the use of their reading comprehension skills (Ahmada, 2019). To help students use these reading skills well, instructional strategies are very essential for the teachers’ success as a professional. Regarding teaching strategies of reading comprehension in ELT, different researchers experimented with different teaching strategies for students’ English reading comprehension achievement. Examples of such experiments include Acim's (2018) research about Socratic method; Ahmada's (2019) Jigsaw learning method, Gouthro's (2020) interactive teaching method, Ostovar-Namaghi and Shahhosseini's (2011) reciprocal teaching method, Swartz's (2017) questioning method, and Yang's (2018) English translation teaching method. These studies confirmed that teaching methods could help students improve their learning achievement in reading comprehension. However, no instructional method is perfect, and thus the teacher’s reflection practices are necessary to fill in some gaps of method-centered teaching (Aliakbari & Adibpour, 2018).

Reciprocal teaching, involving four strategies of predicting, questioning, clarifying and summarizing are activities of great importance for improving students’ reading comprehension and improving vocabulary knowledge by sharing concepts/ideas among students with the use of their background knowledge (Lestari, 2016). The four strategies of reciprocal teaching can also improve students’ cognitive and metacognitive reading skills (Cooper & Greive, 2009). Furthermore, many other studies (Mannong, 2018; Okkinga et al., 2018; Stricklin, 2011) investigated the effectiveness of reciprocal teaching on students’ reading comprehension achievement. They commonly agreed that reciprocal teaching is very effective for improving students’ reading comprehension skills. Therefore, we are also curious to investigate the effectiveness of reciprocal teaching in the Myanmar context.

However, there are some weaknesses of the reciprocal teaching method. If a teacher does not have expertise in coaching, modeling, guiding students’ teamwork, and in managing the hands-on tools for guiding their collaborative work, the reciprocal teaching method cannot be effective enough for students’ reading comprehension achievement (Okkinga et al., 2018). To effectively use reciprocal reaching, Mannong (2018) also cautioned that teachers should consider the suitability of the method, the tools, the characteristics of the classroom environment, the characteristics of the learning group, and students’ learning preferences in order to improve student achievement and learning motivation. Richards and Lockhart (2005) stated that “there are many factors that influence how teachers approach their work and which particular strategies they employ to achieve their goals” (p. 97). And they also mentioned that the instructional context involving the students themselves, the teacher’s strategy, the reading text, and the kinds of classroom activities in which teachers work has an important influence on their teaching for students’ achievement. Therefore, these factors, weaknesses of reciprocal teaching alone and the importance of instructional context, call for the teacher’s reflective action to examine whether the instructional context is effective in producing better students’ achievement (Richards & Lockhart, 2005). In this study we aimed to apply the reciprocal teaching method combined with reflective teaching to improve students’ reading comprehension achievement, using what we term the reflection-based reciprocal teaching (RBRT) approach.

Theoretical background

Reciprocal teaching method

Reciprocal teaching is a strategy that aims at increasing students’ reading comprehension skills by scaffolding instructional procedures through four comprehension-fostering and comprehension-monitoring strategies (Navaie, 2018): (1) the student’s own questioning based on the text, (2) summarizing using the student’s words based on the text, (3) clarifying what the student does not understand in the text, and (4) the student’s own prediction of what comes next in the text.

In the general methodology of reciprocal teaching, it is not only about students’ discussions in small groups. Ostovar-Namaghi and Shahhosseini (2011) suggest that the teacher should model for students how to form a group, how to participate in a group, what to do, and how to take different roles in the learning process. The author discussed students’ roles in the reciprocal teaching approach, which are questioning, clarifying, summarizing, and predicting.

In the step of questioning, the term “questioning” means identifying the key words or main information, ideas, and themes from the text, and creating questions based on the student’s own words. These questions should not merely ask about the unknown words but also construct a good foundation for understanding the whole text (Rodli & Prastyo, 2017). The step “clarifying” is the process of understanding unknown words, answering questions arising from the difficulties of comprehending the text, and clarifying the meaning of the text. This step is important for all students. If the meaning is clear, students will understand the whole text and this will support other steps such as summarizing and predicting. Therefore, Stricklin (2011) also suggested that students use extra tools (e.g., dictionaries or a thesaurus) as part of this process. The step of summarizing involves identifying key ideas or information from the text, and organizing this information into a meaningful statement in the students’ own words. This summarized statement should cover the essential parts of a paragraph or text. According to Williams (2010), students should select these ideas from the text and write the main ideas in their book or on a worksheet using their own words. The last step, predicting, is the process in which students compare their prior knowledge about the text to the new information they obtain from the text. After making a comparison between the old and the new knowledge, they then create future statements. This step, “prediction,” refers to students’ ideas in the form of statements regarding upcoming events (Doolittle et al., 2006).

If necessary, the teacher provides further feedback to student groups to facilitate the students’ effective and interactive participation in the reading comprehension process (Ghorbani et al., 2013).

Importance of transformative learning theory

In education, most kinds of learning fundamentally stem from the traditional theories of behaviorist, cognitivist, and constructivist theories (Şahin & Doğantay, 2018). In behaviorist theory, the student learns the new information or behaviors by associating the stimuli with the response (Nussbaum, 2019). In cognitivist theory, the student learns the new information and remembers what has been learnt based on the cognitive process or information process in his/her mind (Çeliköz et al., 2016). From a constructivist point of view, the individual constructs new knowledge or information based on their prior knowledge (Şahin & Doğantay, 2018). In the case of students’ learning by the reciprocal teaching method, they are taught to perform four strategies of predicting, questioning, clarifying and summarizing (Navaie, 2018). These reciprocal teaching strategies are based on the students’ stimuli and behavioral responses (predicting, questioning, clarifying and summarizing), their cognitive functions, and constructive processes based on their prior knowledge. Therefore, it is certain that these three theories are supporting the reciprocal teaching method to help students’ reading comprehension progress.

Furthermore, in Mezirow’s transformative learning theory, Mezirow (1991) stated that transformative learning can improve students’ learning although not all types of students’ learning are transformative. Transformative learning is a developmental learning model in which individuals construct new knowledge by reflection on their background schema (knowledge they had already learnt) (Mezirow, 1996; Wang & Cranton, 2011). In the twenty-first century, as the world is in rapid change and the learning process is also complex and changing, the transformative learning theory has become more essential in adult learning (Şahin & Doğantay, 2018). The transformative learning theory is a type of learning theory which is based on radical change from the concepts, assumptions, and expectations that adults had already stored in their mind, and that kind of radical change craves for a critical reflection effort to construct meaningful information (Mezirow, 2006). Therefore, reflective teaching is an essential part of transformative learning theory. In this research of adult learning, we tried to qualify the reciprocal teaching method by incorporating reflective teaching which encourages students’ transformative learning.

Reflective teaching model for reading comprehension

In education, the term “reflection” is used in different ways, such as reflective teaching, reflective practices, reflective feedback, or reflective questions. The word “reflection” was first used by Dewey (1933) in characterizing the reflective practice for “accessing the grounds (justification) of one’s beliefs, the process of rationally examining the assumptions by which we have been justifying our convictions” (Fook, 2015, p. 13). Regarding reflective teaching, it is the teachers’ meaningful action of recalling, considering, and evaluating their experiences (Spalding & Wilson, 2002). Every teacher has a responsibility to reflect on and evaluate their teaching practice in order to improve their professional capabilities and their students’ effective learning (Fatemipour, 2013). Reflective teaching is a cyclical process involving the teacher’s monitoring, evaluating and revising their teaching learning process (Pollard et al., 2014). Efe (2009) and Gheith and Aljaberi (2018) firmly stated that developing a teacher’s reflective abilities is of great importance not only in teacher training programmes, but it is also the basic requirement for fulfilling students’ well-development. Ways of reflection are either formal or informal. Both, however, can help teachers become aware of students’ individual differences, opinions, feelings, and learning strategies (Gill, 2014).

Richards and Lockhart (2005) established that reflective teaching is a cyclical process in which any kind of teaching method can be used together with the teacher’s reflection. They also presented a model of the reflective teaching process as an action plan that involves “planning, acting, observing and reflecting” (p. 3). Hulsman et al. (2009) pointed out as well that reflective teaching is a cyclical process of action, observation, analysis, presentation of analyses, and feedback.

In the case of the reading comprehension process, there are various approaches to teaching foreign language reading comprehension. In a study by Rastegar et al. (2017), the authors found that foreign language reading comprehension is a process of meaning construction that requires a combination of multiple tasks: “reading of words, knowledge of words, text organization, strategies and even knowledge of the world” (p. 67). Nordin et al. (2013) also suggested that students’ reading comprehension is an interactive process that takes place between their background schemata and the text they read. For ELT teachers, Khaki (2014) suggested three strategies for teaching reading comprehension: top-down, bottom-up, and interactive. In addition to these three approaches, the ELT teachers should consider three other variables during their instructional process: “(1) text characteristics; (2) reader/viewer characteristics; and (3) social context” (Zhang, 2016, p. 132).

In a review of research studies on reflective teaching and English language reading comprehension processes, it was found that some studies (Garzon, 2018; Pollard et al., 2014; Ratminingsih et al., 2018; Richards & Lockhart, 2005) identified four main steps in the process of reflective teaching: planning, acting, reflecting, and evaluating, which teachers should do when teaching. In the case of reading comprehension, some research (Suwanto, 2014; Walker, 2008; Yang, 2016) mentioned that this is a complex context involving reader, strategy, text, and task, which teachers should consider during the instructional process. Based on these two main conceptual reviews (reflective teaching process and reading comprehension process), we developed a reflective teaching model for reading comprehension, which is shown in the following conceptual framework (Oo & Habók, 2020).

Conceptual framework

The present study is based on the conceptual framework of the reflective teaching model for reading comprehension (Fig. 1). According to Oo and Habók (2020), that instructional model comprises four main steps: planning, acting, reflecting, and evaluating. In this reflective model, the teacher applies the reciprocal teaching method to encourage students’ English reading comprehension achievement. Therefore, this instructional design is based on the RBRT approach. The main steps of the RBRT approach (as the conceptual framework) are presented in Fig. 1.

Fig. 1
figure 1

Reflective teaching model for reading comprehension (Oo & Habók, 2020, p. 133)

Full size image

In the planning step, the teacher plans his/her instructional context using the reciprocal teaching method in the way mentioned above, that is, considering whom to teach (reader), what to teach (text), how to teach (strategy), and what kinds of activities the students should do (task). In the step of acting, it is essential for the teacher to carefully complete the previously planned parts. In this part, the teacher engages (acts) in reciprocal teaching as planned ahead of class. The teacher first demonstrates how to predict, question, clarify and summarize the reading text. Then, the teacher gives students the activities related with the reciprocal teaching method, i.e., the students perform the roles of predictor, questioner, clarifier, and summarizer. In this step, the teacher takes on the role of guide by coaching, monitoring, and suggesting as necessary.

In the step of reflecting, after the students’ roles using the four kinds of activities, the teacher revises the whole text-unit with related exercises and questions. These revision exercises give the students an opportunity to reflect on what they had learnt from the reading text and support their transformative learning. And for the teacher’s reflection on the instructional context, various kinds of reflective tools such as a teacher’s journal, reports on lessons, questionnaires, audio and video recordings, classroom observation schemes, and student feedback (Fook, 2015) can be used in this step of the RBRT approach. These reflective tools can be employed to reflect on the instructional context involving reader, text, strategy, and task. In this step of the current research, the participating teachers used two main types of reflecting tools: peer observation (observation scheme) and students’ eyes/evaluation (questionnaire completed by the students based on their learning preferences) according to Brookfield’s (2017) work.

In the last step, evaluating, formative and summative assessments were used to evaluate the instructional context, which consists of reader, text, task, and strategy. In language learning, Houston and Thompson (2017) indicated the importance of formative and summative assessment as “processes leading to judgments about opportunities for improvement in ongoing activities and about the worth of a completed activity, respectively” (p. 2). In the formative assessment (for ongoing activities), the teacher assessed students’ reflective feedback through a student questionnaire, and peer observation through an observation scheme. As the summative assessment (for the completed activity), the teacher employed a posttest at the end of the treatment period (Looney, 2011).

Research background

The study was conducted in Myanmar, where transformative political and economic events have strongly impacted the education system. The Myanmar education system is highly centralized and top-down with Myanmar teachers, schools, colleges, and universities having no autonomy (Ulla, 2017). They are all under government control. That is, the Ministry of Education has responsibility for hiring, placing, and promoting qualified in-service teachers (UNESCO, 2020). Myanmar education has been in a poor state in relation to other countries in the world due to the country’s economic difficulties in the last decades (Hayden & Martin, 2013). The evidence of this is clear in the poor condition of classrooms, school buildings, outdated traditional teaching methods, and a lack of training for upgrading teachers’ skills. Currently, the Myanmar government has joined with some developed countries: the United States (US Institute of International Education; IIE) and the United Kingdom (British Council), to develop Myanmar teachers’ English proficiency skills (Goodman, 2013), and Japan (Japan International Cooperation Agency; JICA), to update its education system (Ulla, 2017). Some studies have focused on teacher training to improve the skills of educators in Myanmar (Simon, 2013; Ulla, 2017), because most Myanmar teachers depend on more conventional and teacher-centered methods. Therefore, the present study aims to help them improve their skills in teaching English language reading comprehension.

Based on the above framework of the RBRT approach (planning to teach using the reciprocal teaching method; acting as the way of reciprocal teaching procedures: reflecting using students questionnaire and an observation scheme; and evaluating using formative and summative assessment), we have elaborated a developmental program for improving students’ English language reading comprehension.

Research questions

The main objective of the research was to study the effectiveness of the RBRT approach for Myanmar students’ reading comprehension in English. Therefore, we devised the following research questions:

RQ1::

Are the instruments reliable and valid for teaching and measuring students’ achievement in English reading comprehension?

RQ2::

What is the effectiveness of the RBRT approach on students' reading comprehension?

RQ3::

Does the teacher’s reflection on the instructional context have an impact on the students’ reading comprehension achievement?

Methods

Procedures

For this research, we conducted three steps for investigating the effectiveness of RTMRC approach on students’ reading comprehension. First, we confirmed the content validities of the instruments with six content experts. We used the content validity index (CVI) to confirm the appropriateness of the instruments with the theoretical content. Second, we conducted a pilot study with 83 grade 9 students from one Upper Secondary School in Myanmar. Based on the pilot study, we could confirm the construct validity of the instruments for teaching reading comprehension, and begin the procedures of planning and modifying the main study.

The third step is the main study, the investigation of the effectiveness of the RBRT approach with larger sample size. In this step, we selected five sample schools from Myanmar using cluster randomized trial (Table 1 below; Sedgwick, 2014). The intact groups in each school were randomly assigned to the experimental group and the control group. First, we administered a pretest to detect any initial differences between the experimental and the control groups to see if the two groups were essentially the same in their levels of reading comprehension before the treatment. Second, as the treatment, the experimental group participated in the developmental sessions and was taught using the RBRT approach. The developmental period took five weeks and consisted of 25 sessions (45 min each). The control group did not have any special developmental session; these students learned in the traditional way. During the treatment period of five weeks for each experimental group, five English language teachers used the RBRT approach by following the lesson plans provided by the researcher. The students were given related activities with reciprocal teaching involving predicting, questioning, clarifying, and summarizing. After these students’ activities, the teachers revised the text with related questions and exercises to clarify any confusion the students had related to information gained from the text. Then, following Brookfield (2017), the teacher’s reflection was done from two different perspectives, (1) from the point of view of the students, by asking them to fill the questionnaire to describe their learning preferences, and (2) from an observer’s point of view, by requesting the colleagues to observe the teacher’s instructional process in the classroom. For three reading text-units, the student questionnaire was used three times (after the teacher’s text-revisions each time) for the experimental groups during the treatment period but not for the control groups. To improve the reflective action of English language teachers, the observers also used the observation scheme to observe their teaching–learning process three times (randomly during three text-units each) during the intervention period of the experimental group. Third, at the end of the treatment period, both groups completed the posttest.

Table 1 Cluster randomized trial procedures
Full size table

Participants

Based on the above cluster randomized trial, we chose 458 grade 9 students from Sagaing Township, Myanmar. Among them, 255 students are schoolgirls and 203 are schoolboys. We randomly assigned 228 students to the experimental group and 230 students to the control group. In addition, the students’ English language teachers (five English teachers from five chosen schools) participated in the research. These teachers taught the students three reading text-units. The teachers who taught the students in both experimental and control groups were the same in all five schools. While these teachers were teaching the experimental group of students with the RBRT approach, another 10 subject deans/peer colleagues (two per each school) were also involved in this investigation as observers. Control groups were taught in a traditional way, with no RBRT support (i.e., without revised exercises/questions, student questionnaires, or peer observations). In total, in this cluster randomized trial study, the participants were 458 students, five English language teachers, and 10 observers.

Instruments

In this study, we used three main types of measuring instruments (Pre- and posttests, student questionnaires and observation schemes). The content validities of these three instruments were confirmed by the six content experts from the field of English language teaching. The results were also shown in this study to confirm their content validities. The construct validities (based on convergent and discriminant validities) were also supported by this study by analyzing the results in a result session. We also provided detailed lesson plans to the five participating English teachers to assist them in their effective instruction using the RBRT approach.

Pre- and posttests

Pre- and posttests were used to measure the effectiveness of the RBRT approach. We used the same concepts but with different types of tasks in both pre- and posttests. The test questions were based on the content of the Grade-9 English text prescribed by the Ministry of Education, Myanmar. There were a total of 23 items (literal comprehension: seven items; reorganizational comprehension: two items; inferential comprehension: five items; evaluative comprehension: five items; appreciative comprehension: four items). These items were constructed in accordance with Barrett’s taxonomy of reading comprehension levels by the given points in a table of specification (Surtantini, 2019).

Student questionnaire

While giving the treatment with the RBRT approach to the students, the teachers reflected on their instructional context (reader, strategy, text, and task) using the questionnaire completed by the students according to their learning preferences related to the teachers’ instruction. It had a four-point Likert scale (strongly disagree, disagree, agree, strongly agree) adapted from Richards and Lockhart’ (2005) questionnaire. This questionnaire was used for reflecting on the instructional context (inter-related with the students, the teacher’s strategy, the reading text, and the tasks/activities they had students do in the class) as defined by Richards and Lockhart (2005). This questionnaire was already translated into the Burmese language and confirmed by Burmese language experts. There were a total of 17 items (five items for reflection on the ‘reader’ factor, five items for ‘strategy’ factor, 4 items for ‘text’ factor, 3 items for ‘task’ factor) in this questionnaire (see Appendix A).

Observation scheme

To help the English language teachers’ reflection on the instructional context, peer colleagues (ten observers) also observed the teachers’ instructional process using the observation scheme adapted from the one by Richards and Lockhart (2005). The observation scheme had a four point Likert scale (very poor, poor, good, excellent) involving 14 items with the availability of open comments (see Appendix B).

Lesson plan

In this research method, we provided the participating English teachers with detailed lesson plans (how to teach) for following the RBRT approach. For teaching the English reading text (a total of 25 sessions) for the experimental groups, these lesson plans were drawn up in detail based on the above conceptual framework about the RBRT approach. In each lesson plan, it was clearly described how to perform the RBRT approach with the exact time limits for doing the steps of the RBRT approach. These steps needed different time limits based on the lengths of text-units.

Data analysis

According to Gliner et al. (2017), construct validity is based on two types of validity measures: convergent and discriminant. They also mentioned three types of reliability measures for addressing convergent validity (measuring how the theory is related to the practice): (1) internal consistency reliability (Cronbach’s alpha), (2) average variance extracted, and (3) composite reliability. For internal consistency reliability, it is recommended that the Cronbach’s alpha value be > .60 (Gliner et al., 2017). Kline (2015) recommended that the value of the composite reliability (CR) should be > .70. In the case of the average variance extracted (AVE), it should be > .50, according to Afari (2013).

For the discriminant validity of the instruments (measuring how the supposed unrelated theory is unrelated to the practice), we compared the square root of the AVE and the inter-construct correlation in the component correlation matrix of SPSS. Kline (2015) advised that if the value of the square root of the AVE is higher than the values of the inter-construct correlation among the components, its discriminant validity is acceptable.

To compare the experimental and control groups, a t-test was used (independent and paired samples t-tests). Lestari (2016) suggested that the normal distribution of the test should be checked before analyzing it with a t-test. Therefore, the item response theory (Rasch analysis) was used, and we ran the Quest program to determine the estimates for both learners’ ability parameters and the levels of item difficulty. The effect size was measured by Cohen’s d. To quantify the size of experimental effects between independent samples and paired samples, Kotrlik et al. (2011) suggested the use of Cohen’s d (d = .3, small; d = .5, medium, and d = .8, large, according to Cohen, 1988).

Structural equation modeling (SEM) was used to measure the teacher’s reflection on the students’ reading comprehension achievement. We also measured the association between the student questionnaire and the students’ achievement; and the association between the observation scheme and the students’ achievement. The posttest scores were used as the students’ achievement. Regarding the connection between the student questionnaire and the students’ achievement, there were some fit indices to show how well the model fit with the data. The following goodness-of-fit indices were used to justify the model fit: comparative fit index (CFI), goodness-of-fit index (GFI), root mean square error of approximation (RMSEA) (Kline, 2015). The CFI and GFI range from 0 to 1, and larger values confirm a better fit, while values larger than .90 show an acceptable model fit. The RMSEA also indicates the model fit. It also ranges from 0 to 1, but .08 or less shows a good model fit (Kline, 2011).

Results

Q1::

Are the instruments reliable and valid for teaching and measuring students’ reading comprehension?

To answer the above research question, we confirmed the content and construct validities of the instruments in this study. In our study, the pre- and posttests were used to measure students’ reading comprehension achievement, while other instruments (student questionnaire and observation scheme) were used to help teachers teach the students reading comprehension effectively with the RBRT approach.

Content validity is an essential requirement for validating the instrument, and it describes whether the instrument could make an adequate measure of the desired content (Taherdoost, 2016). We used the content validity index (CVI), a widely used method, to measure the content validity of the instruments. Although Polit and Beck (2006) suggested that at least three content experts are needed to evaluate the content validity, we asked for help from the six content experts from the field of English language teaching to examine the content validity for instruments in this study. The CVI for each item was calculated “by counting the number of experts who rated the item as three or four and dividing that number by the total number of experts” (Rubio et al., 2003, p. 97). For the CVI analysis, a four-point scale (not relevant, somewhat relevant, quite relevant and highly relevant) was used by the six content experts to rate the relevance of each item from the instruments (Polit et al., 2007). The CVI value is acceptable if it is .80 and above (Newman et al., 2013). The contents of three instruments for this study were valid after deleting some items (four items from the pre- and posttests, three items from the student questionnaire, and one item from the observation scheme) which were lower than .80 (see in Table 2).

Table 2 Items of the instruments rated by experts for content validity
Full size table

Construct validity, which evaluates the degree to which items in the measuring tool relate to each other, is also measured based on the convergent and the discriminant validities of instruments (Habók & Magyar, 2018). For the convergent validity measures, overall alpha values of all instruments are > .60, except for a few of the factors (inferential and appreciative) values. Almost all values of the CR and the AVE from our study were consistent with their recommended values (Table 3). Therefore, the convergent validity of the instruments of this study was confirmed.

Table 3 Convergent validity of instruments
Full size table

For the discriminant validity of the instruments in our study (Table 4), all values of the square root of the AVE are higher than all the inter-construct values in all the instruments. Therefore, this supports the discriminant validity of the instruments.

Table 4 Discriminant validity measures of instruments
Full size table

Based on the above content and construct validity measures of the instruments, we concluded that all the instruments in this study were reliable and valid for measuring the students’ reading comprehension achievement.

Q2::

What is the effectiveness of the RBRT approach on students' reading comprehension?

To answer this question, it was necessary to compare the student group that was taught to read texts with the RBRT approach and the other student group that was not taught with this approach. Before investigating the effectiveness of the RBRT approach, we employed Rasch analysis. To estimate the ability parameters and item difficulty levels of both groups. The distribution between the students’ achievement and item difficulty levels is shown in Fig. 2.

Fig. 2
figure 2

Person-item map indicating person ability levels and item difficulties on the same scale (each “X” represents 2.8 cases)

Full size image

In Fig. 2, the left side of the graph shows the students’ achievement, and the right side shows the difficulty levels of the items. The higher part of the students’ achievement indicates the students’ higher ability, and the lower part shows the students’ lower ability. The higher part of the item difficulty level indicates the most difficult items, and the lower part shows the easiest items. The graph shows that the students achieved highly in lateral comprehension (items 7, 8, 12, 13, 16, and 17) and inferential comprehension (items 1 and 5) because these appear in the middle part of the graph, which means neither too difficult nor too easy. However, the students had low achievement in reorganizational comprehension (item 23) and appreciative questions (item 21) because these are difficult items that are situated in the higher part of the difficulty level. Some items in the lower part of the graph (3, 2, and 15) describe the students’ evaluative comprehension, and these are the easiest items for the students. In a nutshell, the whole test is neither too difficult nor too easy for the students. Therefore, we can interpret that the test item distribution is normal for evaluating student achievement.

After assessing the item discrimination of the test, we could use a t-test. First, to investigate the initial differences in the experimental and the control groups related to the students’ reading comprehension level before the treatment period with the RBRT approach, both groups completed the pretest. The results are presented in Table 5.

Table 5 Results of pretests of experimental and control groups
Full size table

The data from the pretest were analyzed using the independent samples t-test to analyze the differences between the experimental and the control groups. We could not discover any significant difference between the two groups (p > .05) on the pretest. The maximum given score of the pretest is 45 points. The mean scores of both groups are almost equal (14.80 and 14.93). Therefore, it appears that the levels of the students from these two groups were almost the same before the treatment with the RBRT approach was applied.

After the treatment with the RBRT approach, to study the effectiveness of this approach, it is necessary to test whether there is a statistically significant difference between the experimental and the control groups. The findings are shown in Table 6.

Table 6 Results of posttests of experimental and control groups
Full size table

We analyzed the students’ results using the independent samples t-test to compare the differences between the control and the experimental groups. We found a significant difference (p < .000) between the participants who were taught the reading texts through the RBRT approach and those who were not taught with this approach. The maximum score given for the posttest is 45 points. The experimental group’s mean score (30.58) is significantly higher than that of the control group (26.45). The effect size (Cohen’s d = 0.881) is also high. Therefore, it may be said that the RBRT approach had a considerable impact on the participants’ achievement. On the whole, it can be interpreted that teaching with the RBRT approach is more effective than other traditional teaching methods.

We also compared the results from the pretest and posttest of the experimental group to investigate the effectiveness of the RBRT approach. The findings are shown in Table 7.

Table 7 Results from pretest and posttest of experimental group
Full size table

The data obtained from the pretest and posttest of the experimental group were examined by applying a paired sample t-test to compare the differences between the students’ achievement before and after the treatment with the RBRT approach. Because the p-value is .000*** (*p < 0.05), this indicates a significant difference between the students’ achievement. Based on the mean difference, the posttest mean value (M = 30.58) is higher than that of the pretest (M = 14.80). Therefore, it can be said that the students achieved more as a result of the treatment with the RBRT approach. Regarding effect size, Cohen’s d value is 4.02. This means that teaching with RBRT has a significant effect on student achievement.

Q3::

Does the teacher’s reflection on the instructional context have an impact on the students’ reading comprehension achievement?

We have measured the association between the student questionnaire and the students’ achievement; and the association between the observation scheme and the students’ achievement. The posttest scores were used as indicators of student achievement. Concerning the connection between the student questionnaire and the students’ achievement, there were some fit indices to show how well the model fits with the data (GFI = .94; CFI = 1.00; RMSEA = .06). Regarding the connection between the observation scheme and the students’ achievement, the model was also well fitted (GFI = 1.00; CFI = .95; RMSEA = .02). Although the data were weak and not significant in our pilot-testing, we deleted some items and fitted the model well. Thus, in the case of both the student questionnaire and the observation scheme, the fit indices were well fitted with the recommended values, as shown in Table 8.

Table 8 Model fit measures
Full size table

It was found that the teachers’ reflection on the instructional context (the connection between the student questionnaire and student achievement; and the relationship between the observation scheme and student achievement) is effective for encouraging students’ reading comprehension achievement.

Particularly in the association between the student questionnaire and student achievement, there were some positive and significant impacts related to student achievement by reader reflection and text reflection (β = .60, p < .01; and β = .33, p < .05), whereas there were some negative and nonsignificant impacts on student achievement through strategy reflection and task reflection (β = − .78, p > .05; and β = − .56, p > .05), as shown in Fig. 3.

Fig. 3
figure 3

Connection between student questionnaire and student achievement (n = 684; three times of reflection)

Full size image

Specifically, the relationship between the observation scheme and student achievement suggests that the teacher’s use of an observation scheme has a significant and positive impact on students’ reading comprehension achievement (β = .64; p < .05), as indicated in Fig. 4.

Fig. 4
figure 4

Connection between observation scheme and student achievement (n = 30; three times of observation)

Full size image

Generally, therefore, the teacher’s reflection was significant and had a positive impact on the students’ reading comprehension achievement.

Results of the reflection on instructional context with the student questionnaire

To highlight the effectiveness of the teachers’ reflection on students’ reading comprehension achievement, the detailed results of the student questionnaire and observation scheme are also shown. In the present study, we used the student questionnaire three times for the teachers to reflect on the instructional context, more precisely involving reader (five items), strategy (five items), text (four items), and task (three items). Based on the students’ responses through using a student questionnaire, the teachers asked for feedback from the students and considered these in their instructional planning (Fig. 5).

Fig. 5
figure 5

Results from student questionnaire (n = 684; three times of reflection)

Full size image

The teachers reflected on their instructional context (reader, strategy, task, and text) based on students preferences. The results are as follows:

Reflection on reader

Most of the students like learning English by collaborating with others. They prefer conversations to discuss their English learning, and they even prefer discussions with peers about unfamiliar words instead of looking up words in the vocabulary or asking for external assistance. It was found that some students, due to cultural influence, feel ashamed of themselves in their individual work. Therefore, the teachers encouraged their involvement in learning and in collaborative discussions, and created a comfortable and interesting learning environment to facilitate their cooperation.

Reflection on strategy

The teachers who teach reading comprehension using the reciprocal method must take care to balance their active involvement so that their role does not become overemphasized. Some students’ responses on this questionnaire showed that the teachers’ voice remained low. Poor classroom management could lead to a noisy and uncontrollable environment for students. Generally, it was found that the teachers’ strategy use was appropriate, with almost all the students preferring it. The results suggested that for students, the most distinct benefit of the reciprocal teaching strategy is that it is helpful for recalling their vocabulary.

Reflection on text

Most of the students were proficient at doing reading comprehension exercises from the text. However, a few students could not do these exercises successfully. Teachers should therefore consider ways to improve students’ understanding. During the developmental sessions, the teachers gave some revision/reflective exercises and most of the students performed these well. However, it was also found that the students understand the text better if the teachers explain it after their role-play. According to the students’ responses, the teachers’ comments and explanations are helpful. For a complete understanding of the text, students need more time. Therefore, the teachers need to consider time management when employing the reciprocal teaching method.

Reflection on task

When reciprocal teaching is employed in the classroom, students have to play the roles of questioner, clarifier, summarizer, and predictor, showing competence in each. In this study, the students all preferred these role-plays. They also appreciated the reading comprehension tasks, that is, reflective exercises for reading comprehension (literal, reorganizational, inferential, evaluative, and appreciative tasks). However, it was found as well that some students had difficulty responding to certain reading comprehension tasks. In the reciprocal teaching method, a task does not depend on the students’ tasks alone. For example, in the role of “clarifier,” students must clarify the questioner’s questions. During this part of the exercise, the teachers should help students with their tasks to ensure their clear comprehension if the “clarifier” cannot explain something well. The study was also found that some students desire the teachers’ support and a clear explanation.

Based on the results of the first reflection, the teachers addressed their own weaknesses and tried to improve their teaching. Therefore, some improvement is evident in the second and third reflections (in Fig. 5) on their instruction.

Results of the reflection of instructional context by the observation scheme

In the “reflecting step” of the RBRT approach, the teachers also asked for help from 10 peers to observe their reading comprehension instruction. The peer observers observed and evaluated the developmental sessions three times during the treatment period when the RBRT approach was used. Table 9 shows the evaluations of the developmental sessions.

Table 9 Results of peer observations (n = 30, three times of ten peers)
Full size table

From the results of the observation scheme, it was found that teachers were weak in selecting appropriate materials for teaching. They need to prepare some materials, for example, worksheets, practice profiles, a student participation checklist, and so on. If possible, they should clarify the reciprocal teaching method for the students through PowerPoint slides/files. All peer observers agreed that the teachers could give a clear explanation and offer appropriate activities. They mentioned that this strategy can enhance students’ reading comprehension skills and improve communication skills. This is the best way to encourage peer interaction. This strategy is good for quiet students’ communication needs. Moreover, the peer observers suggested that if the teachers plan well, this is a very good strategy for improving students’ reading comprehension.

However, some of the observers suggested the teachers need to organize the class well to use this approach. They mentioned that teachers should focus on their interaction with the pupils. In employing the RBRT approach, teachers should not assume that only the students must do these activities—questioning, clarifying, summarizing, and predicting. They should interact with the students and help them as necessary.

Based on the observers’ suggestions, the teachers saw their weaknesses in the first observation and corrected them and planned for better instruction in later periods. Therefore, some improvements can be seen in later sessions (Table 9).

Discussion and conclusion

In this study, we investigated three research questions. In testing the first question concerning the reliability and validity of the instruments (pre- and posttests, student questionnaire, and observation scheme), the overall content and construct validities of instruments were acceptable, although there were some weak values of internal consistency reliability and AVE values. These content and construct validities of our instruments were sufficient to inquire into the effect of the RBRT approach on students’ reading comprehension achievement. These instruments were also pilot-tested three months earlier, and detailed lesson plans were drawn up for the intervention with this approach.

In testing the second research question regarding the effectiveness of the RBRT approach for students’ reading comprehension achievement, we first tested the normal distribution of the tests through ConQuest analysis of the Rasch model. It was found that the tests were in normal distribution. After the normality measures, we evaluated the effectiveness of the RBRT approach through analysis with a t-test and effect size (Cohen’s d). We found that the RBRT approach is more effective than traditional teaching methods in teaching reading comprehension. It was also found that the students showed high achievement in literal and inferential comprehension skills.

In testing the third research question concerning the teachers’ reflection on the students’ achievement, we analyzed the data from the student questionnaire and observation scheme using SEM. When we checked the connection between the student questionnaire and the students’ achievement, we found that two factors (teachers’ reflection on reader and reflection on text) had a significant impact on student achievement, although there were some negative impacts on the teachers’ reflection on strategy and reflection on task. The teachers’ reflection by the use of the observation scheme had a significant influence on the students’ achievement. Therefore, it can be interpreted that the teachers’ reflection on the instructional context is effective in raising students’ reading comprehension achievement. To highlight the importance of the teachers’ reflection for teaching reading comprehension, the results of these two instruments (student questionnaire and observation scheme) were also described in detail in order for the teachers to improve their instructional planning.

In investigating the effect of the RBRT approach on students’ reading comprehension achievement, not only did the teachers reflect on their instructional context, but the students also reflected on their learning and expressed their opinions to their teacher. The teachers’ reflections were aimed at having a good sense of the instructional context (involving strategy, reader, text, and task), whereas the students reflected on their learning to achieve greater understanding. Therefore, the RBRT approach can provide many benefits to not only teachers but also students.

We stated that students’ vocabulary knowledge is significantly increased in the field of reading comprehension. The study showed that students’ literal and inferential comprehension skills are highly significant in student achievement. A previous study (Mandel et al., 2013) also indicated that students’ vocabulary knowledge can become more developed during the reading comprehension process. This is a shared similarity between the reciprocal teaching method and the RBRT approach. There are some differences between them as well. In the reciprocal teaching method without reflection, Egiyantinah et al. (2018) found that the reciprocal teaching method is effective for improving reading comprehension; however, students’ learning styles should be considered in order to gain more effectiveness. Regarding the RBRT approach, the teachers not only used reciprocal teaching alone but also reflected on their instructional context. Therefore, they could understand what the learners’ preferences and their learning styles are, how effective their teaching strategy is, how well students understand the text, and how they feel about different learning activities.

The RBRT approach involves two main functions: reflection using Oo and Habók’s (2020) reflective teaching model for reading comprehension, and the reciprocal teaching method. Therefore, this combined approach can provide the teachers and students with many benefits of both functions.

Reflection gave the teachers the opportunity to analyze how and why the classroom situation was as it was (Rico et al., 2010). In the current reflective teaching process, the teachers also had a chance to assess their teaching method, what the students’ learning preferences were, how the text and the activities were proceeding, and how they could improve their approaches to teaching based on their reflective analysis. Some examples are the following: (1) Based on one item on the student questionnaire (“I like the English teacher using the relevant questions while teaching the reading text.”) as student feedback, the teachers could emphasize giving the students the related reflective questions on the text to help them better understand the text; (2) Another item (“When I don’t understand something while reading the English text, I like to guess the meaning by connecting with other related words.”) reveals that teachers should know whether their students greatly depend on the teachers/other students or not; and (3) this item reflection (“I do better at reading in English when I work with others”) helped the teachers improve the students’ collaborative work by monitoring and guiding them to work together for better comprehension. Furthermore, according to the reflection by peer observers, the teachers could improve their instruction by planning appropriate activities and materials; effectively organizing the classroom presentation, interacting with the students, giving the students opportunities for using their prior knowledge to relate to their knowledge of the current text for greater understanding, and helping them develop their communication skills.

To conclude, this study confirmed that there was a statistically significant difference between the experimental and control groups. We confirm that the RBRT approach is more effective than other traditional strategies in teaching reading comprehension. Actually, as reflection can promote transformative learning, Oo and Habók’s (2020) reflective teaching model is shown to be useful for ELT teachers. As for future research in teaching reading comprehension, any kind of teaching methods can be used in the framework of a reflective teaching model and allow teachers to examine their effectiveness. Classroom research about ELT in Myanmar is scant (Tin, 2014), and thus, this classroom research using the RBRT approach is useful in improving the ELT teaching learning process in Myanmar. In the Myanmar context, teacher educators and decision makers of the government soundly believe that English language teaching and its trainings are regarded as one main pillar of promoting the national education system (Soe, 2015). Therefore, the RBRT approach can provide many benefits to both teachers and their students in ELT. There were also some limitations to the research. (1) For SEM analysis, more participants are needed. (2) The intervention period could be planned for a longer time span and could be complemented by a follow-up study examining the long-term effect of the program. (3) Some other reflective tools (for example, a portfolio, students’ open feedback, teacher’s diary, and so on) can also be used for the teacher’s work, depending on the teaching–learning situation. (4) Information and communications technology (ICT) could be employed in the developmental sessions, because in the present research, teachers could not use ICT tools due to the lack of infrastructural background. In addition if the observers could observe the teacher's instructional process more frequently (more than three times), this would be of great help to the teacher’s reflections when teaching with the RBRT approach.