Task complexity, language proficiency and working memory: Interaction effects on second language speech performance

2.1 Task complexity

There is little disagreement among TBLT researchers that TC is a complex and multidimensional construct (Robinson 2007; Sasayama 2016; Skehan 1998, Skehan 2014; Vasylets, Gilabert & Manchon, 2017) interacting not only with task material, design and mode, but with learner cognition and individual differences (Révész 2011; Robinson 2011). Liu and Li (2012) argued that, when defining TC broadly, researchers refer to three distinct qualities of a task: a) its structure (e. g. number of elements and characters), b) its resource requirements (e. g. what is needed to perform the task), and c) interaction between a task and learner variables (e. g. cognitions and WM). Many TBLT researchers, however, have sought a more detailed and theoretically supported framework for defining TC, and situate their studies with reference to two influential cognitive-interactionist TC models, i. e. Limited Attentional Capacity (Skehan 1998, Skehan 2015) and the Cognition Hypothesis (Robinson 2007, Robinson 2015).

Drawing on a multiple-resource model of attention, the Cognition Hypothesis proposes that the human brain has access to a pool of multiple resources, and that therefore increasing TC encourages access to multiple resources promoting more complex and accurate language production. It also theorizes that this higher cognitive demand both creates appropriate opportunities for learning and facilitates L2 acquisition. By contrast, Skehan’s Limited Attentional Capacity model (2015) highlights the limited nature of the learner’s processing capacity, and assumes that a higher cognitive demand requires greater attentional resources, thus forcing learners to prioritise their allocation of attention. This often results in a competition between different performance dimensions, especially between form (i. e. complexity and accuracy) and meaning (i. e. fluency), or between different aspects of form. The latter model envisages that the pressure resulting from high cognitive load will limit opportunities for development and acquisition. While we draw on this body of literature in our research overview, we do not aim to match our findings against either model. Instead, we aim to discuss TC in a broader perspective of individual learner variables. More central to our study is the fact that TBLT research has so far repeatedly examined variables such as Here-and-Now versus There-and-Then, task structure and planning time, without sufficiently examining other aspects of TC. IR is one such TC variable.

2.2 Intentional reasoning

Robinson (2007, 2015) proposes that tasks that require describing motion events (spatial reasoning), explaining reasons for actions (causal reasoning), and reading other peoples’ minds (intentional reasoning) have high IR demands, and therefore lead learners’ attention to using more accurate and complex language to convey such demands. Robinson (2007) argues that a task that requires IR encourages L2 learners to adopt complex linguistic structures (e. g. subordinating conjunctions), to create cohesion between intentions, actions and predictions, and to use a lexis with higher complexity (e. g. mental states verbs, adverbs of uncertainty) that allows these intentions to be described. On a less positive side, however, IR is expected to reduce fluency.

To the best of our knowledge, there are only two studies that have examined the impact of IR on speech performance. Robinson (2007) used a continuum of IR through three picture sequencing and picture telling tasks performed by 42 Japanese students. IR was operationalised by the demand to explain the intentions of each story character. The simplest task entailed explaining one character’s IR, whereas the more complex tasks involved more characters whose intentions were reliant on others’ ideas and desires. The findings were not in line with the predictions of the Cognition Hypothesis as increasing IR demands did not elicit more complex and accurate language or lead to reduced fluency. Ishikawa (2008) was the second study to investigate the effect of manipulating TC through IR demands. In this study, IR was operationalised as the demand to explain changes in human relationships at workplace. The 24 participants performed three tasks that required them to report changes in relationships between staff, based on several trouble triggers. Three levels of no reasoning, simple reasoning, and complex reasoning were designed. Similar to Robinson’s (2007) study, the extra IR demand was operationalised by Ishikawa (2008) as the number of characters involved in the tasks, i. e. the complex versions had more characters and hence needed more reasoning. The no reasoning task required only describing current relationships between the characters, the simple reasoning entailed explaining changes in relationships of two characters, and the more complex reasoning task involved four characters whose intentions were reliant on the others’ ideas and desires. Supporting the CH predictions, the findings showed that higher IR demands produced higher complexity and higher accuracy, whereas fluency performance decreased. Notwithstanding the value of the findings of these studies, a methodological limitation in their design makes it difficult to interpret the results. In both studies, IR seems to conflate the number of characters/elements in a task and the amount of reasoning required, i. e. the more complex task has more characters, concepts and elements and as such needs more IR for each character. The interdependence of these aspects of task design is an important issue that researchers such as de Jong and Vercelloti (2015) and Sasayama (2016) have warned fellow researchers against. There are at least two ways of manipulating TC regarding IR demands to avoid the interdependence issue. First, TC can be operationalized in terms of the number of elements in the task with the more complex task having more elements. Second, TC can be operationalized in terms of the amount and level of reasoning required to complete the task. The current study focuses on the latter option.

Drawing on the Cognitive Psychology literature, Awwad et al. (2017) consider IR as “a critical element involved in a) observing others’ actions and behaviours, and b) arriving at conclusions about others’ thoughts, intentions and beliefs” (Awwad et al. 2017: 161). IR, which involves hypothesizing, interpreting and drawing conclusions about others’ thoughts, actions and behaviours, is a serial cognitive process (Leighton 2004) dependent on a chain of logical premises and hypotheses (Gilhooly 2004). The serial processes and semantic operations that are required to create this logical chain add to the cognitive load by placing an extra burden on attention and working memory, especially while generating idea units and constructing an appropriate preverbal message (Levelt 1989). From a linguistic point of view, it can be argued that describing these thoughts and intentions, as well as explaining and justifying them, would invite use of specific language that denotes intentionality and reasoning. We have argued (Awwad et al. 2017) that representing IR in the English language is expected to encourage syntactically more complex structures (e. g. logical subordinators), and lexically more complex words (e. g. cognitive status verbs). The use of hypothetical language of a formulaic nature (e. g. I think and I suppose) would also promote accuracy, at least at the level of short clauses. Testing the aforementioned predictions, our results (Awwad et al. 2017) of examining speech performance of L2 learners on tasks with different degrees of IR, showed that speech performance in the +IR condition was associated with syntactically more complex and accurate but, surprisingly, less lexically diverse language. No statistically significant differences were observed in fluency of the learners’ performances across the two conditions. One way to explain the unanticipated results regarding lexical diversity and fluency, was to hypothesize that TC had interacted with the learners’ individual differences in LP and WM (see Awwad et al. 2017 for further details). This post-hoc observation was the point of departure for the current study.

2.3 Language proficiency and task complexity

Investigating a complex and abstract process such as TC becomes more intricate when learners are performing tasks in a language in which they are not proficient. LP, i. e. “the linguistic knowledge and skills that underlie L2 learners’ successful comprehension and production of the target language” (Gaillard and Tremblay 2016, p.420), is assumed to play a major role in L2 processing that depends on conscious and controlled attention (Kormos 2011), and it can either drive or hinder language performance by the amount of automaticity available for language encoding and attention allocation during performance. Research on the interaction between TC and LP attempts to explain whether variations in LP lead to variations in allocating attention, controlling learner interlanguage, and monitoring speech production. By maintaining a smooth flow of linguistic resources during performance, higher levels of LP are argued to support L2 processing by assisting learners to engage in parallel processing, and freeing up attentional resources to attend to different aspects of performance (Kormos 2011). Therefore, an interaction between LP and TC is expected in that high-proficiency learners are likely to operate more successfully while processing and performing cognitively demanding L2 tasks.

Given TC as a research focus, only a small number of studies have investigated the interaction between LP and TC (e. g. Ishikawa 2006; Kuiken and Vedder 2008; Malicka and Levkina 2012). Employing X-lex and Y-lex vocabulary size tests and the Oxford Placement Test to measure the participants’ LP, Malicka and Levkina (2012) investigated whether LP regulated the effects of reasoning demands and the number of elements in instruction-giving tasks. Their results suggested that the high-proficiency group produced more complex and accurate performance on the complex tasks, whereas the low-proficiency group produced more fluent language. Investigating the interaction between LP and +/– Here and Now on L2 learners’ narrative writing, Ishikawa (2006) found that TC influenced all aspects of performance except for lexis. Using a Cloze test to assess LP, Kuiken and Vedder (2008) examined the interaction between the number of elements in a task and LP on L2 learners’ writing performance. Although their findings revealed major effects of LP on grammatical complexity, accuracy and lexis, no interaction effects were found between +/– Here and Now and LP on any aspect of writing performance. Two conclusions can be reached based on the mixed findings reported above. First, it is possible to link the mixed results with the different aspects of TC, task modes and types used in these studies. Another way to interpret these results is to highlight the lack of consistency in assessing LP in these studies. In the current study, we are keen to investigate LP from a broader perspective by assessing proficiency in terms of both explicit and implicit knowledge. SLA research (DeKeyser 2003, DeKeyser 2009; Ellis 2009; Hulstijn 2005) has postulated that language proficiency is comprised of two different underlying constructs, i. e. implicit and explicit knowledge, characterised by presence or absence of conscious awareness. Drawing on unconscious and intuitive knowledge, implicit knowledge is procedural in nature and results in fluent speech production. Explicit knowledge, on the other hand, draws on declarative knowledge and is mainly acquired through conscious awareness, and therefore assumed to result in controlled processing (DeKeyser 2003; Hulstijn 2005). Despite the significance of these two types of knowledge, most TC studies have examined proficiency in the explicit type only (e. g. Declerck and Kormos 2012; Malicka and Levkina 2012). Given that previous research in this area has failed to examine the differential contributions of explicit and implicit knowledge to task performance, the current study aims to explore the interaction between TC and LP in relation to both implicit and explicit knowledge.

2.4 Working memory and task complexity

The second learner-internal factor we are examining here is WM. WM, “a multi-component system which is responsible for active maintenance of information in face of ongoing processes and/or distraction” (Conway et al., 2005, p. 770), is envisaged to interact with L2 performance and development in general and with TC in particular (Cho 2018; Gilabert and Muñoz 2010; Kormos and Trebits 2011; Mitchell et al. 2015; Mota 2003). The tendency to incorporate WM in TC studies stems from the notion that WM is at stake in the performance of L2 complex tasks (Kyllonen and Christal 1990) for its potential influence on regulating L2 learners’ linguistic repertoire and attentional resources during language performance (Wen et al. 2015). While several studies have investigated the relationship between WM and L2 acquisition, not many have examined the relationship between TC, LP and WM. In a correlational study, Gilabert and Munoz (2010) explored whether variation in WM and LP would explain variation in L2 performance. A reading span test measured the participants’ WM, while three tests assessed LP, i. e. the Oxford Placement Test, vocabulary size tasks, and a phonetic classification task. The 59 participants were allocated to low-high LP groups, and performed only one video-based narrative task. Though Gilabert and Munoz did not find any correlation between WM and LP, they did find that WM correlated with lexical complexity and fluency. Moreover, LP was found to correlate with all aspects of speech performance except syntactic complexity. LP was found to be a reliable predictor of lexis, whereas WM was not.

Kormos and Trebits (2011) investigated the relationship between WM and TC on L2 oral performance. A backward-digit span task measured the participants’ WM. The participants performed two narrative tasks with varying TC, i. e. telling a story (simple) and inventing a story (complex). It was found that high WM benefited syntactic complexity only in the simple task in terms of ratio of subordination and length of clause. The complex task elicited performances of more accuracy and less lexical complexity, but no effect was observed for grammatical complexity or fluency. It is necessary to mention that this study did not employ a standardised LP test, but instead relied on teachers’ judgments of student LP. Using a standardised LP test, Mitchell et al. (2015) investigated the LP-WM interaction effects on L2 processing and development. Based on their TOFEL scores, 36 Chinese learners of English were grouped into three levels of proficiency (beginner, intermediate, advanced). Their WM was measured by an operation span task and forward-digit span tasks in L1 and L2. To measure the participants’ proficiency, the authors used elicited imitation and reading tasks in English. While the findings did not show any correlations between LP scores and L1 digit span and operation span scores, LP did correlate with the L2 digit span scores. The study found a stronger relationship between WM and LP in the case of high proficiency learners.

4.1 Participants

The participants were 48 students at a private secondary school in Jordan. Since the school was a single-sex school, all the participants were males. They were aged 16, with Arabic as their first language. The demographic data showed they had very similar schooling and language instruction experiences, i. e. they had studied English for about ten years at school, and had never lived in an English-speaking country. The school where data were collected teaches the national Jordanian syllabus providing one English lesson every day using textbooks that corresponded to different CEFR levels. CEFR (2001) using an internal placement test, the school had grouped the students into three levels (A, B, C) according to their English proficiency level. The students were then assessed throughout the academic year on the school internal tests and portfolios of continuous assessments. Before the data collection, we sent out an invitation to all students in year X in this school asking for volunteers to participate in the study. The data were collected from 52 participants, but due to a technical problem with the audio recording, we had to remove data from four of the participants.

4.2 Language proficiency test

Despite the abundant research interest in investigating spoken proficiency as a key construct of L2 ability, there is some evidence to suggests that operationalisation, measurement and analysis of spoken proficiency is not always done carefully and systematically, inevitably resulting in poor test reliability and/or validity (Bachman 1990; Fulcher 2014). This limitation has, for instance, been reflected in studies that use a pen-and-paper test to represent speaking proficiency, or when a multiple-choice grammar test is used to investigate learner communicative adequacy. In line with this debate, Leal (2018) argues that measurement and analysis of proficiency in such a limited manner will have inadvertent consequences for research findings both theoretically and empirically. To prevent such negative effects, Leal (2018) proposes that researchers should consider and analyse proficiency in its full sense and as a continuous variable to show variance among the participants.

As discussed earlier, the limited approach to assessing proficiency in previous studies that examined TC across different levels of LP is a source of ambiguity in understanding and interpreting the results of the studies summarized above. To make up for such limitations and to develop a more in-depth insight into the effects of LP on TC, two tests were used to investigate explicit and implicit knowledge. The Oxford Placement Test (OPT) (Alan, 2004) is assumed to measure learners’ L2 explicit knowledge, and elicited imitation tasks (EIT) (Wu and Ortega 2013) are supposed to assess their implicit knowledge (Ellis 2009; Erlam 2006). We used a pen-and-paper version of the OPT containing 60 multiple-choice questions to assess their explicit knowledge (a maximum score of 60). The version of EIT used in the study comprised ten sentences increasing in number of syllables from 8 to 19 (Wu and Ortega 2013; Yan et al. 2015). This version was chosen because previous research has provided evidence that as sentences gradually increase in length, test takers feel under pressure to access their interlanguage to produce the sentences. As the increase in task demands encourages use of implicit knowledge, the process allows researchers to learn more about how L2 learners’ implicit linguistic knowledge is activated (Erlam 2006; Wu and Ortega 2013; Yan et al. 2015). Based on the accuracy of the imitation, each sentence was given 0–4 points with 40 points as the maximum score. As for OPT, since it was a multiple-choice test, a second marker was employed to cross-check the accuracy of the marking and no disagreement was found. As for EIT, a second rater checked the accuracy of the rating. Pearson correlation coefficient of 92% was achieved between the researchers and the rater.

4.3 Working memory test

SLA research suggests that backward-digit span tests are appropriate tools to measure L2 learners’ WM as they are language independent and therefore minimise any impact of proficiency on WM scores (Harrington and Sawyer 1992), and allow for both storage and processing to be examined (Kormos and Trebits 2011; Mitchel et al. 2015). In this study, we used backward-digit span tests in both L1 (Arabic) and L2 (English) to cross-check that they were language independent. Designed by the researchers, the WM tests comprised seven sets of increasing numbers, i. e. 3–9. The sets were audio recorded by one of the researchers at one digit per second. The participants were required to listen to these sets and repeat them backwards. The Arabic and English versions were counterbalanced between participants. The participants were given three attempts for each set. Each participant’s WM span was determined based on the last set of digits he repeated successfully twice. That is if a participant failed to repeat two sets out of three of the same span, his WM span would be the last set he repeated successfully twice. The participants’ WM span scores in the two tests ranged between 4 and 9. Considering the strong correlation between L1 and L2 WM tests in this study (r=0.87, p=0.001), the L1 test scores are used in the analysis. The descriptive statistics for OPT, EIT and WM scores are summarised in Table 1 below.

4.4 The video tasks

For comparability purposes, the same tasks as those in our previous study were used (Awwad et al. 2017). The video-based tasks were adopted from Pat & Mat (Beneš and Jiránek 1976), an animated cartoon series about two friends who deal with everyday challenges and troubles in optimistic, creative and funny ways. de Jong and Vercelloti’s (2015) framework was used to choose the video clips. Based on this framework, a number of factors were carefully considered to ensure that the two clips were similar with respect to the number of characters and elements, duration and storyline. However, the two clips differed in terms of the actions involved and the amount of IR needed to justify the characters’ actions. Besides controlling for IR at content level, we operationalized it at task instruction level. While the instructions in the –IR task asked the participants to tell and describe the story, the +IR task encouraged the participants not only to tell and describe the events, but also to read the characters’ thoughts and intentions and to predict and explain their decisions, actions and reactions. The duration of each video clip was 120 seconds. The choice of tasks was validated through a retrospective questionnaire in which the participants rated the –IR and +IR tasks in terms of the degree of difficulty on a four-point scale. The participants significantly rated the +IR task as more difficult compared to the –IR task (t=−7.43, p=0.00, d=−1.52) suggesting that the task that required more IR was assumed to be more complex as it was designed to be (for further details see Awwad et al. 2017).

The participants met with one of the researchers in a quiet room. To avoid test fatigue, they took the LP and WM tests in one room and performed the tasks in another. Pre-task planning time was not provided to ensure that the relationship between TC, WM and LP was not mediated by strategic planning. However, a demo video clip (30 seconds) was shown to the participants to familiarize them with the task type. The researcher read the instructions in the participants’ L1 and L2 and then asked them to narrate the story to him in their L2, i. e. English. A digital voice recorder with a headphone was used to record the participants’ performances. At the end of the video clips, each participant was given an extra 20 seconds to finish their performance if needed.

6.1 Effects of IR on L2 speech performance

The results of the t-test for syntactic complexity showed that in the +IR task participants produced statistically longer AS-units (t=5.21, p=0.000, d=0.69) a higher ratio of subordination (t=−6.72, p=0.000, d=1.09) with a large effect size. Regarding mean length of clauses, although performances in the –IR task generated longer clauses (M=5.21, SD=0.54) than the +IR task (M=5.12, SD=0.43), the difference did not reach a significant level. For lexical complexity, the findings revealed that performance in the +IR task was lexically less diverse when compared to the –IR task with a statistically significant result and a small effect size (t=2.11, p=0.40, d=0.20). Finally, language performance in the +IR task was characterized by less lexical sophistication measured by PLex Lambda, reaching a significant difference level and a large effect size (t=6.17, p=0.000, d=1.09).

Regarding accuracy, performance in the +IR task elicited a higher percentage of error free clauses with a medium effect size (t=−8.64, p=0.000, d=0.88) and a higher ratio of weighted clauses (t=−5.76, p=0.000, d=0.85). As for fluency, the +IR task elicited a significantly higher speech rate with a small effect size (t=5.54, p=0.000, d=0.45). Regarding mean length of silent pauses, although performances in the –IR task generated longer pauses mid-clause (M=0.94, SD=0.34) and end-clause (M=1.30, SD=0.49) than in the +IR task (M=0.85, SD=0.28), (M=1.20, SD=0.50), the differences did not reach significant levels. Number of repairs showed that the participants produced more repairs while performing the +IR task (M=9.85, SD=4.93) compared to the –IR task (M=8.92, SD=4.05), but this difference was not significant.

6.2 Language proficiency and working memory predicting language performance

Research Question 2 asked whether LP and WM were reliable predictors of L2 speech performance on tasks of varying levels of TC. Multiple regression analyses were performed with LP and WM as predictor factors, where composite measures of syntactic complexity, lexical complexity, accuracy and speed fluency were employed as dependent variables. Separate analyses were run for +IR and –IR task performances.

The non-significant correlation between WM and explicit L2 knowledge (r=0.028, p=0.426) and implicit L2 knowledge (r=0.016, p=0.457) indicated that the two predictors of LP and WM tapped into different aspects of language performance, and therefore were assumed to be suitable for inclusion in the regression analysis. Rather predictably, a significant correlation was found between explicit and implicit L2 knowledge (r=0.65, p=0.001).

As shown in Table 3, the results regarding whether LP and WM predict performance in the –IR task revealed that the regression model was not significant for syntactic complexity (F (3, 44)=0.884, p=0.591), suggesting that the syntactic complexity of learners’ performance could not be explained by variations in their explicit and implicit LP or WM. As for lexical complexity, the non-significant regression (F (3, 44)=0.820, p=0.490) also indicated that neither explicit and implicit LP nor WM could be considered as reliable predictors of learners’ lexical complexity.

The regression model for accuracy reached a significant level (F (3, 44)=3.72, p=0.018), explaining 20% of the variance. All three predictors, i. e. explicit LP (p=0.005), implicit LP (p=0.016), and WM (p=0.047) contributed significantly to the model. For speed fluency, although the regression model failed to reach a statistically significance level (F (3, 44)=2.29, p=0.091), a 14% of the variance in this performance was explained by explicit and implicit LP knowledge (p=0.011 and p=0.036 respectively). WM did not make a contribution to this model (p=0.141). In sum, the results of the multiple regression analyses regarding –IR performance showed that only the model for accuracy was statistically significant, whereas the model for speed fluency showed signs of approaching a statistically significant level. As for the individual contributions, only explicit and implicit LP contributed significantly to both models, whereas WM contribution reached a significant level for only the accuracy model. These results suggested that LP and WM were reliable predictors of accuracy in terms of L2 performance in the –IR task, whereas LP could to some extent predict speed fluency only as well.

Turning to L2 performance in the more complex task (+IR), the regression analyses (see Table 4) showed that the model for syntactic complexity (F (3, 44)=0.856, p=0.471) failed to reach a statistically significant level, implying that LP and WM could not predict syntactic complexity of the learners’ performance in the more complex task. The regression model for lexical complexity reached a statistically significant level (F (3, 44)=3.70, p=0.019), explaining 20% of the variance with explicit knowledge (p=0.026) and WM (p=0.011) making significant contributions to the model. Implicit knowledge, however, did not make a contribution (p=0.325) to this model.

As for accuracy, the regression model was statistically significant (F (3, 44)=2.97, p=0.042), explaining 17% of the variance. WM (p=0.014), explicit knowledge (p=0.05), and implicit knowledge (p=0.05) made significant contributions to the accuracy model. Regarding speed fluency, the regression model was not statistically significant (F (3, 44)=1.17, p=0.331), revealing that speed fluency of the learners’ performance could not be predicted by variations in their explicit or implicit LP or WM in this task.

To sum up, the results of the multiple regression analyses regarding +IR performance showed that the models for lexical complexity and accuracy were statistically significant, whereas the models for syntactic complexity and speed fluency failed to reach a statistically significant level. As for the individual contributions, WM and explicit L2 knowledge had significant contributions to both models of lexical complexity and accuracy. For lexical complexity, however, implicit L2 knowledge did not appear to make a significant contribution.

We conducted a post-hoc power analysis using G*Power 3.1 (Faul et al. 2009) to examine the power of our t-tests. Although the significant results and effect sizes achieved in the study underline the effects of IR on task performance, the results of power analysis can reassure us about the strength of the findings. The power to detect a medium effect size of 0.5 (Plonsky and Oswald 2014) was determined to be 0.96, and critical t (47)=1.68. Running the analysis for a linear regression fixed model, we calculated the power of each individually significant regression model with an alpha level of 0.05 and a sample size of 48. The results showed a power of 0.78 for accuracy, and 0.85 for lexical complexity. The results of the power analysis suggest that although a reliable level of confidence could be maintained in the findings, the results should be interpreted with care.