Introduction

Dual-task situations are common in modern life; we listen to music while we write research papers and we talk while we drive. Although many individuals report minimal difficulty in coordinating two tasks, laboratory tasks reveal that performing two tasks versus just one usually leads to significant performance impairments, termed dual-task costs (for review, see Koch, Poljac, Müller, & Kiesel, 2018). These costs manifest as increases in response time (RT) and error rate (ER) and are robust across a wide range of task pairings (Pashler, 1994).

The source of dual-task costs

Our limited ability to perform multiple tasks simultaneously is a puzzle given the impressive capacity and parallel processing of the brain. Does this inability reflect a limitation in how much neural processing can be devoted to a specific task? Or does it reflect interference between concurrently active neural pathways? From a theoretical perspective, these are two broad classes of accounts that describe our limitations in coordinating two tasks: resource theories and crosstalk.

Resource theories propose that when multiple tasks are performed at the same time, performance suffers when they require the same resources. If a domain-general resource that is shared by all tasks is assumed, then at least one of the tasks must be slowed when operations temporally overlap, although various versions of this account differ regarding whether the resource is shared in an all-or-none (e.g., Pashler, 1984) or graded (e.g., Navon & Miller, 2002; Tombu & Jolicoeur, 2003) fashion. A variant of this account posits that there are multiple, domain-specific resources, but there is no consensus regarding the composition of the set of domain-specific resources (Navon, 1984), making rigorous predictions about patterns of dual-task costs difficult to generate. One proposal is that central resources limiting dual-task performance are specific to combinations of stimulus and response modalities and the central codes (e.g., spatial, verbal) linking them (e.g., Wickens, 1980; Wickens, Sandry, & Vidulich, 1983). When tasks draw from distinct resources, dual-task costs are reduced relative to when they draw from a common resource. For instance, dual-task costs are smaller when a visual-manual (VM) task is paired with an auditory-vocal (AV) task than when the opposite arrangement of stimuli and responses (VV/AM) is used (Göthe, Oberauer, & Kliegl, 2016; Hazeltine, Ruthruff, & Remington, 2006; Stelzel, Schumacher, Schubert, & D’Esposito, 2006). In this case, when performing VM/AV tasks, each task can draw upon distinct resources (VM: visuospatial; AV: verbal/sound). In contrast, when performing VV/AM tasks, the two tasks draw upon similar resources (visuospatial and verbal/sound codes for both tasks), leading to larger costs.

Crosstalk accounts, on the other hand, hold that dual-task costs stem from interference between the representations engaged by central operations during performance (e.g., Hazeltine et al., 2006; Logan & Gordon, 2001; Navon & Miller, 1987). For example, when both tasks involve visuospatial information (e.g., Task 1: visual stimulus; Task 2: manual response), the central operations (i.e., the cognitive processes linking perception and action) activated by the visual stimulus of one task interferes with the central operations activated by the manual response of the competing task, thereby increasing dual-task costs. In contrast, when central operations operate on stimulus and response information that is distinct across the two tasks (e.g., VM: visuospatial; AV: sound), costs are reduced (see Halvorson & Hazeltine, 2015).

Rationale for the present study

One appealing feature of the multiple resource and crosstalk accounts is that they provide straightforward explanations for modality pairing effects on dual-task costs: when two tasks overlap along some dimension (i.e., share a representational medium), costs are larger. However, adjudicating between the two sources of interference has proven difficult, largely because, in many situations, with resource competition, crosstalk may also be present. In the present experiment, we approached this by holding the stimulus–response mappings constant while manipulating the action effects associated with the responses. That is, we presented an additional perceptual event (i.e., action effect) that consistently followed response production.

In natural settings, action effects are the sensory consequences associated with a response (e.g., manual responses are typically followed by tactile feedback and corresponding visual changes to the hand and environment). However, given the difficulty in manipulating the action effects inherent to a response (e.g., removing tactile feedback from a manual key press), we instead add experimentally induced action effects and manipulate the compatibility between these effects and the stimuli cuing the actions. Given that action effects occur only after responses have been produced, they must be integrated into representations used by central operations to affect performance (e.g., Hommel, 1996). This is consistent with proposals that stimuli, responses, and action effects are bound within a shared representational format (e.g., Hommel, Müsseler, Aschersleben, & Prinz, 2001; Prinz, 1990; see also Frings et al., 2020), such that activation of one of these codes triggers the activation of the other. For instance, we choose to flip a light switch (action) based on the anticipated outcome of the light turning on (effect). Thus, action selection is grounded in the retrieval of the perceptual consequences of the action.

Empirical approaches to studying effect anticipation include response–effect compatibility (REC) paradigms, in which responses are followed by compatible (e.g., a right-sided action producing a right-sided effect) or incompatible (e.g., a right-sided action producing a left-sided effect) experimentally induced action effects (for review, see Pfister, 2019). Although effects are only presented after RT is measured, compatible response–effect pairings lead to faster performance (e.g., Kunde, 2001), indicating that participants establish an anticipatory representation of the effect prior to response selection. From a dual-task perspective, having to coordinate two tasks requires the anticipation or activation of multiple effect representations (often, the end goals of the two responses), and this may be a source of dual-task costs (Janczyk & Kunde, 2020).

Design of the present study

In the present study, the modalities of stimuli and effects were compatible across all conditions (i.e., Task 1: visual stimulus–manual response–visual effect; Task 2: auditory–manual–auditory). What varied was the conceptual relationship (e.g., animal stimulus, animal effect) between stimuli and effects, such that these codes were present within or between tasks. Participants were instructed to produce the action effect assigned to the stimulus cuing the action. For example, in the condition with stimuli and effects conceptually compatible across tasks, participants were instructed to produce a visual horse effect in response to a visual sun stimulus. In this instance, the stimuli and effects (visual stimulus, visual effect) are modality compatible within a task—as in all conditions—but, because horse sounds were used as stimuli in the other task, stimuli and effects are conceptually compatible between tasks.

We reasoned that we could use experimentally induced action effects to manipulate conceptual compatibility to distinguish between crosstalk and (domain-general and modality-specific) resource accounts of dual-task costs. The crosstalk account predicts larger dual-task costs when the conceptual representations overlap across tasks because the overlap will cause the stimulus for one task to activate central representations associated with the other task. In contrast, if resources are domain general (Navon & Miller, 2002; Tombu & Jolicoeur, 2003) or modality specific (e.g., Wickens, 1980), then there should be no differences across conditions. That is, given that the stimuli, responses, and action effects do not change modalities across conditions, response selection processes should not engage distinct sets of resources.

Method

Participants

We conducted a power analysis with G*Power (Faul, Erdfelder, Lang, & Buchner, 2007) prior to data collection on the dual-task cost data from Experiment 3 of Schacherer and Hazeltine (2020), in which participants performed VM and AM tasks with experimentally induced action effects. Using an effect size of ƞ2 = .101, we determined that 27 participants for each of the three conditions would be needed to obtain statistical power of .8 (Cohen, 1988). We tested 30 participants per condition (90 total).

A total of 104 undergraduate students from the University of Iowa participated in partial fulfillment of an introductory psychology course requirement. Data from 14 participants whose overall accuracy was less than 80% were discarded and not analyzed, leaving 90 total participants equally divided into three conditions: conceptual-compatible (CC; 18 females, Mage = 18.83, SDage = 0.91), conceptual-within (CW; 24 females, Mage = 18.97, SDage = 0.93), or conceptual-between (CB; 25 females, Mage = 18.80, SDage = 0.96) conditions. Vision and hearing were reported as normal or corrected-to-normal. Verbal consent was obtained prior to the experiment. All methods and procedures were approved by the Institutional Review Board at the University of Iowa.

Stimuli and apparatus

The experiment was conducted using Microsoft Visual Basic software (Version 15.0). Visual stimuli (3 cm × 3 cm; visual angle: ~3°) were a colored sun or tree presented on a black background on a 19-inch computer monitor located approximately 60 cm from the participant. Auditory stimuli were a pig’s oink or horse’s neigh presented binaurally via headphones sampled at 11,025 Hz. All stimuli were presented for 350 ms. Manual responses were the Q/W and I/O keys on a standard QWERTY keyboard for the visual and auditory tasks, respectively. The stimuli used to cue the responses and the responses themselves were identical across all three conditions.

In the conceptual-compatible (CC) condition, the action effects for the VM task were a visually presented black-and-white sun and tree, and the action effects for the AM task were the spoken words “pig” and “horse.” Thus, stimuli in this condition directly corresponded to the conceptually compatible effect (e.g., stimulus: sun; effect: sun). In the conceptual-within (CW) condition, stimuli were mapped to action effects that corresponded to the other stimulus within the task set. For the VM task, the sun stimulus was mapped to the tree effect, and the tree stimulus was mapped to the sun effect. For the AM task, the pig’s oink was mapped to the spoken word “horse” and the horse’s neigh was mapped to the spoken word “pig.” In the conceptual-between (CB) condition, stimuli were mapped to action effects that corresponded to a stimulus from the other task. For the VM task, the action effects were a visual black-and-white picture of a pig and horse. For the AM task, the action effects were the spoken word “sun” and “tree” (see Fig. 1).

Fig. 1
figure 1

Top. Stimulus–response–effect mappings. In the conceptual-compatible (CC) condition (left), stimuli were mapped to conceptually compatible action effects (e.g., sun stimulus, sun effect). In the conceptual-within (CW) condition (middle), stimuli were mapped to action effects that corresponded to the other stimulus within the task set (e.g., sun stimulus, tree effect). In the conceptual-between (CB) condition (right), stimuli were mapped to action effects that corresponded to a stimulus in the other task set. Bottom. Response times for the visual-manual single task, auditory-manual single task, and summed dual-task costs. Light-gray bars represent conceptual-compatible tasks; medium-gray bars represent conceptual-within tasks; and dark-gray bars represent conceptual-between tasks. Error bars represent standard error of the mean. Colored sun photograph was retrieved from https://commons.wikimedia.org/wiki/File:The_Sun_by_the_Atmospheric_Imaging_Assembly_of_NASA%27s_Solar_Dynamics_Observatory_-_20100819.jpg. Colored tree photograph was retrieved from http://clipart-library.com/free/oak-tree-transparent-background.html. See online article for color versions of these figures

Full size image

For the CB condition, we counterbalanced the stimulus–effect pairings across participants. To prevent trials in which there was complete overlap across tasks (e.g., VM task: visual sun stimulus mapped to visual horse effect; AM task: auditory horse’s neigh mapped to spoken word “sun”), no such trials were included in the design. However, this prevented us from having trials in which there was no overlap. Therefore, on each trial in the CB condition, one of the stimuli for one task matched the effect for the other task. For instance, if the VM stimulus–effect mapping was sun–pig, and the AM mapping was pig–tree, there is overlap between the pig (animal) representations.

Design and procedure

Verbal and written instructions were provided at the start of the experiment, and additional written instructions were presented on the computer prior to the start of each block. Instructions emphasized both speed and accuracy. At the end of each block, participants were shown their accuracy and mean RT. The experiment took approximately 45 minutes.

Participants completed 16 blocks of 36 trials each. The first two blocks were practice blocks, in which participants were presented the visual or auditory word, PRESS (for the VM and AM task, respectively), with the instructions to freely choose either of the two responses with the goal of producing the action effect. These blocks were included to establish the link between responses and effects.

The remaining 14 blocks consisted of six homogenous single-task blocks (three for each task, VM or AM), followed by eight alternating OR and AND blocks (four of each block type). The OR (i.e., mixed) blocks consisted of 36 single-task trials of either task (18 of each task) presented pseudorandomly. The AND (i.e., dual-task) blocks consisted of 36 trials in which both stimuli were presented simultaneously and two manual responses were required on each trial. The order of OR and AND blocks was counterbalanced across participants. Participants were instructed to respond as quickly and accurately as possible. No explicit instructions were provided regarding how to prioritize the tasks in OR and AND blocks.

The stimulus–response mappings were identical across the three conditions. What varied was the compatibility between the conceptual identity of the stimulus and the conceptual identity of the action effect (see Fig. 1). For all conditions, participants were instructed to produce the action effect that corresponded to a particular stimulus, as indicated by instructions at the start of each block. Example instructions for each condition were as follows: CC: “When you see the SUN, make the black-and-white SUN appear.”; CW: “When you see the SUN, make the black-and-white TREE appear.”; CB: “When you see the SUN, make the black-and-white PIG appear.” After being instructed on the stimulus–effect mapping, participants were then instructed how to produce the effect (e.g., “To make the [ACTION EFFECT] appear, press the Q key”).

Each trial began with the onset of a fixation cross for 500 ms, followed by presentation of the stimulus for up to 350 ms) and a response interval that lasted up to 3,000 ms. Action effects were presented immediately following response production with a duration of 350 ms. If the RT was less than 350 ms, the action effect replaced the stimulus. After the action effect was extinguished or the response interval expired, there was a 500-ms intertrial interval. No error feedback was given when the response was incorrect, and the next trial began 500 ms later. Participants were instructed to produce the action effect that corresponded with the stimulus (as indicated in the instructions). As such, the action effect served as its own form of corrective feedback.

Statistical analysis

For all three conditions, the first four blocks (two practice, two single-task) were excluded from analysis. Thus, we analyzed data from 12 blocks—four single-task, four OR, and four AND blocks. Additionally, all responses given within the first 200 ms after stimulus onset or any RTs greater than 2,000 ms were excluded from analysis. Lastly, trials in which no response was detected, in which one or both of the responses were incorrect, and trials following an error were removed from the analysis of RT. For the 12 analyzed blocks, we removed 15.8% of trials from our final RT analysis.

For our analysis of single-task performance, we conducted separate analyses of variance (ANOVAs) for each task (VM, AM), with condition as a between-subjects factor. For our analysis of mixing and dual-task costs, we summed the mixing costs (OR RT/ER − Single-task RT/ER) or dual-task costs (AND RT/ER − OR RT/ER) across both tasks because participants may have prioritized the tasks differently (e.g., Hazeltine et al., 2006). The summed costs were submitted to ANOVAs, with condition as a between-subjects factor. If there was unequal variance across conditions (Levene’s p < .05), we applied the Brown–Forsythe test statistic for comparisons. Significant main effects were followed up with independent-samples t tests to compare across conditions. In addition to null hypothesis significance testing, we also report Bayes factors alongside their respective interpretation (Lee & Wagenmakers, 2013) for each of our analyses below. The mean RTs and ERs for all conditions are shown in Table 1.

Table 1 Mean response times (RT) and error rates (ER) for each of the trial types
Full size table

Results

Single-task

For VM RT, we observed significant differences across conditions (CC: 452 ms; CW: 518 ms; CB: 462 ms), F(2, 61.40) = 4.94, corr p = .010, ƞ2 = .102, BF = 4.69. Follow-up tests revealed a significant difference between the CC and CW conditions, t(58) = 2.57, p = .013, d = 0.63, BF = 3.88; and between the CW and CB conditions, t(58) = 2.30, p = .025, d = 0.59, BF = 2.30; but not between the CC and CB conditions, t(58) = 0.56, p = .580, d = 0.14, BF = 0.30. Likewise, for VM ER we observed significant differences across conditions (CC: 4.1; CW: 8.3; CB: 2.8), F(2, 43.55) = 5.90, corr p = .005, ƞ2 = .119, BF = 9.76. Follow-up tests revealed an identical pattern to that observed in RT: significant differences between the CC and CW conditions, t(58) = 2.12, p = .038, d = 0.55, BF = 1.68; and between the CW and CB conditions, t(58) = 2.88, p = .006, d = 0.74, BF = 7.52; but not between the CC and CB conditions, t(58) = 1.41, p = .163, d = 0.37, BF = 0.60.

For AM task RT, there were no significant differences across conditions for both RT (CC: 571 ms; CW: 609 ms; 573 ms), F(2, 87) = 1.04, p = .360, ƞ2 = .023, BF = 0.23; and ER (CC: 3.3; CW: 4.7; CB: 3.0), F(2, 87) = 1.55, p = .218, ƞ2 = .034, BF = 0.34; although there was a similar trend to that observed in the VM task, such that RTs were longest and ERs were largest in the CW condition.

The longer single-task RTs and higher error rates in the CW condition parallel reports of response–effect compatibility (albeit here we assessed stimulus–effect compatibility) on single-task performance (e.g., Koch & Kunde, 2002; Kunde, 2001) demonstrating that anticipatory effect representations affect response selection processes. In other words, when the anticipated action effect corresponds to the stimulus from the opposing stimulus–effect pair, RTs are longer (e.g., Hommel et al., 2001). The shorter RT in the CB condition compared with the CW condition suggests that, during single-task performance, the effects of stimulus–effect compatibility are restricted to representations within the current task set. The analogous observation in ER suggests the single-task results were not due to a speed–accuracy trade-off.

Mixing costs

Mixing costs represent the performance differences between OR blocks and single-task blocks. They are thought to reflect either an increase in working memory load from having to maintain two task sets in an active state (Los, 1996) or greater ambiguity regarding which task to perform (Rubin & Meiran, 2005). There were no significant differences in mixing costs across conditions for both RT (CC: 301 ms; CW: 394 ms; CB: 298 ms), F(2, 87) = 2.77, p = .068, ƞ2 = .060, BF = 0.88 (anecdotal evidence for H0); and ER (Levene’s p = .008) (CC: −1.1; CW: −1.2; CB: −0.7), F(2, 44.43) = 0.10, corr p = .906, ƞ2 = .002, BF = 0.12 (anecdotal evidence for H0). This is consistent with other reports of stimulus–response conceptual-compatibility on mixing costs in task switching (Schacherer & Hazeltine, 2019).

Dual-task costs

Consistent with the crosstalk account, we observed a significant effect of condition (CC: 368 ms; CW: 379 ms; CB: 548 ms), F(2, 87) = 6.23, p = .003, ƞ2 = .125, BF = 12.59. Follow-up independent-samples t tests revealed significant differences between the CC and CB conditions, t(58) = 3.29, p = .002, d = 0.85, BF = 20.00; between the CW and CB conditions, t(58) = 2.84, p = .006, d = 0.74, BF = 6.97; but not between the CC and CW conditions, t(58) = 0.20, p = .843, d = 0.05, BF = 0.27. There were no significant differences in ER across conditions (CC: 3.3; CW: 5.3; CB: 5.7), F(2, 87) = 1.82, p = .169, ƞ2 = .040, BF = 0.42 indicating that these results were not due to a speed–accuracy trade-off.

Notably, the pattern of dual-task costs in RT differed from that of single-task RT (see Fig. 1). Single-task RTs were longest in the CW condition and near equivalent between the CC and CB conditions, whereas dual-task costs were largest in the CB condition and near equivalent between the CC and CW conditions.

Given that both tasks in the CB condition involved an animal representation and a nature representation, the two tasks overlapped in terms of conceptual representations. In contrast, with the CC and CW conditions, these distinct conceptual representations were encapsulated within individual tasks. Thus, unlike single-task performance, the response selection processes engaged during dual-task performance appear to operate at the level of task sets, rather than individual stimulus–effect representations (see Halvorson & Hazeltine, 2015).

Discussion

The goal of the present study was to test whether action effects alter dual-task costs in a manner consistent with resource competition or crosstalk. Strikingly, the manipulation of the action effects affected single-task RTs and dual-task costs differently. Specifically, although the modality-specific resources were identical across conditions, single-task RTs were longer when the stimuli and action effects overlapped within a task, and dual-task costs were larger when they overlapped across tasks. Note that the stimuli and responses and their mappings were identical across conditions. These findings are most consistent with crosstalk accounts.

Crosstalk within and between tasks

While all conditions had the same degree of modality-based crosstalk, the larger dual-task costs observed when similar conceptual codes were present across tasks supports the idea that crosstalk may arise when there is conceptual overlap between concurrently activated task sets (e.g., Navon & Miller, 1987).

This observation is consistent with the proposal that stimuli and action effects are represented by features within a shared representational domain (e.g., Hommel et al., 2001; Prinz, 1990) and that crosstalk between these codes increases dual-task costs. Accordingly, when similar codes are present within a task, central operations can easily bind the appropriate stimulus to the appropriate effect on dual-task trials, reducing between-task interactions (i.e., crosstalk). In the CC and CW conditions, stimuli overlapped conceptually with their action effects, facilitating the binding of these representations. In contrast, in the CB conditions, conceptual overlap may have led to activation of representations across the tasks. Thus, the larger costs may reflect the unwanted interactions between the conceptual codes associated with the stimulus from one task and the conceptual codes associated with the effect from the other task.

Although crosstalk is most often invoked as a source of dual-task costs, it may also arise during single-task performance. In single-task trials in the present experiment, crosstalk differed across conditions. For the CW single-task trials, crosstalk activated the conflicting response alternative within the task set—for example, a pig stimulus and a pig effect associated with the horse response. For CC and CB single-task trials, within-task crosstalk was low as central operations do not have to differentiate between conflicting sources of information activated by the current response alternatives. Thus, single-task RTs were longest in the CW condition and similar in the CC and CB conditions. This pattern was also observed in OR blocks, in which only one stimulus from either task was presented on each trial. Thus, like the findings from dual-task trials, the findings from trials in which only a single response was produced are consistent with the proposal that effect anticipation influences response selection, such that selection requires more time when the anticipated action effect is incompatible with the presented stimulus (e.g., Frings et al., 2020; Hommel et al., 2001). The difference in the patterns of single-task RTs (CW is greatest) and dual-task costs (CB is greatest) is difficult to explain with resource limitations, but straightforward for a crosstalk account.

Crosstalk or resource competition?

When comparing crosstalk and domain-general resource models (Navon & Miller, 2002; Tombu & Jolicoeur, 2003), it is fruitful to examine how single-task RTs relate to dual-task costs. Given the proposal that longer single-task RTs reflect greater competition for resources (Tombu & Jolicoeur, 2003; Wickens, 1980), the length of single-task RTs should be proportional to the magnitude of dual-task costs. Thus, if dual-task costs reflect the demand for resources, as indexed by single-task RTs, costs should have been large in the CW condition, in which single-task RTs were longest. However, this was not observed in the present study, suggesting that dual-task costs were determined by crosstalk between concurrently activated stimulus–effect associations at the level of the task set, independent of these associations’ effect on single-task RTs.

Multiple resource theories are more difficult to rule out because of their inherent flexibility. When dual-task costs are reduced, the two tasks must access separate resources. Resource domains are identified—often post hoc—by the presence of dual-task costs. Thus, this circularity is a significant limitation in that there is no way to independently measure a resource (Navon, 1984). The present experiment was designed to test modality-based resource accounts (e.g., Wickens, 1980), and the differences in dual-task costs were not consistent with this proposal because the stimulus–response mappings were constant across conditions. However, it is possible that conceptual categories (e.g., animals, nature) may each define separate resource domains. Resource domains defined by abstract conceptual representations could explain why costs were larger when similar conceptual representations were present across tasks (CB condition). However, without independent evidence for such resource domains, the crosstalk account appears more parsimonious.

Limitations

We note that we intentionally created situations emphasizing the role of crosstalk by imposing similarities between the stimuli for one task and action effects for the other task. Moreover, with our use of two two-choice tasks, response selection demands may have been minimal, and it remains possible that resource competition is more prominent when tasks involve more complex response selection demands. In short, these particular tasks may have emphasized crosstalk, but resource limitations may also exist and would be more prominent under other conditions.

Additionally, the current study involved a single 1-hour session, and we can only speculate as to how practice would affect the pattern of results. If practice strengthens the associations between stimuli, responses, and effects, then larger costs in the CB condition should persist with practice, as response selection mechanisms must operate on overlapping representations across tasks. Alternatively, if the impact of action effects diminishes over time, then dual-task costs should be similar across all conditions after sufficient practice, as stimulus–response pairings are identical across conditions.

Summary

The present findings add to a growing literature (e.g., Janczyk, Pfister, Crognale, & Kunde, 2012; Janczyk, Pfister, Hommel, & Kunde, 2014; Schacherer & Hazeltine, 2020; Wirth, Janczyk, & Kunde, 2018) illustrating a role of anticipated action effects on dual-task performance. They also provide support for the crosstalk account of dual-task costs rather than resource competition and highlight the range of dimensions—modality and conceptual—over which crosstalk can affect dual-task performance.