Imaginal perspective switches in remembered environments: Transformation versus interference accounts

Abstract

Imaginal perspective switches are often considered to be difficult, because they call for additional cognitive transformations of object coordinates (transformation hypothesis). Recent research suggests that problems can also result from conflicts between incompatible sensorimotor and cognitive object location codes during response specification and selection (interference hypothesis). Three experiments tested contrasting predictions of both accounts. Volunteers had to point to unseen object locations after imagined self-rotations and self-translations. Results revealed larger pointing latencies and errors for rotations as compared to translations, and monotic latency and error increases for both tasks as a function of the disparity of object directions between real and imagined perspective. Provision of advance information about the to-be-imagined perspective left both effects unchanged. These results, together with those from a systematic error analysis, deliver clear support for an interference account of imaginal perspective switches in remembered surroundings.

Introduction

While moving in space humans and other intelligent mobile animals keep track of changes of directions and distances to object locations in their surrounding. Changes in spatial relations can result from bodily movements of the actor, but can also result from imaginal switches of perspective to other points in the environment. Especially in humans, a number of everyday problems are characterized by the need to perform imaginal perspective switches to other vantage points in space; for instance, taking the perspective of another person while giving instructions, planning one’s own as well as anticipating other peoples movement trajectories while playing a ball game; tele-operation of vehicles, etc. Following a short review of previous research, the present studies tested different hypotheses with respect to the cognitive and sensorimotor mechanisms underlying imaginal perspective switches in remembered space.

In developmental psychology there has been a long tradition of thinking of spatial perspective taking as a fundamental cognitive ability that is not fully developed before children reach the age of about ten years (Piaget & Inhelder, 1948/1967). The ability to imaginally switch perspectives is often described as a developmental progress from an exclusively egocentric—or self-centered—mode of spatial processing in the younger child, to a dominantly allocentric—or environment-centered—mode of processing in older children and adults (Millar, 1994). Recent research has made different corrections to Piaget’s original position, demonstrating that the ability to take foreign perspectives depends, for instance, on the testing method used (e.g., Huttenlocher & Presson, 1979; Newcombe & Huttenlocher, 1992), on the locomotor status of the child (e.g., Acredolo, 1990; Bertenthal, 1996), or on geometrical aspects of the environment (e.g., Hermer & Spelke, 1996). An increasing number of researchers consider difficulties with imaginal perspective switches, and related spatial tasks, not so much as resulting from cognitive limitations in the younger child’s construction of the spatial environment, but as resulting from problems to efficiently cope with conflicting spatial information in such situations (Millar, 1994; Newcombe & Huttenlocher, 2000; Thelen, Schöner, Scheier, & Smith, 2001).

While focusing on the ontogeny of spatial abilities, developmental research has almost exclusively relied on qualitative error analyses, limiting the possibilities to test hypotheses about time-critical processing demands of imaginal perspective taking tasks. Over the last years, research in adults has started to fill this gap by using response times in addition to error data. Various studies show that switches of spatial perspective are easy (i.e., comparable to baseline performances without switches), when blindfolded actors are allowed to bodily move into a second position before having to point to, or walked up to, an unseen object location; for distances of more than 10 m, and turns up to at least 360°, vestibular, kinesthetic, and maybe also motor-efferent signals seem to support an automatic updating of spatial relations to objects in the surround (Berthoz, 1997; Farrell & Thomson, 1998; Loomis, Klatzky, Golledge, & Philbeck, 1999; May & Klatzky, 2000; Rieser, 1999; Rieser, Guth, & Hill, 1986; Wang & Spelke, 2000).

Switches of spatial perspective turn out to be much more difficult when the actor has to imagine being located at a position different from the one he or she is actually bodily located at. When asked to point to unseen objects from such an imagined perspective, additional processing costs are reflected in increases in response times or errors using very different methodologies (Amorim & Stucchi, 1997; Boer, 1991; Bryant & Tversky, 1992; Easton & Sholl, 1995; Farrell & Robertson, 1998; Franklin, Tversky, & Coon, 1992; Hintzman, O’Dell, & Arndt, 1981; May, 1996; Rieser, 1989; Rieser, Garing, & Young, 1994; Roskos-Ewoldsen, McNamara, Shelton, & Carr, 1998; Waller, Montello, Richardson, & Hegarty, 2002; Woodin & Allport, 1998; Wraga, Creem, & Proffitt, 2000).

Of special interest for the present research are studies comparing different geometrical types of movements in imagined surroundings. Movements in the horizontal plane can be described as rotations, i.e., changes of facing direction while staying in the same location (e.g., 90°-turn in place), as translations, i.e., changes of location while keeping the facing direction constant (e.g., making three steps in one direction), or as a combination of both, i.e., changing location and facing direction at the same time. Different studies show that pointing judgments are slower and more inaccurate after imaginal rotations than after imaginal translations (Easton & Sholl, 1995; Presson & Montello, 1994; Rieser, 1989), and that response times and errors increase as a function of the self-rotation angle the actor has to imagine being turned around to (Easton & Sholl, 1995; Farrell & Robertson, 1998; Hintzman et al., 1981; May, 1996; May & Wartenberg, 1995; Presson & Montello, 1994; Rieser, 1989; Wraga et al., 2000). These effects seem to be invariant over different procedural variations (e.g., verbal indication of object direction vs. pointing with the extended arm vs. pointing with a joystick), as well as stable for different environments and spatial scales (e.g., from confined experimental rooms up to knowledge about the university campus).

Up to now, debate continues about the nature of the mechanisms underlying imaginal repositionings in remembered surroundings. Memory-based perspective switches constitute a complex cognitive task including processes of stimulus identification, spatial memory retrieval, transformation of position and object coordinates, as well as response planning and execution. Extra costs observed in repositioning tasks have been mainly discussed in terms of mental transformation requirements (e.g., Boer, 1991; Easton & Sholl, 1995; Presson & Montello, 1994; Rieser, 1989; Wraga et al., 2000); in the last years, specific processing problems resulting from spatial response conflicts have also been discussed (May, 1996, May, 2000, May, 2001). The studies reported in this article aimed at contrasting predictions formulated on the basis of both hypotheses. Although both accounts do not exclude each other on logical grounds, experimental evidence that would support or weaken—or maybe even confirm or discredit—the one or the other account seem useful for advancing and further stimulating research in the field.

The mental transformation hypothesis states that imaginal perspective switches are difficult, because they require additional cognitive transformations of object coordinates when switching to a new position in an imagined environment. The account can be considered as an extension of the idea of mental object transformations (Cooper & Shepard, 1978) to spatial imagery of viewer perspectives; very different versions of the more general idea of perspective switches as mental transformations can be found in the literature (e.g., Boer, 1991; Easton & Sholl, 1995; Huttenlocher and Presson, 1973, Huttenlocher and Presson, 1979; Mou, McNamara, Valiquette, & Rump, in press; Presson & Montello, 1994; Rieser, 1989; Wraga et al., 2000).

The most elaborate formulation of the transformation idea was developed in the context of a body-centered memory retrieval model by Easton and Sholl (1995). Easton and Sholl’s model distinguishes between an environment-centered system of object-to-object relations (serving as knowledge basis for allocentric coding of locations in memory) and a body-centered system of self-to-object relations (serving as structure for egocentric retrieval of locations from memory). It assumes that both systems operate in concert, and that body-centered retrieval of object coordinates functions as an imaginal superposition of the self-to-object system onto a portion of the object-to-object representational system. Thus, imaginal repositionings are assumed to be an analog process of mental rotation or translation, leading to processual extra costs the larger the rotation angle or translation distance to superimpose the self-to-object system onto the object-to-object system becomes (for a detailed description of the model see Easton & Sholl, 1995, pp. 483–487; for further treatments and extensions see Sholl, 1995, Sholl, 2000, Sholl, 2001).

Evidence in favor of the mental transformation account, in general, and the imaginal superpositioning model, in particular, comes from studies revealing increases in pointing latency and/or pointing error as a function of increases of the imagined self-rotation angle (Easton and Sholl, 1995, Exp. 1; Presson & Montello, 1994; Rieser, 1989, Exp. 3), as well as from experiments showing increases of pointing latency and/or error as a function of the imagined self-displacement distance (Easton and Sholl, 1995, Exps. 1–4). Some authors take the fact that actors need more time and commit larger errors after imaginal self-rotations than after imaginal self-displacements as evidence that the underlying transformation processes are more simple in the case of mental translations (e.g., Rieser, 1989, argued for a direct access to spatial knowledge after translations, but not rotations), or conversely, more complex in the case of mental rotations (e.g., Presson & Montello, 1994, argued for a higher degree of computational complexity in rotations, under the assumption that a Cartesian coordinate representation is used).

The sensorimotor interference hypothesis, on the other hand, states that imaginal perspective switches are difficult, because actors have to deal with spatial information conflicts when acting from an imagined perspective in the environment. The account does not dispute that additional cognitive computations are necessary when people perform imaginal perspectives switches, but locates the main source of difficulties in an interference conflict between real and imagined perspective; different formulations of the general idea can be found in the literature (e.g., Angyal, 1930; Brockmole & Wang, 2003; May, 1996; Newcombe & Huttenlocher, 2000; Presson, 1987).

A specific version of the idea has recently been worked out by May, 2000, May, 2001. May’s model assumes that imaginal repositionings lead to conflicts between sensorimotor object location codes, as defined by the actor’s real (i.e., bodily taken) perspective, and cognitive codes of the same object locations, as defined by the to-be-imagined perspective in space. Conflicts between the two representations are assumed to lead to interference effects during response selection as a function of the degree and type of spatial incompatibility between the competing codes of the surrounding (for discussions of incompatibility effects in different spatial tasks see Castiello, 1996; Fitts & Seeger, 1953; Logan & Zbrodoff, 1982; Lu & Proctor, 1995; Tipper, 1992).

Assuming that a pointing task is used to examine imaginal perspective switches the model makes a distinction between two independent sources of interference effects: The first source is referred to as object direction disparity, and can be described in terms of the angular difference between body- and task-defined egocentric object directions (which can vary in the range between 0° and 180°). For rotations, the amount of object direction disparity is equivalent to the angle of imagined self-rotation; e.g., a rotation of 90° leads to a object direction disparity of 90° between real and imagined perspective for all objects in the surrounding. For translations, the situation is more complex, since the amount of disparity depends on the relation between the actor’s actual position and the critical object location, as well as on the distance and direction of the imagined self-displacement in space; e.g., for an object 5 m in front of an actor, a self-displacement of 1 m to the right leads to a disparity of about 12°, a displacement of 5 m to the right to a disparity of 45°, and so on. It is postulated that object direction disparity leads to a selection problem between incompatible action vectors during response specification, and that the magnitude of the conflict depends on the degree of angular difference between the competing vectors. More specifically, pointing responses from the imagined perspective are considered to be the final output of a conflict resolution between the incompatible response direction codes from real and the imagined perspectives; increases in interference effects as a function of the amount of object direction disparity are expected for both repositioning tasks (i.e., no conflict at 0° angular disparity, maximal conflict at 180° angular disparity).

It is important to note that there are hardly any experiments comparing rotations and translations while controlling for the amount of object direction disparity (see discussion in next section); the only published data available indicate significant performance differences between imaginal rotations and translations when the amount of object direction disparity was held constant for both (Presson & Montello, 1994).¹ The interference model accounts for these differences by postulating a second source of interference effects, referred to as head-direction disparity. Head-direction disparity is assumed to be responsible for the performance differences between rotations and translations, as changes of heading between actual and tested perspective are found in rotation tasks only. Independent from the problem of deciding between the two incompatible action vectors (first interference source), the problem is one of specifying the action vector from the imagined perspective when the spatial reference system defined by the actor’s current body position leads to a rotational conflict. More specifically, it is postulated that the specification of the response direction becomes more difficult because the reference system underlying the to-be-imagined facing direction is continually interfered with by misleading head-direction signals from the reference system associated with the actor’s actual position in space (for a recent discussion of the neural basis of head-direction signals see Wilson, 2000).²

The present experiments were the first to investigate performance differences between imaginal rotations and translations when object direction disparity was under experimental control and independently varied. Of special interest was whether differences between both movement types would still be found after object direction disparity was under experimental control and of what type the differences would turn out to be (e.g., constant differences or monotonically increasing differences as a function of self-rotation angle). Prior empirical support for a sensorimotor interference account comes from experiments showing that imaginal rotations and translations are affected by object direction disparity in a similar manner (i.e., monotonic latency increases), while bodily performed rotations and translations into the same positions revealed no latency increases compared to baseline conditions without switches (May & Wartenberg, 1995). Further evidence comes from experiments showing that disorientation—actors were turned around in circles until losing track of their orientation to the surrounding—led to significantly more accurate and faster pointing responses as compared to performances of actors remaining oriented to the spatial surrounding (May, 1996); such a facilitation effect agrees well with the assumption that disorientation relieves actors from sensorimotor interferences resulting from directionally incompatible location codes and head-direction signals when responding from an imagined spatial perspective. A recent series of experiments extended on this finding by showing that imaginal repositionings in the actual environment (bodily presence) exerted stronger detrimental effects on pointing performances as compared to imaginal switches in a remote environment the actor was not bodily attending while being tested (May, Rieser, & Young, in preparation).

Different methods have been used to test the assumptions of both theoretical accounts. Tests of transformation assumptions have generally used methods of single-trial imaginal switches. Participants are instructed to point to a target location A, as if facing towards (rotation) or standing at (translation) a reference location B; single-trial testing means that target and reference locations switch from trial to trial. In contrast, tests of interference assumptions have generally used blocked testing of multiple target locations per to-be-imagined perspective. The present studies used the method of single-trial switches, because it seems best suited to ensure that transformation processes are an effectual part of the repositioning task requirements examined, as has been shown in earlier work using this method (e.g., Easton & Sholl, 1995; Rieser, 1989). Conversely, blocked testing was not used, because it could lead to an underestimation of the role of transformational mechanisms in imaginal repositionings, since spatial transformations might be partially or completely executed at the beginning of a testing block, not revealing themselves proper anymore in the latencies and errors measured.

With the exception of a study by Presson and Montello (1994, cf. Footnote 1) experiments using single-trial testing methods have generally neglected the potential influence of disparity of egocentric object directions in imaginal perspective switching tasks. Controlling for object direction disparity does not seem critical when testing rotations, because rotation angle and object direction disparity increase proportionally. Control, however, becomes important when testing imaginal translations, or tasks that have a translational as well as a rotational component. A closer inspection of spatial tasks used in previous studies helps to illustrate the point: A reanalysis of the translation tasks examined by Rieser (1989, Exp. 3) revealed constant amounts of 38° object direction disparity for all eight, equally distant, reference locations. Finding no performance differences between the different translation conditions is therefore in agreement with a transformation account (constant distances), as well as with an interference account (constant amounts of disparity). In a similar vein, a reanalysis of Easton and Sholl’s (1995) studies, in which translation distance was varied, revealed a confounding of repositioning distance and object direction disparity; for example, in their Experiment 1 distances (in feet) and corresponding disparity amounts (in °) were: 2^′: 13°; 3^′: 21°; 4^′: 28°; 5^′: 37°; 6^′: 48°; 7^′: 50°; 8^′: 55°; 9^′: 56°. Again, it is not possible to tell, whether the observed performance decreases went back to imagined self-displacement distance (transformation hypothesis), or whether they resulted from movement-induced disparities of egocentric object directions (interference hypothesis).³

The present studies tested imaginal rotations and translations by choosing an environmental layout that allowed to control for and independently vary the amount of object direction disparity in both repositioning tasks. The overall goal was to generate experimental results that would help to elucidate the potential contributions of transformation and interference processes to imaginal perspective switches in space.

Experiment 1 replicated Rieser’s pioneering study (1989, Exp. 3) with single-trial switches, while introducing object direction disparity as an additional experimental factor. The experiment aimed at contrasting transformation and interference hypotheses with respect to the causes of extra costs in imaginal repositionings (i.e., movement type and amount versus object direction disparity and head-direction disparity). In order to reach comparable amounts of object direction disparity for rotation and translation tasks a spatial layout with separate sets of objects markers on an inner circle, and position markers on an outer circle was used (instead of a single set objects and positions arranged as a circle in previous studies). This type of layout of objects and positions provided for translation trials with large amounts of object direction disparity (i.e., by enforcing imaginal traversals of objects), that had been missing in earlier single-trial experiments. Rotations and translations were compared on the basis of a reclassification of object-position combinations into four increasing classes of disparity (0–45°, 46–90°, 91–135°, and 136–180°).

Experiment 2 changed the presentation order of position and object information used in Experiment 1, as well as in most earlier studies, from object-position to position-object; also was a variation of the time-interval between presentation of position and object information introduced (SOA of 1 s, 3 s, and 5 s). The experiment tested contradictory predictions transformation and interference hypotheses make about the possibility to process a to-be-imagined perspective in advance, that is before the target object is presented. More specifically, an account postulating cognitive transformations during imaginal self-relocation should allow for pre-processing, while an account postulating interferences during response specification and selection should not allow for pre-processing.

Experiment 3 used the same experimental design, but had participants learn the environment from a topographic map instead of having them learn it by directly exploring the real-world layout. Map learning was used to ensure that actors would have to use an allocentric representation of the environment as considered important by the retrieval model of Easton and Sholl, 1995, Sholl, 1995. In addition to the three experiments, an analysis of pointing errors is reported; the aim was to test potential distractor influences of the irrelevant (body-defined) object directions on the observed (task-defined) pointing responses.