Don’t Go Chasing Waterfalls: Motion Aftereffects and the Dynamic Snapshot Theory of Temporal Experience

Abstract

The philosophical investigation of perceptual illusions can generate fruitful insights in the study of subjective time consciousness. However, the way illusions are interpreted is often controversial. Recently, proponents of the so-called dynamic snapshot theory have appealed to the Waterfall Illusion, a kind of motion aftereffect, to support a particular view of temporal consciousness according to which experience is structured as a series of instantaneous snapshots with dynamic qualities. This dynamism is meant to account for familiar features of the phenomenology of time, such as succession, continuity, and change. Previous theories have typically appealed to a subjective present occupying an interval of time; that is, a “specious present.” I argue, through analysis of motion aftereffect illusions and the rare condition of akinetopsia, i.e. motion-blindness, that the Waterfall Illusion fails to support the dynamic snapshot theory as intended. Furthermore, I suggest that future theories of subjective time should see temporal phenomenology as the result of non-localised processes closely tied to the mechanism underlying consciousness generally.

1 Introduction

The Waterfall Illusion is a type of motion aftereffect (MAE) known at least since the time of Aristotle (1908) (On Dreams: Part 2). It involves the apparent motion of a static object following a subject’s prolonged exposure to moving stimuli. Recently, this phenomenon has been recruited¹ in support of the dynamic snapshot view of subjective time (e.g. Prosser 2016; Prosser 2017; Arstila 2018). The dynamic snapshot view holds that temporal phenomenology can be analysed as snapshot-like experiences that encode information about change, motion, succession, etc., at an instant. The dynamic snapshot view therefore rejects the orthodox “specious present” doctrine, first popularised by William James (1890: 609), which claims the experienced present must be extended in time rather than instantaneous, as phenomena like change, persistence, continuity, and succession require intervals of time.² The contention of this paper is that motion aftereffects like the Waterfall Illusion have in fact been misappropriated and do not provide evidence for the conclusions of the dynamic snapshot theorists.

The argument against the dynamic snapshot theorists’ use of MAEs has two strands. First, it will be argued that the phenomenology of motion aftereffects should in fact compel us to recognise that subjective temporal properties encoded at an instant are not sufficient for a subjective experience of motion, contrary to the position of the dynamic snapshot theorists.³ This becomes apparent when the phenomenology of the Waterfall Illusion is appropriately interpreted. Second, it will be argued that MAEs like the Waterfall Illusion should be seen as illusions of motion, and therefore a result of visual processing, rather than temporal illusions per se. The rare condition of akinetopsia (motion blindness) helps to demonstrate that the experience of visual motion is in fact unrelated to distinctly temporal phenomenology, so drawing analogies between their mechanisms is misguided. Ultimately, the connection between subjective time and motion aftereffects is too strained for advocates of the dynamic snapshot view to effectively make a case for generalising to temporal phenomenology.

Underlying the misappropriation of MAEs is a methodological error regarding the kind of evidence that constitutes appropriate fodder for models of temporal phenomenology. The kind of theories that seek to explain temporal phenomenology are not the kind that can be supported by evidence from any one particular sensory modality or cognitive function, such as visual motion.⁴ The dynamic snapshot theorists have therefore missed their target by focusing on a visual motion illusion. As a general constraint on models of temporal phenomenology and explanations of the cognitive mechanisms underlying such models, we should see temporality as a global precondition for subjective experiences. It is thus more plausible that temporal structure is an inherent, fundamental feature of the information processing mechanism underlying consciousness (whatever account we give of this), rather than a result of localised processes.

This paper proceeds first by describing, in section two, the phenomenology of the Waterfall Illusion as traditionally understood, highlighting the difference between illusions of motion and distinctively temporal illusions. The third section outlines the dynamic snapshot theorists’ understanding of the illusion. The fourth section gives a different interpretation—one which is more faithful to the phenomenology and which reveals the illusion is not supportive of a snapshot view of any kind. On this interpretation, encoding change-like properties at an instant is insufficient for the phenomenology of motion without the perception of a change in position over time. The fifth section demonstrates the disconnect between visual motion and temporal phenomenology by considering the phenomenon of akinetopsia, or motion blindness, and also by delving into the neurophysiological underpinnings of motion perception. The sixth section develops the second strand of the argument against the effectiveness of MAEs as evidence for the dynamic snapshot theory. It is argued that MAEs are unrelated to the systems or processes responsible for the conscious experience of time generally. The connection, therefore, between visual motion-like properties of experiential objects and aspects of temporal phenomenology, like succession, is not strong enough to support inferences about the nature of the latter.

2 The Waterfall Illusion

An especially vivid early account of the Waterfall Illusion came from Robert Addams, a Scot who observed the phenomenon at the Falls of Foyers near Loch Ness in 1834. Addams reported that, “having steadfastly looked for a few seconds at a particular part of the cascade, admiring the confluence and descussation of the currents forming the liquid drapery of waters,” he then “suddenly directed [his] eyes to the left, to observe the vertical face of the sombre age-worn rocks immediately contiguous to the water-fall,” and, upon doing so, “saw the rocky face as if in motion upwards, and with apparent velocity equal to that of the descending water” (Addams 1834: 373). In other words, just a few seconds staring at a moving object (the Falls), resulted in the perceived motion of a stationary object (the cliff) in the opposite direction and at a speed apparently equal to that of the moving object from which his gaze was diverted.

The illusion does not only apply to waterfalls and cliffs, but also other forms of motion and stationary objects, e.g. spiral motion and dot patterns on a screen.⁵ It is telling, however, that experimental researchers and early observers both thought of the Waterfall Illusion as a visual illusion, and not an illusion of time in any significant sense. In other words, on the face of it, the illusion has to do with what we see, which in this case does not match the state of the world (i.e. the cliff face is not in fact moving). It is not immediately obvious that the illusion concerns how we experience time, although friends of snapshot theories have insisted otherwise (e.g. Robin Le Poidevin 2007: 88; Valterri Arstila 2018: 290; and Simon Prosser 2016: 123).

The contrast between MAEs and postdictive effects helps to illustrate the difference between distinctively temporal phenomenology and phenomenology that occurs over time. Visual motion, it will be argued, is an example of the latter. In contrast, phenomena like succession, continuity, and persistence are fundamental aspects of temporal phenomenology, indispensable to subjectivity in a way that motion is not. Postdictive effects are the archetypal illusions of time in the sense that they seem, prima facie, to affect the normal experience of succession. They are called postdictive because it seems in these situations that, paradoxically, what happens in the future is capable of dictating the present experience. Examples include the flash lag illusion (Khoei et al. 2017),⁶ colour phi (Bach 2014), and the tactile illusion of the “cutaneous rabbit” (Geldard and Sherrick 1972).

For brevity’s sake, we can focus on the flash lag illusion as an illustrative example of how postdictive effects concern temporality in a way that MAEs do not.⁷ In one instance of the flash lag illusion, a red square moves from left to right across a screen. When the red square reaches the midpoint, a green square is presented below it at exactly the same horizontal position. However, if the red square will continue moving to the right, subjects perceive that the red square is to the right of the green square at the time of the latter’s presentation, rather than exactly aligned on top (as it is in fact). If the red square doesn’t continue moving right, no illusion is reported.

In the flash lag illusion, it therefore seems the perceptual system peers into the future before generating the “present” experience. Ostensibly, then, the ordering of perceived events is modulated retrospectively. We have in this case an illusion of succession, such that the perceived event corresponding to the presentation of the green square is perceived to lag that of the moving red square when in fact they are simultaneous. While postdictive illusions appear to be related to the way our brain constructs and orders experiences, MAEs do not concern such things. Rather, MAEs, along with other visual illusions of size, shape, and colour, are temporally normal experiences of things that are not the case.

Because visual motion illusions like the Waterfall Illusion do not concern mistakes related to the temporal structure of experience, and because motion is not a distinctively temporal property, we should not see MAEs as temporal illusions at all. There is nothing particularly special about a moving ball as opposed to a stationary ball in the way that we perceive them both lasting through time. We do not perceive a stationary ball as static in time, but rather only static in space. Perceived visual properties like colour or motion persist through time just like other perceptual phenomena and have little bearing on the temporality of experience. As Kristie Miller points out, it seems natural to say, “we have the relevant temporal phenomenology even if nothing perceptually changes,” (Miller 2019). Indeed, empirical studies do not show that we feel time stops flowing in a sensory deprivation chamber, although this passage feels slower than usual (Wittmann 2017: 127; 125).

In contrast to visual properties of objects, like changes and movements, properties like succession and continuity are deeply and inextricably connected to experience itself. The latter properties are distinctively temporal because these sorts of properties are the sine qua non of experience through time—they are what make us think phenomena, whether stationary or moving, are lasting through time in the first place. It is not the aim here to give a definitive, exhaustive list or phenomenological description of truly temporal properties, but hopefully these considerations make it clear that, because visual motion illusions are not distinctively temporal phenomena in the way that illusions of succession are, we should not think MAEs are temporal illusions.

3 The Dynamic Snapshot Account of the Illusion

For proponents of the dynamic snapshot view, the Waterfall Illusion is meant to demonstrate the plausibility of a model of subjective time according to which experience can be analysed as successive instants or moments that exhibit temporal properties like succession, continuity, and persistence. We can recall that, after Addams diverted his gaze from the Falls of Foyers, he saw a static object, the neighbouring cliff face, appear to move. The dynamic snapshot theorist takes it that static moments, like static objects, can exhibit such dynamic qualities as Addams perceived. Our experiences of succession, continuity, persistence, change, and motion are not unlike the illusory motion of the cliff face in this way: what is actually a procession of static snapshots is imbued at each step with qualities that lend a certain non-static appearance to each moment.

Before considering the dynamic snapshot proposal in more detail, it is important to understand where it comes from and why it is appealing. All snapshot views, dynamic or otherwise, maintain the temporal structure of our experience is akin to a sequence of frames in a flipbook, except in this case each frame is an instantaneous experience. According to Barry Dainton’s influential formulation, the “classic” snapshot view, which he calls the cinematic model, holds that “our immediate awareness lacks any (or any significant) temporal extension, and the same applies to the contents of which we are directly aware—they are akin to static, motion-free ‘snapshots’ or ‘stills’.⁸ Our streams of consciousness are composed of continuous successions of these momentary states of consciousness” (Dainton 2017).

The perennial popularity of snapshot views⁹ is perhaps attributable to the simplicity of atomic theories¹⁰ and the intuitiveness of the analogy with film. If one thinks the mind-independent world proceeds as a series of moments, like movie frames, it seems reasonable to suppose that our experience of the world has a similarly “cinematic” temporal structure.¹¹ However, to account for the further experience of succession, appealing to a mere succession isn’t enough, for we might just as easily experience each event as entirely unrelated to the others.¹² Thomas Reid, responding to John Locke’s (1690/1975) simple snapshot view,¹³ therefore claimed experiences of succession arise through the reflective comparison of experiential snapshots via memory (Reid 1785/2002: 271). Reid, in a rare departure from his usual reverence for common sense, thus endorses what Dainton calls phenomeno-temporal antirealism (PT-antirealism); in other words, for Reid, succession, change, and duration are not directly experienced (Dainton 2017). Unfortunately, Reid’s view doesn’t fit well with the way we usually take our experiences of succession to be, as they do not obviously require reflection or memory retrieval to occur. One way to get around this issue is to reject the snapshot view entirely and adopt a specious present view—one in which a window or interval of consciousness can take in successive events together. However, for those enamoured with the simplicity of the snapshot picture, another alternative is to tweak the theory.

With a view to preserving more common-sense notions of temporal phenomenology and account for our seemingly direct experience of change and succession, the dynamic snapshot theory adds that the instantaneous snapshots of experience have dynamic qualities. More precisely, Valterri Arstila, the most vociferous defender of the dynamic snapshot theory, identifies four essential theses of the dynamic snapshot view (Arstila 2018: 291). These are:

• 1 realism about temporal phenomenology (PT-realism)

• 2 punctuality of phenomenal contents

• 3 purity of temporal phenomenology

• 4 encapsulated mechanisms

Phenomeno-temporal realism (PT-realism) is the claim that, “we have immediate experiences of change, motion, and other temporal phenomena, as the majority of philosophers claim” (Arstila 2018: 291). Taking an example from C.D. Broad, under PT-realism, there is a phenomenological difference between experiencing the second-hand of a clock changing position and noticing, perhaps via inference from memory, that an hour-hand has changed positions, with the latter not immediately part of our present experience in the same way as the former (Broad 1923: 351). The divergence between the dynamic snapshot theory and standard snapshot theories is stark here. As Hoerl (2017) argues, the standard snapshot theorist cannot draw a principled distinction between experiences of the second hand and the hour hand if both kinds of experience are based on memory, a la Reid. By endorsing PT-antirealism and denying the phenomenological distinction between experiences of the second hand and hour hand, standard snapshot theorists apparently ignore a basic explanandum. In contrast, the dynamic snapshot theorist hopes to avoid this oversight by accounting for the second hand with dynamic snapshots, while holding there is nothing dynamic about the hour hand.

Arstila’s second thesis constitutes the core of any snapshot view. This is the claim that experience is ultimately composed of snapshot-like instants. Such a thesis amounts to a denial that experience is extended over time in the way non-snapshot views would have it, e.g. retentionalism and extensionalism, which see the present as consisting of an interval known as the “specious present.” Formulated in an intentional way, according to dynamic snapshot theory the contents of experience are not extended over time, unlike retentionalism, where the contents are extended, or extensionalism, where the content/vehicle distinction collapses but the extension of the experience itself is preserved.¹⁴

The third thesis (we can call it “the purity thesis”) maintains that temporal phenomenology is possible without “an associated phenomenology of things being different at different times” (Arstila 2018: 291). In other words, according to the purity thesis, we only need to experience one instant to have a phenomenology of change, motion, succession, continuity, and whatever else is thought of as belonging to temporal phenomenology. Dynamic snapshot theorists take the Waterfall Illusion to support this thesis. By challenging the dynamic snapshot theorist’s use of the Waterfall Illusion, this paper will cast doubt on the plausibility of the purity thesis.

Lastly, the fourth thesis, concerning encapsulated mechanisms, contends that, “temporal phenomenology is brought about by primitive mechanisms, each separate from the other” (ibid.). This means that temporal phenomenology is the result of many different mechanisms, perhaps different ones for each modality, as well as aspects of cognition. Such a position multiplies the complexity of any possible account of subjective time drastically, ruling out explanations that seek to ground temporal phenomenology in fundamental structural features of any single overarching mechanism responsible for consciousness. The Waterfall Illusion is meant to support the thesis of disparate primitive encapsulated mechanisms by demonstrating that motion phenomenology, as one aspect of temporal phenomenology, is produced by vector encoding occurring in the visual cortex, while other aspects of temporal phenomenology would be the result of similar mechanisms elsewhere.

Advocates of the dynamic snapshot theory use the Waterfall Illusion as evidence for the claim that features of temporal phenomenology can be experienced in isolation without requiring an extended, experiential specious present. The Waterfall Illusion supposedly demonstrates this by showing that even a static object, in the absence of a perceived change in position over time, nonetheless can appear to change or move. As Arstila puts it, “…the most significant aspect of this explanation [of the Waterfall Illusion] is that the experience of motion is explained in a framework where the contents can, subjectively speaking, be confined to an instant” (Arstila 2018: 290). Prosser likewise claims “‘Moving’ is a state that something can be in at an instant,” (Prosser 2017: 149) before leaning on the Waterfall Illusion for empirical support. To put it in the terms of Addam’s visit to the Falls of Foyers, for Arstila the cliff face would take on an immediate sensation of motion without requiring any time to elapse.

Although the purported phenomenology of “instantaneous motion” does not appear in the descriptions of Addams and others, Arstila nevertheless thinks this is what is happening in such situations based on the cognitive and neural mechanisms thought to be responsible for the illusion. Arstila’s idea here is particularly indebted to Robin Le Poidevin’s description of the processes underlying MAEs. Le Poidevin (2007: 89) identifies two neural mechanisms, one that detects motion “by a change in retinal stimulation” and another that “register[s] the relative position of an object and store[s] it in the short-term memory for comparison with later perceptions of its relative position,” resulting in perceived change when these diverge.¹⁵ Le Poidevin believes the first mechanism is more “primitive,” does not require any change in position, and gives rise to a sense of the change occurring “now.” He thinks, following Gregory (1966), that the first mechanism results in so-called “pure motion” phenomenology in that it does not concern a relation. The second mechanism, meanwhile, “employs short-term memory, takes a series of snapshots of an object’s relative positions, and compares them,” (Le Poidevin 2007: 89). Le Poidevin extends this analysis to temporal phenomenology generally, such that, for instance, “the conjunction of the very recent memory of C [a musical note] with the perception of E [another musical note] gives rise to an experience of ‘pure succession’” (ibid.: 91). This kind of extrapolation from the case of visual motion is a move that proponents of the dynamic snapshot view will also take up.

Le Poidevin is committed to a more traditional (non-dynamic) snapshot view. Consequently, he maintains the only difference between the illusory case and the case of veridical motion is that, while in the former only the first mechanism is engaged, i.e. we experience motion without a change in position, in the normal case both mechanisms contribute to motion phenomenology. Arstila and Le Poidevin agree that motion and change should be considered aspects of temporal phenomenology. However, whereas Le Poidevin, like Reid, sees the comparative function afforded by short-term memory as crucial, Arstila’s purity thesis (3) disputes this. Instead, for Arstila, “both pure motion and pure succession are explained by appealing to a primitive mechanism specific to those experiences. For example, the second mechanism involved in the waterfall illusion is likely to be a second-order motion processing mechanism” (Arstila 2018: 291). The dynamic snapshot theorist can then avoid the snapshot theorist’s somewhat counterintuitive claim that memory is involved in motion perception even though it doesn’t feel this way.

Like Arstila, Simon Prosser (2016) argues the Waterfall Illusion supports the dynamic snapshot view in that it supposedly provides an instance of a static object exhibiting motion. Prosser believes one of the reasons philosophers generally don’t think experience can be punctate in the way demanded by a snapshot view is because it seems that for a full-blooded temporal phenomenology, including the experience of change, succession, and the like, we must have an experience “including different states at different times” (ibid.: 123). The thinking here is that the two states must be contained within a single experience for the transition between them to be directly perceived in experience (i.e. PT-realism), and it is this thinking that motivates specious present views.

For Prosser, as with Arstila, the Waterfall Illusion challenges the specious present view by demonstrating that phenomena like motion, change, etc. do not require an extended experience, but only a snapshot. Based on the illusion, Prosser argues vector encoding of perceptual information (i.e. encoding of “both the rate and direction of motion”) could be the mechanism responsible for the experience of motion (Prosser 2016: 124; Prosser 2017: 149), which allows for instantaneous states to exhibit properties like change over time.¹⁶ He then generalises this mechanism, like Arstila and Le Poidevin, to offer an account of “all such continuous perceptible changes” of any sort. Prosser (2017: 149) explicitly offers the model of “instantaneous vector rate of change” as an alternative for the phenomenology of change to the specious present view, which he views as unnecessary. Notably, however, Prosser does not think such a mechanism can accommodate discontinuous changes like the sudden change of a light from on to off.

Prosser, with his focus on the connection between temporal experience and metaphysical views of time, is more interested in establishing the mere possibility of a mechanism that eschews the specious present. Despite the spirited arguments he presents for the dynamic snapshot view, Prosser is careful not to fully commit to it. That said, Prosser does claim that, “for experiential content there is no logical entailment from the lack of temporal extension of the content to the content containing only what is ‘static’, or lacking change” (Prosser 2016, 122). The possibility of a world in which we perceive motion only in a way that is similar to the illusory case can be granted. However, it is of crucial importance to an adequate understanding of motion phenomenology, let alone subjective time, that such a world is not our world, as we shall see.

4 Understanding Visual Motion Phenomenology

Until the recent resurgence of snapshot theories, the Waterfall Illusion derived most of its fame in philosophy from its paradoxical character. As several philosophers have noted, the initial description of the illusion, in which the stationary object is said to exhibit motion, is misleading (Blakemore 1973; Frisby 1980; Crane 1988). Tim Crane, for instance, observes that, “[A]lthough the stationary object appears to move, it does not appear to move relative to the background of the scene. That is, there is a clear sense in which it also appears to stay still. There is a distinct appearance of lack of motion as well as motion [emphasis original]” (Crane 1988, 142). Thus, we are faced with an apparent contradiction: the cliff face next to the waterfall appears to be simultaneously in two incompatible states, i.e. moving and not moving. An illusion where we perceive objects as both moving and not moving at the same time calls out for some explanation.

The key to dissolving this paradox, it seems, is to disentangle two senses of “motion.” We do not have the impression that the cliff face is really moving in a full-blooded sense. We do not think that the cliff face is moving in the same way the waterfall is. Neither are we confused or unsure about this, as we might be in the case of judgments of length in the Müller-Lyer Illusion.¹⁷ It is not merely that our beliefs about what is moving remain unchanged, but rather that we do not directly perceive the object as moving at all. It is a strange experience, both like and unlike movement, but it is not the direct perception of movement per se. Instead, it is more plausible to say we perceive a distortion in our perception rather than the movement of an object we perceive.¹⁸

The “motion” detected in the illusion is easily distinguishable from its counterpart in the normal case, when an object is perceived as actually changing spatial location. Le Poidevin readily admits this in his discussion of MAEs: “ordinary perception does not seem at all like those cases of perceptual illusion where we are aware of some inconsistency” (Le Poidevin 2007: 88). At best we think of the formerly stationary object as exhibiting motion-like properties or having the “feel” of motion without actually being perceived as truly moving.¹⁹

Thus, it is probably more accurate to interpret motion aftereffects as causing a motion-like distortion of a part of the visual field. In other words, instead of perceiving a change in the spatial location or extension of the objects we see, it is rather that we note a disturbance in our own vision. As we move our heads, the objects and the surrounding context suffer the same distortion, which is not perceived as a property of the objects in themselves but instead a progressive warping of a section of the visual field, regardless of what falls within that section. The sense that the distortion is one of the visual field becomes even more salient when the gaze is shifted to a collection of objects at various distances and at various orientations, as the warping of that section of our visual field then seems quite unnatural for the objects themselves to be exhibiting individually (as it ignores relative location, distance from the viewer, or even whether or not an object is present, instead distorting the whole section in the same way).²⁰

We have isolated two different perceptual scenarios with differing phenomenal characters—one illusory and one veridical. Under normal circumstances motion refers to a process whereby an object changes position in space. Evidently, this occurs in cases of perceived veridical motion like looking at an actual waterfall, but, arguably, not when we perceive the illusory case. John Frisby’s (1980) description of the illusory scenario is telling in this regard: “we are still aware of features remaining in their ‘proper’ locations even though they are seen as moving” (ibid., 101). In other words, the relations between the parts of the objects seem to remain stable relative to each other, despite the impression of movement. What we perceive, rather than objects moving from one position to another, is the apparent warping of perceptual space. However, in occupying a part of space in the visual field that is affected by the illusion, we can say that the objects, which do not appear to change location in space, nonetheless appear to exhibit motion-like properties, in the sense of being affected by this underlying distortion.

The apparent paradox of the Waterfall Illusion results from conflating two perceptual phenomena under the heading “motion”: (1) cases in which objects are perceived as having motion-like properties and (2) cases where we perceive that an object has changed spatial location over time. Only cases of the second type exhibit the phenomenology of veridical motion,²¹ while cases of the first might arise in both illusory and veridical cases. Although both (1) and (2) would presumably be concurrent in the case of a long exposure to the ongoing motion of an object, this is not so in the illusory case. The fact that we do not perceive normal motion in the same way as the illusory motion of the cliff face, and that we do not confuse the two, indicates that we cannot isolate the phenomenology of motion to the one neural mechanism to which Prosser and Arstila appeal. The mechanism responsible for the illusion is not all that is in play in the normal case, for, if it were, the illusory motion would appear normal and convincing, much the same as the actual waterfall. Some change in position must be perceived to have convincing experiences of motion, contrary to Arstila’s purity thesis. Following Russell (1937), Le Poidevin (2017) calls the requisite kind of motion displacement, which cannot occur at any one particular instant because it necessarily involves a succession of differing positions (in contrast with the “dynamic sensation” sought by Italian futurist painters, and perhaps by the dynamic snapshot theorists). As indicated earlier, accommodating this kind of motion provides a strong motivation for specious present views, where a conscious window can contain the successive positions of whose relations we become aware.

As far as normal motion perception goes, we are left with a more sophisticated view than the dynamic snapshot theorists can offer, involving both perceived motion-like properties and perceived changes in spatial location over time. Against the dynamic snapshot theorists, encoding information about motion-like properties at an instant is insufficient for a normal experience of motion. As a result, on the basis of the phenomenology of MAEs, there is no reason to suppose that temporal experiences can be explained only in virtue of information encoded at an instant, as this, too, following the analogy, would be insufficient. Vector encoding, were the analogy to hold, would not result in our normal temporal phenomenology but only an unconvincing impression of it (although even this is difficult to make sense of). Such a consequence does not speak in favour of the dynamic snapshot theory.

It seems we still need both information encoded at a particular time and information that requires perception over an interval. With just the former we end up with the unconvincing illusory case, which does not seem to us like a true case of perceived motion, while with just the latter, moving objects may be perceived in staccato fashion, or at least as failing to convey a distinctive sense of movement at any given time. Indeed, the latter scenario is one of the immediate phenomenological inconsistencies one might think of against a snapshot theory, and casts doubt on Arstila’s punctuality thesis (2). In fact, such staccato experiences occur when the motion pathway of the visual cortex malfunctions, as in the rare phenomenon of akinetopsia, or motion blindness. The next section will consider this condition in order to illustrate how visual motion and temporal phenomenology are quite distinct.

5 Visual Motion and Temporal Phenomenology Come Apart

Akinetopsia, or motion blindness, is instructive concerning both the mechanisms underlying motion perception and the disconnect between motion perception and temporal phenomenology. Akinetopsia reveals that normal motion phenomenology cannot be isolated to vector encoding of motion properties, although the latter, associated with activity in areas MT/V5 and MT+ of the visual cortex,²² does play an important role in visual phenomenology (Newsome and Paré 1988; Ajina et al. 2015). Akinetopsia also reveals that, for all the importance attached to it by the dynamic snapshot theorists, visual motion is not especially significant to our overall temporal phenomenology. For this reason, it is misguided to use the case of motion perception to make inferences about the mechanisms underlying temporal phenomenology.

Akinetopsia is a rare condition in which a subject fails to perceive motion normally, instead perceiving a stroboscopic-like effect when confronted with visual movement. In some extremely rare cases, moving objects can seem to become “stuck” or remain frozen for some period of subjective time. The key insight about these cases regarding subjective time, however, is that subjective time itself does not freeze in the same way as the particular moving objects in the visual field do. Rather, the subject is able to say that the object has remained stuck for some length of time. The absence of visual motion no more affects the subject’s overall sense of time than the absence of colour vision.

The most studied subject to exhibit this rare condition is known as patient L.M. It is helpful to consider her case, a condensed version of which is provided by Heywood and Kentridge (2009):

[L.M.’s] chief complaint was that she no longer saw movement; moving objects appeared ‘restless’ or ‘jumping around’. Although she could see objects at different locations and distances, she was unable to find out what happened to them between these locations. She was severely handicapped in her daily activities, e.g. she had substantial difficulty in pouring drinks into a cup or glass, because the fluid appeared ‘frozen like a glacier’; she could not see the fluid rising and was unable to judge when to stop pouring. (ibid.: 24).

In the example above of liquid becoming frozen like a glacier, it is of crucial importance that only the liquid is apparently frozen. The static objects of visual experience (like the glass or kettle—in fact, everything that is not exhibiting observable motion) still persist through time, along with objects perceived through other sensory modalities, whether in motion or not. The subject’s cognition also continues normally through time. She does not cease to experience time; rather, certain perceptual objects do not exhibit motion as they normally would.

The fact that L.M. has trouble judging when to stop pouring indicates she is aware that the liquid, which should be moving, is frozen for some duration. In her case, though, she must make a conscious calculation about how long a certain quantity of liquid would normally take to fill a cup in order to determine when to stop, instead of perceiving this unfold and acting accordingly as unaffected subjects might. That the moving object is experienced as static for some interval reveals that extinguishing the phenomenology of visual motion does not prevent us from experiencing intervals of time as normal. Vector encoding, as a potential cognitive mechanism associated with visual motion, is a red herring when we consider that knocking out this mechanism does not grossly interfere with temporal phenomenology properly understood.

As we might expect, patient L.M. was also unable to perceive motion aftereffects like the Waterfall Illusion. Zihl et al. (1983) presented L.M. with a spiral-type MAE and found that she did not report experiencing the illusion at all. Nonetheless, L.M. continued to experience time, and, in fact, was able to use the timing of associated auditory cues to make inferences about visual motion, e.g. the speed and direction of vehicles nearby, to help her navigate a confusing world of staccato, yet persisting, visuals (ibid.: 315). L.M. was thus not temporally impaired but visually impaired. Whatever mechanism is responsible for the type of visual motion experienced in MAEs, that mechanism therefore does not seem related to temporal phenomenology in any interesting sense.

Granted, the dynamic snapshot theorist is left with the logical possibility of vector encoding as the mechanism responsible for temporal phenomenology. In essence, the dynamic snapshot theorist can still say something like, “the brain appears to use vector encoding in one application, so it is possible it uses the same mechanism when it comes to other aspects of phenomenology.” However, this gives us no reason to think the brain is actually doing this in the domain of temporal phenomenology. In fact, temporal phenomenology, which heretofore has not been connected with any particular sensory modality or area of the brain, would seem especially resistant to such an explanation. This is because, as we see in the case of akinetopsia, it is not as easily extinguished, for example by brain trauma, transcranial magnetic stimulation, or psychopharmacological intervention, as other, less fundamental aspects of phenomenology such as visual motion. Indeed, the extinction of temporal phenomenology seems impossible without eliminating consciousness entirely.

Besides the disconnect between visual motion phenomenology and temporal phenomenology more generally, akinetopsia also makes clear that normal motion phenomenology is not generally a matter of just one mechanism, like vector encoding. This becomes clear when we consider that, even though the akinetopsic patient L.M. suffered from the near total elimination of normal functioning in area MT/V5, motion phenomenology was not completely eliminated. Heywood and Kentridge elaborate: “Although L.M. has been dubbed ‘motion-blind’, she retains rudimentary movement vision, e.g. she can discriminate speed and direction of motion of high-contrast gratings at low speeds” (Heywood and Kentridge 2009: 25).

The closely related underlying neurophysiology of akinetopsia and MAEs indicates the former condition is much like a chronic inversion of motion aftereffects. Akinetopsia typically results from lesions to visual cortex area MT/V5 and can also be induced by transcranial magnetic stimulation (TMS) to this area (Beckers and Zeki 1995). MT/V5 is active in cases of illusory motion like the Waterfall Illusion,²³ as well as in cases where subjects perceive “implied motion,” for instance in comic strips (Heywood and Kentridge 2009: 25). In addition to inducing akinetopsia, when TMS is applied to MT/V5 it is capable of eliminating motion aftereffects in normal subjects (ibid.). Naturally, MT/V5 is the proposed site of Prosser and Arstila’s vector encoding mechanism. In the case of the akinetopsic subject this area is essentially deactivated.

One conclusion we can draw from the neurophysiological analysis of akinetopsia is that visual motion is not the result of one simple process. Information perceived over a span of time is also relevant to our normal phenomenology of motion and is not parasitic or secondary to the operation of V5/MT, meaning the purity thesis looks increasingly dubious. This becomes even clearer when we consider that important perceptual features are retained in akinetopsic patients, including some degree of motion perception:

[…] Akinetopsia provides a clear example of selective loss of phenomenal consciousness for visual motion [emphasis added]. Patient L.M. […] retained the ability to use biological motion cues which are probably processed by brain areas distinct from those concerned with processing motion of rigid bodies and global motion of a scene. (Heywood and Kentridge 2009: 25).

Progress in understanding the areas of the brain responsible for vision reveals that besides processing properties like shape and colour, the brain has a specialised “visual motion pathway.”²⁴ According to Newsome and Paré (1988: 2201), this pathway “originates in striate cortex and terminates in higher cortical areas of the parietal lobe,” processing and giving rise to experiences of the motion-like properties of objects (see Rokszin et al. 2010 for a detailed neurophysiological description). As mentioned, crucial parts of this pathway, most notably processing in area MT/V5, can be severely disrupted for patients with akinetopsia and it is also this pathway that is affected in visual motion illusions, of which the Waterfall Illusion is one example. However, this pathway is more complex than the dynamic snapshot theorist appreciates, involving several stages, multiple cell types, multiple streams, and more areas than V5/MT alone (Rokszin et al. 2010). More importantly, this pathway is not obviously related to general temporal phenomenology, as becomes apparent when we consider that its malfunction does not disrupt akinetopsic patients’ overall sense of time continuing to flow. For these reasons we should be sceptical of claims that the alleged vector encoding mechanism of V5/MT responsible for some aspects of visual motion perception is in any way related to our experience of time.

One might think of the motion detection pathway as providing a quick and easily accessible way of encoding potentially life-threatening information, like the speed and direction of an incoming predator, which is quickly registered by these neurons without having to consider a large change in location over a span of time.²⁵ A tiger, to follow this example, can be simply attributed motion and direction without having to perceptually register a substantial change in spatial location, as might be required for the full-blooded perception of motion (stripes and all) that we would normally have upon seeing a tiger run for some length of time. Arguably, the length of time necessary for the normal, fleshed out perception may not always be enough, in which case the visual motion pathway could prove highly adaptive. The neurophysiology of MAEs therefore indicates the Waterfall Illusion phenomenon and its associated physical substrate are the result of unique adaptations likely unrelated to temporal properties like succession and continuity, which would be more primitive, global features of consciousness.

As our look at akinetopsia shows, it is certainly not the case that encoding of motion-like properties in the visual cortex just amounts to our experience of time. We should also be wary of the claim that similar processes would be involved in experiences of all temporal properties. One reason to resist such an extrapolation is that, unlike in the motion case, there are no identifiable neural populations or brain areas picking up on fundamental features of temporality like succession from sensory stimuli in the same way that motion can be picked up from vision. We also do not find cases of patients that lack fundamental aspects of temporal phenomenology like the experience of succession (i.e. one thing happening after another). If it were possible for brain lesions to result in disturbances of this kind, we might think the processes underlying temporal phenomenology were localized in specific areas. These cases might convince us that temporal structure is not a global, fundamental feature of consciousness, but a bolt-on module or a series of modules. However, such cases are not forthcoming.

So long as a subject is conscious, there seems to be a basic experience of time passing. Barry Dainton (2010: 104) aptly illustrates the ubiquity of the sense of passage through the example of lying in a deckchair looking at the sky, as well as examples from other sensory modalities:

For some moments now you have been staring at an empty region of blue sky and nothing has changed. Your inner monologue has (if only briefly) ground to a halt, you have seen no movement, your visual field is filled with an unvarying expanse of blue. But is your consciousness entirely still or frozen? Have you come to a complete stop? No. Throughout this period you remain conscious, and conscious of the blue presence continuing on; you have a (dim, background, passive) awareness of the blue constantly being renewed from moment to moment. This passive awareness of continuation and renewal is perhaps more vivid in the case of auditory experience. Imagine hearing a sustained but unwavering note played on a cello: you hear a continuous and continuing flow of sound. This feature—call it “immanent phenomenal flow”—is possessed by all forms of experience (think of the burning sensation on the tongue caused by biting on a chilli pepper), and is a dynamic feature of experience that is independent of changes of the ordinary qualitative sort (the chilli-induced burning is felt as continuing on even when its intensity and qualitative character remains constant).

This ubiquitous sense of flow is not dependent on movement or change. This sense of flow would not lend itself to description in the vector-like terms of trajectory or velocity and does not easily analogize to the movement of objects through space.

It is also not clear, given the ubiquity of the sense of passage, that this can or should be analysed as a consequence of disparate encapsulated mechanisms, rather than a feature of a process responsible for consciousness in general. However, Arstila’s fourth thesis posits “encapsulated mechanisms,” that is, separated, primitive mechanisms, rather than a general mechanism responsible for temporal phenomenology. For Arstila, a “second-order motion processing mechanism” is responsible for the “temporal” phenomenology in the waterfall illusion (Arstila 2018: 291). However, according to Arstila, similar mechanisms also exist for every other part of our temporally structured subjective experience. For example, similar, though separate mechanisms must be found for the experience of succession in thinking and deliberating, the entertaining of beliefs and desires, interoception, proprioception, emotion and mood states, etc. Not only is there no evidence of analogous mechanisms to MAEs here, but the dynamic snapshot appeal to vectors (representing direction and velocity) just doesn’t seem to coherently apply. We might also consider that different adaptive pressures would have resulted in different evolutionary paths in the development of our cognitive faculties. As visual animals, quickly registering the motion of a tiger would be of the utmost importance to survival. There is no pressure to develop a similar mechanism for encoding information about the succession of thoughts at an instant, were this even possible.

There are other compelling reasons to reject the contention that temporal phenomenology is the result of many disparate encapsulated mechanisms. Besides neurological implausibility, we might consider the principle of parsimony. Why adopt the encapsulated mechanism approach when it is simpler, more explanatory, and truer to the phenomena to locate temporality as a feature of a general mechanism responsible for consciousness, whatever that may be? A feature of a general mechanism also has the benefit of accounting for why people never seem to go “timeless” the way people go “blind”—temporality is indispensable to consciousness because it is inextricably bound up with the processes responsible for it.

6 Inescapable Temporality

A certain strand of philosophy dating back at least to Kant (1781/2007) has contended that temporality is really a necessary precondition for conscious experience. For Kant, “Time is […] given a priori. In it alone is actuality of appearances [phenomena] possible at all. Appearances may, one and all, vanish; but time (as the universal condition of their possibility) cannot itself be removed,” (Kant, 1781/2007: A31). The necessity of time for experience can be argued by appeal to the inconceivability of experience without temporal structure. This inconceivability is not just a failure of imagination on our part. Experiences as we know them are essentially in time and occur over time—an experience that was not so would not be worthy of the name, besides being inconceivable.

Given consciousness is essentially temporal, it is plausible that temporality is an inherent feature of the widely distributed processes responsible for subjective experience generally. A recent proposal from Hohwy, Paton, and Palmer (2016) takes this notion to heart, placing temporality at the heart of the predictive mechanism that Bayesian-brain-style information processing accounts see as responsible for consciousness. For Hohwy, it is the continuous update and replacement of our cognitive system’s predictions about the hidden causes of our sensory stimuli (our models of the world) in light of new information from an ever-volatile environment that results in a feeling of rolling along through time (ibid.). Such a theory presents temporality as a deep feature, found throughout a multilevel mechanism of cognition, which is inherited by any conscious event. Isolating subjective temporality to one particular neural or cognitive mechanism, e.g. vector encoding, associated with a particular modality or phenomenon is too restrictive. Rather, it is in global processes that we may have more luck identifying the neural and cognitive correlates of temporal phenomenology in a way that illuminates why it is so fundamentally inescapable for a conscious being.

Connecting temporal phenomenology to relatively local processes like the mechanisms at work in visual motion processing is thus a deeper flaw than it appears at first. The mistake becomes clearer when we look at the implications of the view, were we to accept the dynamic snapshot theory. Suppose we grant that motion can be perceived in a robust way solely via the vector encoding mechanism that illusory motion from MAEs is meant to demonstrate. Instead of explaining temporal phenomenology, such a position only postpones explanation. This is because the phenomenology of visual motion, if determined by vector encoding at an instant, must still appeal to an overarching temporal phenomenology for the illusion, or indeed anything, to be experienced at all.

To elaborate, let us assume, as per the dynamic snapshot theory, that perceived motion of the illusory kind is sufficient for the phenomenology of visual motion and there is no direct perception of change in spatial location over an interval of time. Despite this, the experience of the illusory motion persists through time and must do so in order for us to have it. When faced with a static object perceived as having motion-like properties, it is not the case that our experience has stopped, is static, or is confined to an instant. The organism’s overall consciousness does not become instantaneous (i.e. confined to an instant or solely about instantaneous states of affairs) just because we are experiencing an unmoving object. For the experienced illusory object (the cliff face, say) to appear static in space, but with motion-like properties, it must exhibit persistence over some length of time. This temporality is inescapable; events continue to succeed one another over time, even if the content remains the same.

Whether in the illusory or non-illusory case, while the cliff next to the waterfall is either stationary or apparently exhibiting motion, our experience of that object must continue through a window of time for us to experience that object as either static or moving. It doesn’t give the dynamic snapshot theorists any further ammunition that the cliff-face can be seen to exhibit motion-like properties, as the perception of any properties whatsoever still requires an extended experience in the way that the competing, “specious present” models of temporal phenomenology identify (i.e. retentionalism and extensionalism). These views agree, contrary to the purity thesis, that the experiential present must be an interval of time for us to enjoy the phenomenology we are familiar with, like succession and continuity.

Against the idea of a present window, Arstila cites the frequent disagreement over lengths of the temporal interval²⁶ to suggest that no such specious present exists. However, insofar as an interval of any length is indicated at all by empirical studies it cannot be concluded that a snapshot view is more reasonable than a specious present view. Rather, it would be more reasonable to maintain there may be some inherent task-dependent flexibility in the duration of the specious present, as James (1890) long ago suggested.

7 Conclusion

Temporal phenomenology cannot be adequately explained in the way that the dynamic snapshot theorists wish. We do not have good reason to think that things like succession, continuity, and other aspects of subjective time result from or are related to similar mechanisms as the vector encoding that may occur in area V5/MT when confronted with the Waterfall Illusion. This should be a cautionary tale for theories seeking to provide explanations of temporal phenomenology. Evidence for or against models of temporal phenomenology will not be found at the level of the neural mechanisms responsible for any one specific sensory modality. Rather, models of temporal phenomenology should be intimately related to models of subjective consciousness generally, as temporal phenomenology is a precondition for having subjective experiences in the first place and not an “additional” feature of consciousness.

The Waterfall Illusion and motion aftereffects do not support the dynamic snapshot theory. The misappropriation of motion aftereffects in support of the dynamic snapshot theory results from a faulty interpretation of these illusions. Furthermore, the connection between visual motion and temporal phenomenology turns out to be spurious, meaning that, despite the role of vector encoding mechanisms in the former, it is an unjustified leap from that mechanism to the assertion of a similar mechanism underlying temporal phenomenology. Consequently, motion aftereffects do not support views of subjective time consciousness that reject the specious present, while countervailing considerations remain strong in favour of the specious present.