In the River of Consciousness

1.

“Time,” says Jorge Luis Borges, “is the substance I am made of. Time is a river that carries me away, but I am the river….” Our movements, our actions, are extended in time, as are our perceptions, our thoughts, the contents of consciousness. We live in time, we organize time, we are time creatures through and through. But is the time we live in, or live by, continuous—like Borges’s river? Or is it more comparable to a chain or a train, a succession of discrete moments, like beads on a string?

David Hume, in the eighteenth century, favored the idea of discrete moments, and for him the mind was “nothing but a bundle or collection of different perceptions, which succeed each other with an inconceivable rapidity, and are in a perpetual flux and movement.”

For William James, writing his Principles of Psychology in 1890, the “Humean view,” as he called it, was both powerful and vexing. It seemed counterintuitive, as a start. In his famous chapter on “the stream of thought,” James stressed that to its possessor, consciousness seems to be always continuous, “without breach, crack, or division,” never “chopped up, into bits.” The content of consciousness might be changing continually, but we move smoothly from one thought to another, one percept to another, without interruption or breaks. For James, thought flowed; hence his introduction of the term “stream of consciousness.” But, he wondered, “is consciousness really discontinuous… and does it only seem continuous to itself by an illusion analogous to that of the zoetrope?”

Before 1830 (short of making an actual working model, or toy theater), there was no way of making representations or images that had movement. Nor would it have occurred to anyone that a sensation or illusion of movement could be conveyed by still pictures. How could pictures convey movement if they had none themselves? The very idea was paradoxical, a contradiction. But the zoetrope proved that individual images could be fused in the brain to give an illusion of continuous motion, an idea that was soon to give rise to the motion picture.

Zoetropes (and many other similar devices, with a variety of names) were extremely popular in James’s time, and few middle-class Victorian households were without one. All of these instruments contained a drum or disc on which a series of drawings—of animals moving, ball games, acrobats in motion, plants growing—was painted or pasted. The drawings could be viewed one at a time through radial slits in the drum, but when the drum was set into motion, the separate drawings flicked by in rapid succession, and at a critical speed, this suddenly gave way to the perception of a single, steady moving picture. When one slowed the drum again, the illusion vanished. Though zoetropes were usually seen as toys, providing a magical illusion of motion, they were originally designed (often by scientists or philosophers) with a sense that they could serve a very serious purpose: to illuminate the mechanisms both of vision and of animal motion.

Had James been writing a few years later, he might indeed have used the analogy of a motion picture. A movie, with its taut stream of thematically connected images, its visual narrative integrated by the viewpoint and values of its director, is not at all a bad metaphor for the stream of consciousness itself. And the technical and conceptual devices of cinema—zooming, fading, dissolving, omission, allusion, association and juxtaposition of all sorts—rather closely mimic (and perhaps are designed to mimic) the streamings and veerings of consciousness.

It is an analogy that Henri Bergson used twenty years later, in his 1908 book Creative Evolution, where he devoted an entire section to “The Cinematographic Mechanism of Thought, and the Mechanistic Illusion”:

We take snapshots, as it were, of the passing reality, and…we have only to string these on a becoming, …situated at the back of the apparatus of knowledge, in order to imitate what there is that is characteristic in this becoming itself…. We hardly do anything else than set going a kind of cinematograph inside us…. The mechanism of our ordinary knowledge is of a cinematographical kind.

Were James and Bergson intuiting a truth in comparing visual perception—and indeed, the flow of consciousness itself—to such a mechanism? Are the brain mechanisms that give coherence to perception and consciousness somehow analogous to motion picture cameras and projectors? Does the eye/brain actually “take” perceptual stills and somehow fuse them to give a sense of continuity and motion? No clear answer was forthcoming during their lifetimes.

There is a rare but dramatic neurological disturbance that a number of my patients have experienced during attacks of migraine, when they may lose the sense of visual continuity and motion and see instead a flickering series of “stills.” The stills may be clear-cut and sharp, and succeed one another without superimposition or overlap, but more commonly they are somewhat blurred, as with a too-long photographic exposure, and they persist for so long that each is still visible when the next “frame” is seen, and three or four frames, the earlier ones progressively fainter, are apt to be superimposed on each other. While the effect is somewhat like that of a film (albeit an improperly shot and presented one, in which each exposure has been too long to freeze motion completely and the rate of presentation too slow to achieve fusion), it also resembles some of E.J. Marey’s “chronophotographs” of the 1880s, in which one sees a whole array of photographic moments or time frames superimposed on a single plate.¹

I heard several accounts of such visual effects while working in the late 1960s with a large number of migraine patients, and when I wrote about this in my 1970 book Migraine, I noted that the rate of flickering in these episodes seemed to be between six and twelve per second. There might also be, in cases of migraine delirium, a flickering of kaleidoscopic patterns or hallucinations. (The flickering might then accelerate to restore the appearance of normal motion or of a continuously mod- ulated hallucination.) Finding no good accounts of the phenomenon in the medical literature—perhaps not entirely surprising, for such attacks are brief, rare, and not readily predicted or provoked—I used the term “cinematographic” vision for them; for patients always compared them to films run too slow.

This was a startling visual phenomenon, for which, in the 1960s, there was no good physiological explanation. But I could not help wondering then whether visual perception might in a very real way be analogous to cinematography, taking in the visual environment in brief, instantaneous, static frames, or “stills,” and then, under normal conditions, fusing these to give visual awareness its usual movement and continuity—a “fusion” which, seemingly, was failing to occur in the very abnormal conditions of these migraine attacks.

Such visual effects may also occur in certain seizures, as well as in intoxications (especially with hallucinogens such as LSD). And there are other visual effects that may occur. Moving objects may leave a smear or wake in the direction they move; images may repeat themselves; and afterimages may be greatly prolonged. I have experienced this myself, following the drinking of sakau, a hallucinogen and intoxicant popular in Micronesia. I described some of these effects in a journal, and later in my book The Island of the Colorblind:

Ghost petals ray out from a flower on our table, like a halo around it; when it is moved…it leaves a slight train, a visual smear…in its wake. Watching a palm waving, I see a succession of stills, like a film run too slow, its continuity no longer maintained.

I heard strikingly similar accounts in the late 1960s from some of my post-encephalitic patients, when they were “awakened,” and especially overexcited, by taking the drug L-DOPA. Some patients described cinematic vision; some described extraordinary “standstills,” sometimes hours long, in which not only visual flow was arrested, but the stream of movement, of action, of thought itself.

These standstills were especially severe with one patient, Hester Y. Once I was called to the ward because Mrs. Y. had started a bath, and there was now a flood in the bathroom. I found her standing completely motionless in the middle of the flood.

She jumped when I touched her, and said, “What happened?”

“You tell me,” I answered.

She said that she had started to run a bath for herself, and there was an inch of water in the tub…and then I touched her, and she suddenly realized that the tub must have run over and caused a flood. But she had been stuck, transfixed, at that perceptual moment when there was just an inch of water in the bath.

Such standstills showed that consciousness could be brought to a halt, stopped dead, for substantial periods, while automatic, nonconscious function—maintenance of posture or breathing, for example—continued as before.

Another striking example of perceptual standstill could be demonstrated with a common visual illusion, that of the Necker cube. Normally, when we look at this ambiguous perspective drawing of a cube, it switches perspective every few seconds, first seeming to project, then to recede, and no effort of will suffices to prevent this switching back and forth. The drawing itself does not change, nor does the retinal image. The switching is a cortical process, a conflict in consciousness itself, as it vacillates between alternative perceptual interpretations. This switching is seen in all normal subjects, and can be observed with functional brain imaging. But a post-encephalitic patient, during a standstill state, may see the same unchanging perspective for minutes or hours at a time.

The normal flow of consciousness, it seemed, could not only be fragmented, broken into small, snapshot-like bits, but could be suspended intermittently, for hours at a time.² I found this even more puzzling and uncanny than cinematic vision, for it has been accepted almost axiomatically since the time of William James that consciousness, in its very nature, is ever-changing and ever-flowing; but now my own clinical experience had to cast doubt on even this.

Thus I was primed to be further fascinated when, in 1983, Josef Zihl and his colleagues in Munich published a single, very fully described case of motion blindness: a woman who became permanently unable to perceive motion following a stroke. (The stroke had damaged the highly specific areas of the visual cortex which physiologists have shown in experimental animals to be crucial for motion perception.) In this patient, whom they call L.M., there were “freeze frames” lasting several seconds, during which Mrs. M. would see a prolonged, motionless image and be visually unaware of any movement around her, though her flow of thought and perception was otherwise normal. For example, Mrs. M. might begin a conversation with a friend standing in front of her, but not be able to see her friend’s lips moving or facial expressions changing. And if the friend moved around behind her, Mrs. M. might continue to “see” him in front of her, even though his voice now came from behind. She might see a car “frozen” a considerable distance from her, but find, when she tried to cross the road, that it was now almost upon her; she would see a “glacier,” a frozen arc of tea coming from the spout of the teapot, but then realize that she had overfilled the cup, and that there was now a puddle of tea on the table. Such a condition was utterly bewildering, and sometimes quite dangerous.

There are clear differences between cinematic vision and the sort of motion blindness described by Zihl; and perhaps between these and the very long visual and sometimes global freezes experienced by some post-encephalitic patients. These differences imply that there must be a number of different mechanisms or systems for the perception of visual motion and the continuity of visual consciousness—and this accords with evidence obtained from perceptual and psychological experiments. Some or all of these mechanisms may fail to work as they should in certain intoxications, some attacks of migraine, and some forms of brain damage—but can they also reveal themselves under normal conditions?

An obvious example springs to mind, which many of us have seen and perhaps puzzled over when watching evenly rotating objects—fans, wheels, propeller blades—or when walking past fences or palings, when the normal continuity of motion seems to be interrupted. Thus, occasionally, as I lie in bed looking up at my ceiling fan, the blades seem suddenly to reverse direction for a few seconds, and then to return equally suddenly to their original forward motion. Sometimes the fan seems to hover or stall, and sometimes to develop additional blades or dark bands broader than the blades.

It is similar to what happens when, in a film, the wheels of stagecoaches sometimes appear to be going slowly backward or scarcely moving. This wagon-wheel illusion, as it is called, reflects a lack of synchronization between the rate of filming and that of the rotating wheels. But I can have a real-life wagon-wheel illusion even when I look at my fan with the morning sun flooding into my room, bathing everything in a continuous, even light. Is there, then, some flickering or lack of synchronization in my own perceptual mechanisms—analogous, again, to the action of a movie camera?

Dale Purves and his colleagues at Duke University have explored wagon- wheel illusions in great detail, and they have confirmed that this type of illusion or misperception is universal among their subjects. Having excluded any other cause of discontinuity (intermittent lighting, eye movements, etc.), they conclude that the visual system processes information “in sequential episodes,” at the rate of three to twenty such episodes per second. Normally, these sequential images are experienced as an unbroken perceptual flow. Indeed, Purves et al. suggest, we may find movies convincing precisely because we ourselves break up time and reality much as a movie camera does, into discrete frames, which we then reassemble into an apparently continuous flow.

In Purves’s view, it is precisely this decomposition of what we see into a succession of moments that enables the brain to detect and compute motion; for all it has to do is to note the differing positions of objects between successive “frames,” and from these calculate the direction and speed of motion.³

2.

But this is not enough. We do not merely calculate movement as a robot might—we perceive it. We perceive motion, just as we perceive color or depth, as a unique qualitative experience that is vital to our visual awareness and consciousness. Something beyond our understanding occurs in the genesis of qualia, the transformation of an objective cerebral computation to a subjective experience. Philosophers argue endlessly over how these transformations occur, and whether we will ever be capable of understanding them. Neuroscientists, by and large, are content for the moment to accept that they do occur, and to devote themselves to finding the underlying basis or “neural correlates” of consciousness, starting from such elemental forms of consciousness as the perception of motion.

James dreamed of zoetropes as a metaphor for the conscious brain, Bergson of cinematography—but these were, of necessity, no more than tantalizing analogies and images. It has only been in the last twenty or thirty years that neuroscience could even start to address such issues as the neural basis of consciousness.

Indeed, from having been an almost untouchable subject before the 1970s, the neuroscientific study of consciousness has now become a central concern, one that engages scientists all over the world. Every level of consciousness is now being explored, from the most elemental perceptual mechanisms (mechanisms common to many animals besides ourselves) to the higher reaches of memory, imagery, and self-reflective consciousness.

It is now possible to monitor simultaneously the activities of a hundred or more individual neurons in the brain, and to do this in unanesthetized animals given simple perceptual and mental tasks. We can examine the activity and interactions of large areas of the brain by means of imaging techniques like functional MRIs and PET scans, and such non-invasive techniques can be used with human subjects, to see which areas of the brain are activated in complex mental activities.

In addition to physiological studies, there is the relatively new realm of computerized neural modeling, using populations or networks of virtual neurons, and seeing how these organize themselves in response to various stimuli and constraints.

All of these approaches, along with concepts not available to earlier generations, now combine to make the quest for the neural correlates of consciousness the most fundamental and exciting adventure in neuroscience today. A crucial innovation has been “population-thinking,” thinking in terms that take account of the brain’s huge population of neurons (a hundred billion or so), and the power of experience to differentially alter the strengths of connections between them, and to promote the formation of func-tional groups or constellations of neurons throughout the brain—groups whose interactions serve to categorize experience.⁴

Instead of seeing the brain as rigid, fixed in mode, programmed like a computer, there is now a much more biological and powerful notion of “experiential selection,” of experience literally shaping the connectivity and function of the brain (within genetic, anatomical, and physiological limits, of course).

Such a selection of neuronal groups (groups consisting of perhaps a thousand or so individual neurons), and its effect on shaping the brain over the lifetime of an individual, is seen as analogous to the role of natural selection in the evolution of species; hence Gerald M. Edelman, who was a pioneer in such thinking in the 1970s, speaks of “neural Darwinism.” J.P. Changeux, the French neuroscientist, is more concerned with the connections of individual neurons, and speaks of “the Darwinism of synapses.” Both Changeux and Edelman will soon publish highly readable, general accounts of their work.

William James himself always insisted that consciousness was not a “thing” but a “process.” The neural basis of these processes, for Edelman, is one of dynamic interaction between neuronal groups in different areas of the cortex (and between the cortex and the thalamus, and other parts of the brain). He speaks here of “re-entrant” (i.e., reciprocal) interactions, and sees consciousness as arising from the enormous number of such interactions between memory systems in the anterior parts of the brain and systems concerned with perceptual categorization in the posterior parts of the brain.

Other pioneers in the study of the neural basis of consciousness are Francis Crick (of the “double helix”) and his younger colleague Christof Koch, who, from their first collaborative work in the 1980s, have focused more narrowly on elementary visual perception and processes. Koch gives a detailed but vivid and personal history of their work, and of the search for the neural basis of consciousness generally, in his new book, The Quest for Consciousness. Mechanisms of visual consciousness, Crick and Koch feel, are an ideal starting point, because they are the most amenable to investigation at present, and can serve as a model for investigating and understanding higher and higher forms of consciousness.

In a synoptic paper called “A Framework for Consciousness,” published in Nature Neuroscience in February 2003, Crick and Koch speculate on the neural correlates of motion perception, how visual continuity is perceived or constructed, and, by extension, the seeming continuity of consciousness itself. They propose that “conscious awareness [for vision] is a series of static snapshots, with motion ‘painted’ on them…[and] that perception occurs in discrete epochs.”

I was startled when I first came across this passage a few months ago, because their formulation seemed to rest upon the same notion of consciousness that James and Bergson had intimated a century ago, and that had been in my mind since I first heard accounts of cinematic vision from my migraine patients in the 1960s. Here, however, was something more, a possible substrate for consciousness based in neuronal activity.

But the “snapshots” that Crick and Koch postulate are not uniform, like cinematic ones. The duration of successive snapshots, they feel, is not likely to be constant; moreover, the time of a snapshot for shape, say, may not coincide with one for color. While this “snapshotting” mechanism for visual sensory inputs is probably a fairly simple and automatic one, a relatively low-order neural mechanism, each visual percept must include a great number of visual attributes, all of which are bound together on some pre-conscious level.⁵ How, then, are the various snapshots “assembled” to achieve apparent continuity, and how do they reach the level of consciousness?

While a particular motion, for example, may be represented by neurons firing at a particular rate in the motion centers of the visual cortex, this is only the beginning of an elaborate process. To reach consciousness, this neuronal firing, or some higher representation of it, must cross a certain threshold of intensity and be maintained above it—consciousness, for Crick and Koch, is a threshold phenomenon. To do that, this group of neurons must engage other parts of the brain (usually in the frontal lobes) and ally itself with millions of other neurons to form a “coalition.” Such coalitions, they conceive, can form and dissolve in a fraction of a second, and involve reciprocal connections between the visual cortex and many other areas of the brain. These neural coalitions in different parts of the brain “talk” to one another in a continuous back-and-forth interaction. A single conscious visual percept may thus entail the parallel and mutually influencing activities of billions of nerve cells.

Finally, the activity of a coalition, or coalition of coalitions, if it is to reach consciousness, must not only cross a threshold of intensity, but must be held there for a certain time—roughly a hundred milliseconds. This is the duration of a “perceptual moment.”⁶

To explain the apparent continuity of visual consciousness, Crick and Koch suggest that the activity of the coalition shows “hysteresis,” that is, a persistence outlasting the stimulus. This notion is very similar, in a way, to the “persistence of vision” theories advanced in the nineteenth century.⁷ In his Physiological Optics of 1860, Hermann Helmholtz wrote, “All that is necessary is that the repetition of the impression shall be fast enough for the after-effect of one impression not to have died down perceptibly before the next one comes.” Helmholtz and his contemporaries supposed that this aftereffect occurred in the retina, but for Crick and Koch it occurs in the coalitions of neurons in the cortex. The sense of continuity, in other words, results from the continuous overlapping of successive perceptual moments. It may be that the forms of cinematographic vision I have described—with either sharply separated stills or blurred and overlapping ones—represent abnormalities of excitability in the coalitions, with either too much, or too little, hysteresis.⁸

Vision, in ordinary circumstances, is seamless and gives no indication of the underlying processes on which it depends. It has to be decomposed, experimentally or in neurological disorders, to show the elements that com- pose it. Thus it is decomposed vision—the flickering, perseverative, time-blurred images experienced in certain intoxications or severe migraines—which above all lends credence to the notion that consciousness is composed of discrete moments.

Whatever the mechanism, the fusing of discrete visual frames or snapshots is a prerequisite for continuity, for a flowing, mobile consciousness. Such a dynamic consciousness probably first arose in reptiles a quarter of a billion years ago. It seems probable that no such stream of consciousness exists in an amphibian, like a frog, which shows no active attention, and no visual following of events. The frog does not have a visual world or visual consciousness as we know it, only a purely automatic ability to recognize an insect-like object if this enters its visual field, and to dart out its tongue in response. It has been said that a frog’s vision is, in effect, no more than a fly-catching mechanism.⁹

If a dynamic, flowing consciousness allows, at the lowest level, a continuous, active scanning or looking, it allows, at a higher level, the interaction of perception and memory, of present and past. And such a “primary” consciousness, as Edelman puts it, is highly efficacious, highly adaptive, in the struggle for life.¹⁰

From such a relatively simple primary consciousness, we leap to human consciousness, with the advent of language and self-consciousness and an explicit sense of the past and the future. And it is this which gives a thematic and personal continuity to the consciousness of every individual. As I write I am sitting at a café on Seventh Avenue, watching the world go by. My attention and focus dart to and fro—a girl in a red dress goes by, a man walking a funny dog, the sun (at last!) emerging from the clouds. These are all events which catch my attention for a moment as they happen. Why, out of a thousand possible perceptions, are these the ones I seize upon? Reflections, memories, associations lie behind them. For consciousness is always active and selective—charged with feelings and meanings uniquely our own, informing our choices and interfusing our perceptions. So it is not just Seventh Avenue that I see, but my Seventh Avenue, marked by my own selfhood and identity.

Christopher Isherwood starts his Berlin Diary with an extended photographic simile: “I am a camera with its shutter open, quite passive, recording, not thinking. Recording the man shaving at the window opposite and the woman in the kimono washing her hair. Some day, all this will have to be developed, carefully printed, fixed.” But we deceive ourselves if we imagine that we can ever be passive, impartial observers. Every perception, every scene, is shaped by us, whether we intend it, know it, or not. We are the directors of the film we are making—but we are, equally, its subjects too: every frame, every moment, is us, is ours—our forms (as Proust says) are outlined in each one, even if we have no existence, no reality, other than this.

But how then do our frames, our momentary moments, hold together? How, if there is only transience, do we achieve continuity? Our passing thoughts, as James says (in an image which smacks of cowboy life in the 1880s) do not wander round like wild cattle. Each one is owned, our own, and bears the brand of this ownership, and each thought, in James’s words, is born an owner of the thoughts that went before, and “dies owned, transmitting whatever it realized as its Self to its own later proprietor.”

So it is not just perceptual moments, simple physiological moments—though these underlie everything else—but moments of an essentially personal kind, which seem to constitute our very being. Finally, then, we come around to Proust’s image, itself slightly reminiscent of photography (and even of Hume), that we consist entirely of “a collection of moments,” even though these flow into one another like Borges’s river.¹¹