Time and the Observer

The Where and When of Consciousness in the Brain

Daniel Dennett and Marcel Kinsbourne

Part I Part II

Part II

The model of the Cartesian Theater creates artifactual puzzle questions that cannot be answered, whereas for our model these questions cannot meaningfully arise. This can be seen by applying both models to other experiments that probe the limits of the distinction between perception and memory. A normally sufficient, but not necessary, condition for having experienced something is subsequent verbal report, and this is the anchoring case around which all the puzzle cases revolve. Suppose that although one's brain has registered--that is, responded to--(some aspects of) an event, something intervenes between that internal response and a subsequent occasion for verbal report. If there was no time or opportunity for an initial overt response of any sort, and if the intervening events prevent later overt responses (verbal or otherwise) from incorporating reference to some aspect(s) of the first event, this creates a puzzle question: were they never consciously perceived, or have they been rapidly forgotten?

Consider the familiar span of apprehension. Multiple letters are simultaneously briefly exposed. Some are identified. The rest were certainly seen. The subject insists they were there, knows their number, and has the impression that they were clear-cut and distinct. Yet he cannot identify them. Has he failed "really" to perceive them, or has he rapidly "forgotten" them? Or consider an acoustic memory span test, administered at a rapid rate, e.g., 4 items a second, such that the subject perforce cannot respond till the acoustic event is over. He identifies some, not others. Yet, subjectively he heard all of them clearly and equally well. Did he not genuinely perceive, or did he forget, the rest?

And if, under still more constricted circumstances such as metacontrast, the subject even lacks all conviction that the unrecallable items were there, should we take this judgment as conclusive grounds for saying he did not experience them, even if they prove to have left other contentful traces on his subsequent behavior? If there is a Cartesian Theater, these questions demand answers, since what gets into the Theater, and when, is supposedly determinate, even if the boundaries appear fuzzy due to human limitations of perception and memory.

Our Multiple Drafts model suggests a different perspective on these phenomena. When a lot happens in a short time, the brain may make simplifying assumptions (For a supporting view, see Marcel, 1983). In metacontrast, the first stimulus may be a disc and the second stimulus a ring that fits closely outside the space where the disc was displayed. The outer contour of a disc rapidly turns into the inner contour of a ring. The brain, initially informed just that something happened (something with a circular contour in a particular place), swiftly receives confirmation that there was indeed a ring, with an inner and outer contour. Without further supporting evidence that there was a disc, the brain arrives at the conservative conclusion that there was only a ring. Should we insist that the disc was experienced because if the ring hadn't intervened the disc would have been reported? Our model of how the phenomenon is caused shows that there is no motivated way of settling such border disputes: information about the disc was briefly in a functional position to contribute to a later report, but this state lapsed; there is no reason to insist that this state was inside the charmed circle of consciousness until it got overwritten, or contrarily, to insist that it never quite achieved this state. Nothing discernible to "inside" or "outside" observers could distinguish these possibilities.

In color phi, the processes that calculate that the second spot is green and that there is motion proceed roughly simultaneously (in different parts of the brain), and eventually contribute to the process that concludes that the red spot moved over and abruptly turned green on the way. That conclusion is achieved swiftly enough, in the standard case, to overwhelm or replace any competing contents before they can contribute to the framing of a report. So the subject says--and believes--just what Kolers and von Grünau report, and that is what the subject was conscious of. Was the subject also conscious a fraction of a second earlier of the stationary red spot? Ask him. If the interstimulus interval is made somewhat longer, there will come a point where the subject does report an experience of first a stationary red spot, then a green spot, and then a noticeably retrospective sense that the red spot ("must have") moved over and changed color. This experience has--as the subject will tell you--a quite different phenomenology. Apparent motion is experienced under such conditions, but it is obviously different from ordinary motion, and from swifter varieties of apparent motion. In what way is it different? In this way: the subject notices the difference! In this case it does seem to him as if he only later "realized" that there had been motion. But in cases in which this retrospective element is lacking it is still the case that the discrimination of motion-with-color-change is achieved after the colors and locations of the spots were discriminated--and there is no later process of "filling in" required.

In the cutaneous "rabbit," the shift in space (along the arm) is recorded over time by the brain. The number of taps is also recorded. Although in physical reality the taps were clustered at particular locations, the simplifying assumption is that they were distributed regularly across the space-time extent of the experience. The brain relaxes into this parsimonious though mistaken interpretation after the taps are registered, of course, and this has the effect of wiping out earlier (partial) interpretations of the taps, but some side effects of those interpretations (e.g., the interpretation that there were five taps, that there were more than two taps, etc.) may live on.

Although different attributes are indeed extracted by different neural facilities at different rates (e.g., location versus shape versus color), and although if asked to respond to the presence of each one in isolation, we would do so with different latencies, we perceive events, not a successively analyzed trickle of perceptual elements or attributes. As Efron remarks:

There are no grounds for an a priori assumption that the specificity of our awareness of an object of perception, or an aspect of that object, gradually increases or grows following the moment of its onset from the least specific experience to some maximally specific experience. . . . We do not, when first observing an object with central vision, fleetingly experience the object as it would appear with the most peripheral vision, then as it would appear with less peripheral vision . . . .Similarly, when we shift our attention from one object of awareness to another, there is no experience of 'growing' specificity of the new object of awareness--we just perceive the new object.(1967, p.721)

Is there an "optimal time of probing"? On the plausible assumption that after a while such narratives degrade rather steadily through both fading of details and self-serving embellishment (what I ought to have said at the party tends to turn into what I did say at the party), one can justify probing "as soon as possible" after the stimulus sequence of interest. At the same time, one wants to avoid interfering with the phenomenon by a premature probe. Since perception turns imperceptibly into memory, and "immediate" interpretation turns imperceptibly into rational reconstruction, there is no single, all-contexts summit upon which to direct one's probes. Any probe may elicit a narrative (or narrative fragment), and any such elicited narrative determines a "time line," a subjective sequence of events from the point of view of an observer. This time line may then be compared with other time lines, in particular with the objective sequence of events occurring in the brain of that observer. For the reasons discussed, these two time lines may not superimpose themselves in orthogonal registration. There may be order differences that induce kinks.

(figure 5 about here)

There is nothing metaphysically extravagant or challenging about this failure of registration (Snyder, 1988). It is no more mysterious or contra-causal than the realization that the individual scenes in movies are often shot out of sequence, or that when you read the sentence "Bill arrived at the party after Sally, but Jane came earlier than either of them." you learn of Bill's arrival before you learn of Jane's earlier arrival. The space and time of the representing is one frame of reference; the space and time of what the representing represents is another. But this metaphysically innocuous fact does nevertheless ground a fundamental metaphysical category: when a portion of the world comes in this way to compose a skein of narratives, that portion of the world is an observer. That is what it is for there to be an observer in the world, a something it is like something to be.

3. The Libet Controversies Re-examined

3.1. Libet's experiments allegedly showing "backwards referral"

Libet's experiments with direct cortical stimulation have provoked a great deal of discussion and speculation, in spite of the fact that they involved very few subjects, were inadequately controlled, and have not been replicated (Churchland, 1981, 1981b). No doubt they have attracted this unusual attention, in spite of their serious technical flaws, because, according to Libet, they demonstrate "two remarkable temporal factors":

(1) There is a substantial delay before cerebral activities, initiated by a sensory stimulus, achieve "neuronal adequacy" for eliciting any resulting conscious sensory experience.

(2) After neuronal adequacy is achieved, the subjective timing of the experience is (automatically) referred backwards in time, utilizing a "timing signal" in the form of the initial response of cerebral cortex to the sensory stimulus. (1981, p182)

The "timing signal" is the primary evoked potential in the cortex 10 to 20msec after peripheral stimulation. Libet suggests that the backwards referral is always "to" the timing signal.

Libet's model is Stalinesque: various editing processes occur prior to the moment of "neuronal adequacy", at which time a finished film is projected. How is it projected? Here Libet's account vacillates between an extreme view and a moderate view (cf. Honderich, 1985):

(a) backwards projection: it is projected backwards in time to some Cartesian theater where it actually runs in synch with the primary evoked potentials. (The primary evoked potentials, as "timing signals", serve rather like the slateboard used in film-making, showing the projector exactly how far back in time to project the experience.)

(b) backwards referral: it is projected in ordinary time, but it carries something like a postmark, reminding the viewer that these events must be understood to have occurred somewhat earlier. (In this case the primary evoked potentials serve simply as dates, which might be represented on the Cartesian screen by a title "On the eve of the Battle of Waterloo" or "New York City, summer, 1942")

Libet's own term is "referral" and he defends it by reminding us of the "long recognized and accepted" phenomenon of spatial referral, which might suggest the moderate reading. But since he also insists that this backwards referral is "remarkable" and a challenge to the theory of "psychoneural identity," he invites the extreme interpretation. Endnote 9 And this interpretation is further supported by a passage at the close of Libet 1981:

there is experimental evidence for the view that the subjective or mental "sphere" could indeed "fill in" spatial and temporal gaps. How else, for example, could one view the already mentioned enormous discrepancy that is known to exist between a subjective visual image and the configuration of neuronal activities that gives rise to the experience of the image? (p.196.) Endnote 10

Let us consider the details. "Neuronal adequacy," which Libet estimates to require up to 500msec of cortical activity, is determined by seeing how late, following initial stimulation, a direct cortical stimulation can interfere with the consciousness subsequently reported. Beyond that critical interval, a direct cortical stimulus would be reported by the subject to be a subsequent experience. (Having arrived too late for incorporation by the editing room into the "final print" of the first stimulus experience, it would appear in the next installment.) Libet's data suggest a tremendously variable editing window: "The conditioning cortical stimulus could be started more than 500msec following the skin pulse and still modify the skin sensation, although in most cases retroactive effects were not observed with S-C intervals greater than 200msec." (1981, p.185.) Libet is careful to define neuronal adequacy in terms of effect on subsequent unhurried verbal report: "the subject was asked to report, within a few seconds after the delivery of each pair of . . .stimuli" (1979, p.195), and he insists that "The timing of a subjective experience must be distinguished from that of a behavioral response (such as in reaction time), which might be made before conscious awareness develops . . ." (1979, p.193)

This proviso permits him to defend a rival interpretation of Churchland's data. Churchland (1981) attempted to discredit Libet's claim about the long rise time to "neuronal adequacy" for consciousness, by asking subjects in an experiment to say "go" as soon as they were conscious of a skin stimulus like those used by Libet. She reported a mean response time over 9 subjects of 358msec, which, she argued, showed that the subjects must have achieved neuronal adequacy by the 200msec mark at the latest (allowing time for the production of a verbal response). Libet's reply is Stalinesque: a verbal reaction can be unconsciously initiated. "There is nothing magical or uniquely informative when the motor response is a vocalization of the word 'go' instead of the more usual one of a finger tapping a button . . . The ability to detect a stimulus and to react to it purposefully, or be psychologically influenced by it, without any reportable conscious awareness of the stimulus, is widely accepted." (1981, p.187-8) And to the objection, "But what did Churchland's subjects think they were doing, if not saying, as requested, just when they were conscious of the stimulus?" Libet could give the standard Stalinesque reply: they did indeed eventually become conscious of the stimulus, but by then, their verbal report had already been initiated. Endnote 11

For this reason Libet rejects reaction time studies such as Churchland's as having "an uncertain validity as a primary criterion of a subjective experience," (1981, p.188). He favors letting the subject take his time: "The report is made unhurriedly within a few seconds after each trial, allowing the subject to introspectively examine his evidence." (p.188) How, then, can he deal with the rival prospect that this leisurely pace gives the Orwellian revisionist in the brain plenty of time to replace the veridical memories of consciousness with false memories?

Reporting after the trial of course requires that processes of short-term memory and recallability be operative, but this presents no difficulty for subjects with no significant defects in these abilities. (p.188)

This begs the question against the Orwellian, who is prepared to explain a variety of effects as the result of normal mis-remembering or hallucinatory recall, in which a prior, real event in consciousness is obliterated and replaced by subsequent memories. (For related discussions, see Allport, 1988, p.171-76; Bisiach, 1988, pp.110-12.) Has Libet let the stew cook too long, or has Churchland sampled it too soon? If Libet wants to claim a privileged status for his choice of probe time, he must be prepared to combat the counterarguments.

Libet comes close to pleading nolo contendere: "Admittedly, a report of relative timing order cannot, in itself, provide an indicator of the 'absolute' time (clock-time) of the experience: as suggested, there is no known method to achieve such an indicator." (1981, p.188), echoing his earlier remark that there seemed to be "no method by which one could determine the absolute timing of a subjective experience." 1979, p.193. What Libet misses, however, is the possibility that this is because there is no such moment of absolute time. (Cf. Harnad, unpublished, and 1987)

Churchland too falls prey to the failure to distinguish time represented from time of representing, in her criticisms (1981, 1981b):

The two hypotheses differ essentially on just when the respective sensations were felt [our italics]. (1981, p177)

Even if it be supposed that the sensations arising from the simultaneous skin and LM [medial lemniscus] sensations are felt at exactly the same time [our italics], the delay in neuronal adequacy for skin stimuli may well be an artifact of the setup. (1981b, p494)

Suppose that all such artifacts were eliminated, and still the sensations are "felt at exactly the same time". Will this mean that there is a time t such that stimulus 1 is felt at t and stimulus 2 is felt at t (the anti-materialist prospect) or only that stimulus 1 and stimulus 2 are felt as (experienced as) simultaneous? Churchland doesn't discourage the inference that Libet's findings, if vindicated, would wreak havoc (as he claims) on materialism. Elsewhere, however, she correctly notes that "intriguing as temporal illusions are, there is no reason to suppose there is something preternatural about them, and certainly there is nothing which distinguishes them from spatial illusions or motion illusions as uniquely bearing the benchmark of a non-physical origin." (1981, p178) This could only be the case if temporal illusions were phenomena in which time was misrepresented; if the misrepresentings take place at the "wrong" times, something more revolutionary is afoot.

Where does this leave Libet's experiments with cortical stimulation? As an interesting but inconclusive attempt to establish something about how the brain represents temporal order. Primary evoked potentials may somehow serve as specific reference-points for neural representations of time, although Libet has not shown this, as Churchland's technical criticisms make clear. Alternatively, the brain keeps its representations of time more labile. We don't represent seen objects as existing on the retina, but rather at various distances in the external world; why should the brain not also represent events as happening when it makes the most "ecological" sense for them to happen? When we are engaged in some act of manual dexterity, "fingertip time" should be the standard; when we are conducting an orchestra, "ear time" might capture the registration. "Primary cortical time" might be the default standard (rather like Greenwich Mean Time for the British Empire), a matter, however, for further research.

The issue has been obscured by the fact that both proponent and critic have failed to distinguish consistently between time of representing and time represented. They talk past each other, with Libet adopting a Stalinesque position and Churchland making the Orwellian countermoves, both apparently in agreement that there is a fact of the matter about exactly when (in "absolute" time as Libet would put it) a conscious experience happens. Endnote 12

3.2. Libet's claims about the "subjective delay" of consciousness of intention

The concept of the absolute timing of an experience is exploited in Libet's later experiments with "conscious intentions," in which he seeks to determine their absolute timing experimentally by letting the subjects, who alone have direct access (somehow) to their experiences, do self-timing. He asked subjects to look at a clock (a spot of light circling on an oscilloscope) while they experience consciously intending, and to make a judgment about the position on the clock of the spot at the onset of intention, a judgment they can later, at their leisure, report.

Libet is clearer than most of his critics about the importance of keeping content and vehicle distinguished: "One should not confuse what is reported by the subject with when he may become introspectively aware of what he is reporting." (Libet 85, p.559) He recognizes (p.560), moreover, that a judgment of simultaneity need not itself be simultaneously arrived at or rendered; it might mature over a long period of time (consider, for instance, the minutes it may take the stewards at the race track to develop and then examine the photo-finish picture on which they eventually base their judgment of the winner or a dead heat).

Libet gathered data on two time series:

the objective series, which includes the timing of the external clock and the salient neural events: the readiness potentials (RPs) and the electromyograms (EMGs).

the subjective series (as later reported), which consists of mental imagery, memories of any pre-planning, and crucially of a single benchmark datum for each trial: a simultaneity judgment of the form: my conscious intention (W) began simultaneously with the clock spot in position P.

Libet seems to have wanted to approximate the elusive acte gratuit discussed by the existentialists (e.g., Gide, 1948, Sartre, 1943), the purely motiveless--and hence in some special sense "free"--choice, and as several commentators have pointed out (Breitmeyer 1985, Bridgeman 1985, Danto 1985, Jung 1985, Latto 1985) such highly unusual actions (which might be called acts of deliberate pseudo-randomness) are hardly paradigms of "normal voluntary acts" (Libet 1987, p.784). But has he in any event isolated a variety of conscious experiences, however characterized, that can be given absolute timing by such an experimental design?

He claims that when conscious intentions to act (at least of his special sort) are put into registration with the brain events that actually initiate the acts, there is an offset: consciousness of intention lags 300-500msec behind the relevant brain events. This does look ominous to anyone committed to the principle that "our conscious decisions" control our bodily motions. It looks as if we are located in Cartesian theaters where we are shown, with a half-second tape delay, the real decision-making that is going on elsewhere (somewhere we aren't). We are not quite "out of the loop" (as they say in the White House), but since our access to information is thus delayed, the most we can do is intervene with last-moment "vetoes" or "triggers." One who accepts this picture might put it this way:

Downstream from (unconscious) command headquarters, I take no real initiative, am never in on the birth of a project, but do exercise a modicum of executive modulation of the formulated policies streaming through my office.

This picture is compelling but incoherent. For one thing, such a "veto" would itself have to be a "conscious decision", it seems, and hence ought to require its own 300-500msec cerebral preparation--unless one is assuming outright Cartesian dualism. (See MacKay, 1985, who makes a related point). Setting that problem aside, Libet's model, as before, is Stalinesque, and the obvious Orwellian alternative is raised by Jasper (1985), who notes that both epileptic automatisms and behaviors occurring under the effect of such drugs as scopolamine show that "brain mechanisms underlying awareness may occur without those which make possible the recall of this awareness in memory afterward." Libet concedes that this "does present a problem, but was not experimentally testable." (p.560)Endnote 13

Given this concession, is the task of fixing the absolute micro-timing of consciousness ill-conceived? Neither Libet nor his critics draw that conclusion. Libet, having carefully distinguished content from vehicle--what is represented from when it is represented--nonetheless tries to draw inferences from premises about what is represented to conclusions about the absolute timing of the representing in consciousness (Cf. Salter, 1989). Wasserman (1985) sees the problem: "The time when the external objective spot occupies a given clock position can be determined easily, but this is not the desired result." But he then goes on to fall in the Cartesian trap: "What is needed is the time of occurrence of the internal brain-mind representation of the spot."

" The time of occurrence" of the internal representation? Occurrence where? There is essentially continuous representation of the spot (representing it to be in various different positions) in various different parts of the brain, starting at the retina and moving up through the visual system. The brightness of the spot is represented in some places and times, its location in others, and its motion in still others. As the external spot moves, all these representations change, in an asynchronous and spatially distributed way. Where does "it all come together at an instant in consciousness"? Nowhere. Wasserman correctly points out that the task of determining where the spot was at some time in the subjective sequence is itself a voluntary task, and initiating it presumably takes some time. This is difficult not only because it is in competition with other concurrent projects (as stressed by Stamm, p.554), but also because it is unnatural--a conscious judgment of temporality of a sort that does not normally play a role in behavior control, and hence has no natural meaning in the sequence. The process of interpretation that eventually fixes the judgment of subjective simultaneity is itself an artifact of the experimental situation, and changes the task, therefore telling us nothing of interest about the actual timing of normal representation vehicles anywhere in the brain.

Stamm likens the situation to Heisenbergian uncertainty: "self-monitoring of an internal process interferes with that process, so that its precise measurement is impossible." (p.554)

This observation betrays a commitment to the mistaken idea that there is an absolute time of intersection, "precise measurement" of which, alas, is impossible for Heisenbergian reasons (see also Harnad, 1989). This could only make sense on the assumption that there is a particular privileged place where the intersection matters.

The all too natural vision that we must discard is the following: somewhere deep in the brain an act-initiation begins; it starts out as an unconscious intention, and slowly makes its way to the theater, picking up clarity and power as it goes, and then, at an instant, t, it bursts on stage, where a parade of visual spot-representations are marching past, having made their way slowly from the retina, getting clothed with brightness and location as they moved. The audience or I is given the task of saying which spot-representation was "on stage" exactly when the conscious intention made its bow. Once identified, this spot's time of departure from the retina can be calculated, as well as the distance to the theater and the transmission velocity. In that way we can determine the exact moment at which the conscious intention occurred in the Cartesian theater.

Some have thought that although that vision is incoherent, that does not require one to give up the idea of absolute timing of experiences. There is an alternative family of models for the onset of consciousness that avoid the preposterousness of the Cartesian centered brain. Couldn't consciousness be a matter not of arrival at a point but rather a matter of a representation exceeding some threshold of activation over the whole cortex or large parts thereof? On this model, an element of content becomes conscious at some time t, not by entering some functionally defined and anatomically located system, but by changing state right where it is: by acquiring some property or by having the intensity of one of its properties boosted above some criterial level.

The idea that content becomes conscious not by entering a subsystem, but by the brain's undergoing a state change of one sort or another has much to recommend it (see, e.g., Kinsbourne, 1988, Neumann, 1990, Crick and Koch, 1990). Moreover, the simultaneities and sequences of such mode-shifts can presumably be measured by outside observers, providing, in principle, a unique and determinate sequence of contents attaining the special mode. But this is still the Cartesian theater if it is claimed that the real ("absolute") timing of such mode-shifts is definitive of subjective sequence. The imagery is different, but the implications are the same. Conferring the special property that makes for consciousness at an instant is only half the problem; discriminating that the property has been conferred at that time is the other, and although scientific observers with their instruments may be able to do this with microsecond accuracy, how is the brain to do this? We human beings do make judgments of simultaneity and sequence of elements of our own experience, some of which we express, so at some point or points in our brains the corner must be turned from the actual timing of representations to the representation of timing. This is a process that takes effort in one way or another (Gallistel, 1990), and wherever and whenever these discriminations are made, thereafter the temporal properties of the representations embodying those judgments are not constitutive of their content.

Suppose that a succession of widely spread activation states, with different contents, sweeps over the cortex. The actual, objectively measured simultaneities and sequences on this broad field are of no functional relevance unless they can also be accurately detected by mechanisms in the brain. What would make this sequence the stream of consciousness if the brain could not discern the sequence? What matters, once again, is not the temporal properties of the representings, but the temporal properties represented, something determined by how they are "taken" by subsequent processes in the brain.

3.3. Grey Walter's experiment: a better demonstration of the central contention of the Multiple Drafts model

It was noted above that Libet's experiment created an artificial and difficult judgment task, which robbed the results of the hoped-for significance. This can be brought out more clearly by comparing it to a similar experiment by Grey Walter (1963), with patients in whose motor cortex he had implanted electrodes. He wanted to test the hypothesis that certain bursts of recorded activity were the initiators of intentional actions. So he arranged for each patient to look at slides from a carousel projector. The patient could advance the carousel at will, by pressing the button on the controller. (Note the similarity to Libet's experiment: this was a "free" decision, timed only by an endogenous rise in boredom, or curiosity about the next slide, or distraction, or whatever.) Unbeknownst to the patient, however, the controller button was a dummy, not attached to the slide projector at all. What actually advanced the slides was the amplified signal from the electrode implanted in the patient's motor cortex.

One might suppose that the patients would notice nothing out of the ordinary, but in fact they were startled by the effect, because it seemed to them as if the slide projector was anticipating their decisions. They reported that just as they were "about to" push the button, but before they had actually decided to do so, the projector would advance the slide--and they would find themselves pressing the button with the worry that it was going to advance the slide twice! The effect was strong, according to Grey Walter's account, but apparently he never performed the dictated follow-up experiment: introducing a variable delay element to see how large a delay had to be incorporated into the triggering in order to eliminate the "precognitive carousel" effect.

An important difference between Grey Walter's and Libet's design is that the judgment of temporal order that leads to surprise in Grey Walter's experiment is part of a normal task of behavior monitoring. In this regard it is like the temporal order judgments by which our brains distinguish moving left-to-right from moving right-to-left, rather than "deliberate, conscious" order judgments. The brain in this case has set itself to "expect" visual feedback on the successful execution of its project of advancing the carousel, and the feedback arrives earlier than expected, triggering an alarm. This could show us something important about the actual timing of content vehicles and their attendant processes in the brain, but it would not, contrary to first appearances, show us something about the "absolute timing of the conscious decision to change the slide."

Suppose, for instance, that an extension of Grey Walter's experiment showed that a delay as long as 300msec (as implied by Libet) had to be incorporated into the implementation of the act in order to eliminate the subjective sense of precognitive slide-switching. What such a delay would in fact show would be that expectations set up by a decision to change the slide are tuned to expect visual feedback 300msec later, and to report back with alarm under other conditions. The fact that the alarm eventually gets interpreted in the subjective sequence as a perception of misordered events (change before button push) shows nothing about when in real time the consciousness of the decision to press the button first occurred. The sense the subjects reported of not quite having had time to "veto" the initiated button push when they "saw the slide was already changing" is a natural interpretation for the brain to settle on (eventually) of the various contents made available at various times for incorporation into the narrative. Was this sense already there at the first moment of consciousness of intention (in which case the effect requires a long delay to "show time" and is Stalinesque) or was it a retrospective reinterpretation of an otherwise confusing fait accompli (in which case it is Orwellian)? This question should no longer seem to demand an answer.

Conclusion

The Multiple Drafts model has many other implications for scientific theories of consciousness (Dennett, 1991), but our main conclusion in this paper is restricted to temporal properties of experience: the representation of sequence in the stream of consciousness is a product of the brain's interpretive processes, not a direct reflection of the sequence of events making up those processes. Indeed, as Ray Jackendoff has pointed out to us, what we are arguing for in this essay is a straightforward extension to experience of time of the common wisdom about experience of space; the representation of space in the brain does not always use space-in-the-brain to represent space, and the representation of time in the brain does not always use time-in-the-brain. It may be objected that the arguments presented here are powerless to overturn the still obvious truth that our experiences of events occur in the very same order as we experience them to occur. If someone thinks the thought "One, two, three, four, five," his thinking "one" occurs before his thinking "two" and so forth. The example does illustrate a thesis that is true in general, and does indeed seem unexceptioned so long as we restrict our attention to psychological phenomena of "ordinary," macroscopic duration. But the experiments we selected for discussion are concerned with events that were constricted by unusually narrow time-frames of a few hundred milliseconds. At this scale, we have argued, the standard presumption breaks down.

It might be supposed, then, that we are dealing only with special cases. These limiting cases may interestingly reveal how the brain deals with informational overload, but, one might suggest, they are unrepresentative of the brain's more usual manner of functioning. The contrary is the case, however, as might be anticipated, in view of the brain's well-known propensity for applying a limited number of basic manners of proceeding across a wide range of situations. The processes of editorial revision that are dramatically revealed in the time-pressured cases continue indefinitely as the brain responds to the continued demands of cognition and control. For instance, as time passes after an event has occurred, that event may be recalled to episodic memory, but to an ever more limited extent. After some days, an occurrence that may have unrolled over minutes or more is remembered within as restricted a time frame as those we have been discussing. Such memories present not as randomly blurry or depleted versions, but as internally coherent simplified renderings of what are taken to be the most important elements. Temporal succession is typically an early victim of this reorganization of the event, sacrificed in favor of (apparently) more useful information (as instanced in the phi phenomenon).

We perceive--and remember--perceptual events, not a successively analyzed trickle of perceptual elements or attributes locked into succession as if pinned into place on a continuous film. Different attributes of events are indeed extracted by different neural facilities at different rates, (e.g. location versus shape versus color) and people, if asked to respond to the presence of each one in isolation, would do so with different latencies, depending on which it was, and on other well-explored factors. The relative timing of inputs plays a necessary role in determining the information or content in experience, but it is not obligatorily tied to any stage or point of time during central processing. How soon we can respond to one in isolation, and how soon to the other, does not exactly indicate what will be the temporal relationship of the two in percepts that incorporate them both.

There is nothing theoretically amiss with the goal of acquiring precise timing information on the mental operations or informational transactions in the brain (Wasserman and Kong, 1979). It is indeed crucial to developing a good theory of the brain's control functions to learn exactly when and where various informational streams converge, when "inferences" and "matches" and "bindings" occur. But these temporal and spatial details do not tell us directly about the contents of consciousness. The temporal sequence in consciousness is, within the limits of whatever temporal control window bounds our investigation, purely a matter of the content represented, not the timing of the representing.

References

Allport, A., 1988, "What Concept of Consciousness?" in Marcel and Bisiach, 1988, pp. 159-82.

Barlow, H. B., and Levick, W. R., 1965, "Mechanisms of pattern selectivity in retina," Journal of Physiology, 178, 477.

Bisiach, E., 1988, "The Haunted Brain and Consciousness," in Marcel and Bisiach, Consciousness in Contemporary Science, Cambridge: Cambridge Univ. Press, pp. 101-20.

Blakemore, C., 1976, Mechanics of the Mind, Cambridge Univ. Press.

Breitmeyer, B. G., 1984, Visual Masking, Oxford: Clarendon Press.

Breitmeyer, B. G., 1985, "Problems with the psychophysics of intention," Behavioral and Brain Sciences, 8, p.539-40.

Bridgeman, B., 1985, "Free Will and the functions of consciousness," Behavioral and Brain Sciences, 8, p.540.

Calvin, W., 1990, The Cerebral Symphony: Seashore Reflections on the Structure of Consciousness, New York: Bantam

Churchland, P. S., 1981, "On the alleged backwards referral of experiences and its relevance to the mind-body problem," Philosophy of Science, 48, pp.165-181.

Churchland, P. S., 1981b, "The timing of sensations: reply to Libet," Philosophy of Science, 48, pp.492-97.

Crick, F. and Koch, C., 1990, "Towards a Neurobiological Theory of Consciousness," in A. R. Damasio, ed., Seminars in the Neurosciences, Philadelphia: W. B. Saunders, pp.263-75.

Danto, A. 1985, "Consciousness and motor control," Behavioral and Brain Sciences, 8, p.540-41.

Dennett, D. C. 1975, "Are Dreams Experiences?" Phil. Review, 73, 151-71.

Dennett, D. C., 1978, "Skinner Skinned," ch. 4 (pp. 53-70), in D. C. Dennett, Brainstorms: Philosophical Essays on Mind and Psychology, Montgomery, VT: Bradford Books.

Dennett, D. C., 1979, review of Popper and Eccles 1977, J.Phil., p.91-97.

Dennett, D. C., 1982, "How to study human consciousness empirically," Synthese, 53, pp.159-80.

Dennett, D. C., forthcoming, Consciousness Explained, Boston: Little, Brown.

Descartes, R. 1664, Traité de l'Homme, Paris.

Dixon, N. F. 1986, "On Private Events and Brain Events," Behavioral and Brain Sciences, 9,pp29-30.

Eccles, J. C., 1985, "Mental summation: the timing of voluntary intentions by cortical activity," Behavioral and Brain Sciences, 8, p. 542-3.

Efron, R., 1967, "The duration of the present," Proc NY Acad Sci., 138, pp 713-29.

Erdelyi, M. H., 1986, "Experimental Indeterminacies in the Dissociation Paradigm of Subliminal Perception," Behavioral and Brain Sciences, 9, pp.30-31.

Farrell, B. A., 1950, "Experience," Mind, 59, pp. 170-98

Gallistel, C. R., 1990, The Organization of Learning, Cambridge, MA: MIT Press.

Geldard, F.A., 1977, "Cutaneous stimuli, vibratory and saltatory," Journal of Investigative Dermatology, 69, pp.83-87.

Geldard, F.A., and Sherrick, C.E., 1972, "The Cutaneous 'rabbit': a perceptual illusion," Science, 178, pp.178-9.

Geldard, F.A., and Sherrick, C. E., 1983,"The cutaneous saltatory area and its presumed neural base," Perception and Psychophysics, 33, pp.299-304.

Geldard, F.A., and Sherrick, C. E., 1986, "Space, Time and Touch," Scientific American, 254, pp.90-95.

Gide, A., 1948, Journal des Faux Monnayeurs, Paris: Gallimard

Glynn, I. M., 1990, "Consciousness and Time," Nature, 348, p.477-79.

Goodman, N., 1978, Ways of Worldmaking, Hassocks, Sussex: Harvester.

Grey Walter, W., 1963, presentation to the Ostler Society, Oxford University.

Harnad, S., 1982, "Consciousness: an Afterthought," Cognition and Brain Theory, 5, pp.29-47

Harnad, S., 1989, "Editorial commentary," Behavioral and Brain Sciences, 12, p. 183.

Harnad, S., (unpublished), "Conscious events cannot be localized in time."

Hawking, S., 1988, A Brief History of Time, New York: Bantam.

Hoffman, R. E., and Kravitz, R. E., 1987, "Feedforward action regulation and the experience of will," Behavioral and Brain Sciences, 10, p.782-3.

Holender, D., 1986, "Semantic Activation without Conscious Identification," Behavioral and Brain Sciences, 9,pp.1-66.

Honderich, T. 1984, "The time of a conscious sensory experience and mind-brain theories," J. Theoretical Biology, 110, p.115-29.

Hoy, R. C., 1982, "Ambiguities in the Subjective Timing of Experiences Debate," Philosophy of Science, 49, pp.254-62.

Jasper, H. H., 1985, "Brain mechanisms of conscious experience and voluntary action," Behavioral and Brain Sciences, 8, pp.543-4.

Julesz, B, 1971, Foundations of Cyclopean Perception, Chicago: Univ of Chicago Press.

Jung, R., 1985, "Voluntary intention and conscious selection in complex learned action," Behavioral and Brain Sciences, 8, pp. 544-5.

Kinsbourne, M., 1988, "Integrated Field Theory of Consciousness," in A. J. Marcel and E. Bisiach, eds., Consciousness in Contemporary Science, Oxford: Oxford Univ. Press.

Kolers, P. A., 1972, Aspects of Motion Perception, London: Pergamon Press.

Kolers, P. and von Grünau, M., 1976, "Shape and color in apparent motion," Vision Research, 16, pp.329-35.

Latto, R., 1985, "Consciousness as an experimental variable: problems of definition, practice, and interpretation," Behavioral and Brain Sciences, 8, pp.545-6

Libermann, A. M., 1970, "The Grammar of Speech and Language," Cognitive Psychology, 1, pp.301-323.

Libet, B., 1965, "Cortical activation in conscious and unconscious experience," Perspectives in Biology and Medicine, 9, pp. 77-86.

Libet, B., Wright, E.W., Feinstein, B., and Pearl, D. K., 1979, "Subjective referral of the timing for a conscious sensory experience," Brain, 102, pp.193-224.

Libet, B., 1981, "The experimental evidence for subjective referral of a sensory experience backwards in time: reply to P. S. Churchland," Philosophy of Science, 48, pp.182-97.

Libet, B., 1982, "Brain stimulation in the study of neuronal functions for conscious sensory experiences," Human Neurobiology, 1, 235-42.

Libet, B., 1985, "Unconscious cerebral initiative and the role of conscious will in voluntary action," Behavioral and Brain Sciences, 8, p529-566.

Libet, B., 1985b, "Subjective antedating of a sensory experience and mind-brain theories: reply to Honderich," J.Theoretical Biology, 114, p.563-70.

Libet, B., 1987, "Are the mental experiences of will and self-control significant for the performance of a voluntary act?" Behavioral and Brain Sciences, 10, p.783-786.

Libet, B., 1989, "The timing of a subjective experience," Behavioral and Brain Sciences, 12, p.183-85.

MacKay, D. M., 1985, "Do we 'control' our brains?" Behavioral and Brain Sciences, 8, p[.546-7.

Marcel, A. J., 1983, "Conscious and Unconscious Perception: An Approach to the Relations between Phenomenal Experience and Perceptual Processes," Cognitive Psychology, 15, pp.238-300.

Marcel, A. J., 1986, "Consciousness and Processing: Choosing and Testing a Null Hypothesis," Behavioral and Brain Sciences, 9,pp 40-41.

Mellor, H., 1981, Real Time, Cambridge: Cambridge Univ. Press.

Merikle, P M. and Cheesman, J., 1986, "Consciousness is a 'Subjective' State," Behavioral and Brain Sciences, 9, p42.

Minsky, M., 1985, The Society of Minds, New York: Simon and Schuster.

Nagel, T., 1974, "What is it like to be a bat?" Phil. Review, 83,, pp.435-445.

Neisser, U. 1967, Cognitive Psychology, New York: Appleton-Century-Crofts.

Neisser, U., 1981, "John Dean's memory: a case study," Cognition, 9, pp.1-22.

Neumann, O., 1990, "Some aspects of phenomenal consciousness and their possible functional correlates," presented at the conference, "The phenomenal mind - how is it possible and why is it necessary?" Center for Interdisciplinary Research (ZiF), Bielefeld, May 14-17, 1990.

Newell, A., Rosenbloom, P. S., and Laird, J. E., 1989, "Symbolic Architectures for Cognition," in M. Posner, ed., Foundations of Cognitive Science, MIT Press, pp.93-132.

Pagels, H., 1988, The Dreams of Reason, New York: Simon & Schuster.

Penfield, W., and Jasper, H., 1954, Epilepsy and the Functional Anatomy of the Human Brain, Boston: Little Brown.

Penrose, R., 1989, The Emperor's New Mind: Concerning computers, minds, and the laws of physics, Oxford: Oxford Univ. Press.

Pöppel, E., 1985, Grenzen des Bewusstseins, Stuttgart: Deutsche Verlags-Anstal, translated as Mindworks: Time and Conscious Experience, New York: Harcourt Brace Jovanovich, 1988.

Popper, K R. and Eccles, J.C., 1977, The Self and Its Brain, Berlin: Springer-Verlag.

Pylyshyn, Z., 1979, "Do mental events have durations?" Behavioral and Brain Sciences, 2, pp.277-8.

Reingold, E. M., and Merikle, P. M., 1990, "On the Inter-relatedness of Theory and Measurement in the Study of Unconscious Processes," Mind and Language, 5, pp. 9-28.

Ringo, J. L., 1985, "Timing volition: questions of what and when about W," Behavioral and Brain Sciences, 8, pp.550-51.

Ryle, G., 1949, The Concept of Mind, London: Hutchison

Salter, D., 1989, "Voluntary process and the readiness potential: asking the right questions," Behavioral and Brain Sciences, 12, pp. 181-2.

Sarris, V., 1989, "Max Wertheimer on seen motion: theory and evidence," Psychological Research, 51, pp.58-68.

Sartre, J. P., 1943, L'Etre et le Néant, Paris: Gallimard.

Snyder, D. M., 1988, "On the time of a conscious peripheral sensation," J. Theoretical Biology, 130, 253-54.

Sperling, G. 1960, "The Information available in brief visual presentations," Psychological Monographs, 74, No. 11,

Stamm, J. S., 1985, "The uncertainty principle in psychology," Behavioral and Brain Sciences, 8, p.553-4.

Uttal, W. R., 1979, "Do central nonlinearities exist?" Behavioral and Brain Sciences, 2, p.286.

Van der Waals, H. G., and Roelofs, C. O., 1930, "Optische Scheinbewegung," Zeitschrift für Psychologie und Physiologie des Sinnesorgane, 114, pp.241-88, (1931) 115, pp.91-190.

Velmans, M., 1991, "Is Human Information Processing Conscious?" Behavioral and Brain Sciences,

Vendler, Z., 1972, Res Cogitans, Cornell: Cornell Univ. Press.

Vendler, Z., 19784, The Matter of Minds, Oxford: Clarendon Press.

Welch, R. B., 1978, Perceptual Modification: Adapting to Altered Sensory Environments, New York: Academic Press.

Wasserman, G. S. and Kong, K-L, 1979, "Absolute Timing of Mental Activities," Behavioral and Brain Sciences, 2, pp.243-304

Wasserman, G. S., 1985, "Neural/mental chronometry and chronotheology," Behavioral and Brain Sciences, 8, p.556-60.

Wertheimer, M. 1912, "Experimentelle Studien über das Sehen von Bewegung," Zeitschrift für Psychologie, 61, pp.161-265.

Endnotes
1. The original draft of this essay was written while the authors were supported by the Rockefeller Foundation as Scholars in Residence at the Bellagio Study Center, Villa Serbelloni, Bellagio, Italy, April, 1990. We are grateful to Kathleen Akins, Peter Bieri, Edoardo Bisiach, William Calvin, Patricia Churchland, Robert Efron, Stephan Harnad, Douglas Hofstadter, Tony Marcel, Odmar Neumann, Jay Rosenberg, and David Rosenthal for comments on subsequent drafts.

2. A philosophical exception is Vendler (1972, 1984) who attempts to salvage Cartesian dualism. A scientific exception is Eccles (E.g., Popper and Eccles, 1977).

3. What about the prospect of a solitary Robinson-Crusoe scientist who performs all these experiments wordlessly on himself? Would the anomalies be apparent to this lone observer? What about reconstructing these experiments with non-language-using animals? Would we be inclined to interpret the results in the same way? Would we be justified? These are good questions, but their answers are complicated, and we must reserve them for another occasion.

4.Such a "postmark" can in principle be added to a vehicle of content at any stage of its journey; if all materials arriving at a particular location come from the same place, by the same route at the same speed, their "departure time" from the original destination can be retroactively stamped on them, by simply subtracting a constant from their arrival time at the way station. This is an engineering possibility that is probably used by the brain for making certain automatic adjustments for standard travel times.

5."The essence of much of the research that has been carried out in the field of sensory coding can be distilled into a single, especially important idea--any candidate code can represent any perceptual dimension; there is no need for an isomorphic relation between the neural and psychophysical data. Space can represent time, time can represent space, place can represent quality, and certainly, nonlinear neural functions can represent linear or nonlinear psychophysical functions equally well." (Uttal, 1979) This is a widely acknowledged idea, but, as we will show, some theorists (mis-)understand it by tacitly reintroducing the unnecessary "isomorphism" in a dimly imagined subsequent translation or "projection" in consciousness.

6.Cf. Pylyshyn, 1979: "No one . . . is disposed to speak literally of such physical properties of a mental event as its color, size, mass and so on . . .though we do speak of them as representing (or having the experiential content of) such properties. For instance, no one would not properly say of a thought (or image) that it was large or red, but only that it was a thought about something large or red (or that it was an image of something large or red) . . . It ought to strike one as curious, therefore, that we speak so freely of the duration of a mental event."

7. P. S. Churchland (1981, p172) notes a difference between "masking in the usual sense" and "blanking in short term memory," which perhaps is an allusion to these two possibilities, but does not consider how one might distinguish between them.

8. Consider the region of cortex, MT, which responds to motion (and apparent motion). Suppose then that some activity in MT is the brain's concluding that there was intervening motion. There is no further question, on the Multiple Drafts model, of whether this is a pre-experiential or post-experiential conclusion. It would be a mistake to ask, in other words, whether this activity in MT was a "reaction to a conscious experience" (by the Orwellian historian) as opposed to a "decision to represent motion" (by the Stalinesque editor).

9. See also his dismissal of MacKay's suggestion of a more moderate reading (1981, p.195, 1985b, p.568).

10. Libet's final summation in 1981, on the other hand, was inconclusive: "My own view . . .has been that the temporal discrepancy creates relative difficulties for identity theory, but that these are not insurmountable." (p.196) Presumably they would be undeniably insurmountable on the backwards projection interpretation, and Libet later (1985b, p.569) describes these difficulties in a way that seems to require the milder reading: "Although the delay-and-antedating hypothesis does not separate the actual time of the experience from its time of neuronal production, it does eliminate the necessity for simultaneity between the subjective timing of the experience and the actual clock-time of the experience." Perhaps Eccles' enthusiastic support for a radical, dualistic interpretation of the findings has misdirected the attention of Libet (and his critics) from the mild thesis he sometimes defends.

11. In an earlier paper, Libet conceded the possibility of Orwellian processes and supposed there might be a significant difference between unconscious mental events and conscious-but-ephemeral mental events: "There may well be an immediate but ephemeral kind of experience of awareness which is not retained for recall at conscious levels of experience. If such experiences exist, however, their content would have direct significance only in later unconscious mental processes, although, like other unconscious experiences, they might play an indirect role in later conscious ones." (1965, p.78)

12. Harnad (1989) sees an insoluble problem of measurement, but denies our contention that there is no fact of the matter: "introspection can only tells us when an event seemed to occur, or which of two events seemed to occur first. There is no independent way of confirming that the real timing was indeed as it seemed. Incommensurability is a methodological problem, not a metaphysical one." So Harnad asserts what we deny: that among the real timings of events in the brain is a "real timing" of events in consciousness.

13.In a later response to a similar suggestion of Hoffman and Kravitz (1987) Libet asks the rhetorical question "Are we to accept the primary evidence of the subjects' introspective report (as I do), or are we going to insist that the subject had a conscious experience which he himself does not report and would even deny having had?" (1987,p.784) This is another expression of Libet's a priori preference for a Stalinesque position.