1 Introduction

1The body, including the brain, is something scientists think of in terms of (neuro)biological mechanisms. Society, by and large, tends to follow the lead of science in this respect; when we suffer from illness we are more likely to consult a medical practitioner, rather than a quack. But when it comes to the mind, we are less scientifically minded. For mental and moral advice, we turn to self-help books, psychoanalysts, or ministers of various religious persuasions.

3The prevalence of popular belief notwithstanding, the neurosciences have been encroaching on the realm of mind. But these advances have evoked a backlash. Since brain science apparently has no place for conscious, free will, the public worries that science threatens moral responsibility. The brain as excuse: “It wasn’t me who murdered the guy and robbed his money, your honor; my brain did it.”

4In the first part of this paper I will argue that science does not offer any compelling arguments to exclude free will. I will also argue, however, that in thinking of how body and mind relate, no doubt under Cartesian influence we have been overly focused on issues of freedom of will. Free will is not the most suitable arena for addressing the mind-body problem. As is obvious from the public interest this issue has received, society has invested heavily in the prevailing free-will talk and its cultural, legal, moral, and religious ramifications. To enjoy a culturally unencumbered view to mind-body relationships it would be better to move to a less contested aspect of it. This I shall do in the second part of this paper.

2 Will, conscious and free

5In what way, if any, is science incompatible with free will? I will deal with methodological, ontological, theoretical and empirical considerations. First, let us clarify to what extent free will is barred from science on account of methodological principles. When free will is to be considered an agency that causes but that is not itself being caused, then sure enough any science would oppose to it, as it is in violation of the principle that nothing happens without a cause. The public, however, does not typically conceive of free will that way. Rather, it considers a free will subject to external causal influences as perfectly acceptable, as long as your decisions matter to yourself and others; in other words, as long as they make a difference to what happens next [Nahmias 2011]. Thus, free will in the public perception is not necessarily an unmoved mover, and science cannot object to it on these grounds.

6Some authors worry that with this prevailing attitude, all brain science could offer is a Brave New World version of free will: you are free insofar you do what your brain is telling you to do (see, for instance [Sommers 2007]). Such an understanding of free will may not be as grim as it seems. It will not preclude you from changing your behavior as your brain matures, develops or gains in experience. Neither does it stand in the way of changes through societal or therapeutic intervention. In fact, this realization may have beneficial consequences to society, in the sense that a greater understanding to the causes of deviant behavior makes us less willing to consider retaliation and more prone to consider therapy [Evans 2012].

7Second, we may consider if there are objections in an ontological sense. The overwhelming attitude in the science of the mind is physicalism; a kind of materialism that proclaims that ultimately the only entities capable of causing things are physical ones. Conceded: whatever meaning causation has in a system of elementary particles, including bosons, gravitons, or even in such ephemeral beasts like superstrings, differs markedly from how we understand causation to operate in neurons, circuits and brain areas. Nevertheless, as scientists we believe that there is no phenomenon at this level which cannot be explained in reference to a complex of relations, laws, mechanisms, or events at deeper level of science, ultimately all the way down to this elemental physical reality. This, even though some theorists would permit that multiple discrete entities at a deeper level can instantiate types featuring in higher-level theories of science [Kim 2003]. As such higher-order concepts are necessary for the economy of our theories, these concepts might have a certain, derived claim to existence. Still, on account of such an understanding of “what exists”, free will should ultimately have to be made out of physical things.

8Proposals as to what could be the “free will stuff” range from quantum uncertainties in elementary particles to undecidability in Turing machines. The first is highly controversial, as it is unclear how these microscopic uncertainties could be “elevated” to the scale of macroscopic brain activity. Also the second is subject to debate [Feferman s.d.]. We should distinguish between the abstract competence of such devices and the actual performance of empirical systems—only the latter should concern us here. To these, undecidability bears no relevance. It is for such reasons that the existence of free will has come under debate.

9Within scientific discourse all claims about “what exists” can, in principle, be placed between parentheses and subject to critical examination. In fact, there are even those who believe that the whole theoretical machinery of science exists just for the sake of predicting things, and should therefore be perennially parenthesized. We should not consider the existence of free will as being under threat from science, just because its ontological status is the subject of debate. Conversely, scientific ontology is fundamentally open to accept new entities, should theoretical developments necessitate this.

11Some scientists have therefore made efforts to crush, what they consider, the illusion of free will (e.g., [Wegner 2002]). This may sound heroic. But it might be more like posturing. It is possible to call the well-known Müller-Lyer illusion of size an illusion, because you have a yardstick to measure actual size. But what is the yardstick for free will? Given how little we know, theories of the brain cannot be the yardstick by which we can evaluate the subjective experience of free will.

12Perhaps the software/hardware metaphor is one-sided or misleading. Recently, scientists of the mind have turned away from such concepts in favor of the notion of dynamical self-organization. Such a view enables the emergence of macroscopic order (e.g., a wave of synchronized activity) from underlying particles (e.g., firing neurons). Emergence can be said to produce a genuinely new entity [Atmanspacher 2007], [Jordan & Ghin 2006]. Once established, it runs on its own dynamics and thus constitutes a source of causal efficacy at a higher level. We will see some examples in the second part of the paper. In addition, the movements of these higher-order entities may also influence what happens at lower level in the system [Haken & Stadler 1990]. Note: the latter is not the same as software being run on hardware. Rather processes occurring on a large scale are sweeping along the ones that occur on a small scale, like the surf sweeping along the sand. Such downward causality is typically lacking in the above-mentioned mechanical systems; yet it is quite common in physical and biological systems. It is not unlikely that it also is a characteristic of the brain. Consider the waves in mesoscopic field potential activity (LFP) or even the macroscopic ones that can be observed in electrocortical recordings at the scalp (EEG). Such activity is not necessarily a mere epiphenomenon. These waves could modulate the spiking probabilities of individual neurons (for LFP: [Ito, Maldonado & Grun 2009], for EEG: [Radman, Su, An et al. 2007]; see also [Alexander 2007]). Emergent phenomena such as these waves could, at least in principle, be associated with mental causation, phenomenal experience, and free will.

13This brings me to the empirical arguments. Pioneered by Libet ([Libet, Gleason, Wright et al. 1983] for an updated version see [Soon, Brass, Heinze et al. 2008]), several studies have shown that we subjectively allocate our voluntary decision to a point in time, much later than where our brain has started the execution of a decision. These studies, however interesting, are extremely limited in scope: they involve simple go-nogo or choice tasks in a repetitive setting, to which the individual has limited personal involvement. No surprise that, under these circumstances, participants run these decisions on an automatic pilot—and our illusory experience of consciously choosing the response comes limping afterwards. In more naturalistic settings, decisions emerge in a process of extended deliberation. We do not know whether Libet’s results generalize to these settings. The decision processes observed by Libet and his colleagues may be too simple and sparse on recurrent activity to allow for the above-mentioned type of processes involving downward causality to occur.

14We may conclude that there are neither methodological principles, nor ontological commitments, theoretical or empirical reasons for science to oppose free will. But little do we have that speaks for it either, from a scientific point of view. Society has invested heavily in the prevailing attitudes with respect to this topic. There is no mature scientific understanding of free will or, for that matter, the illusion thereof, that could substitute for these. Thus when, in the above, the murderer’s innocence plea strikes us as absurd, perhaps, this is because of the clash of a pre-scientific understanding of mind and a scientifically informed understanding of brain. Maybe, therefore, we should shift our view towards aspects of conscious experience where we do have scientific knowledge of. A worthier endeavor would be to confront the scientific understanding of the brain with a scientific understanding of the mind.

3 Objective consciousness

15There are mature branches of science representing an integral part of perception research that are studying conscious experience. Think of psychophysics and the studies of the visual perception of structure and shape that have emerged from experimental phenomenology [Kanizsa & Luccio 1995], [Metzger 1934], for instance [Kubovy, Holcombe & Wagemans 1998] or [Van Lier 1999]. Let us refer to the data collected by such sciences as “objective experience”.

16Objective experience includes a range of scientific results. Psychophysics has been dealing with, amongst other things: visual detection and discrimination of colors, contrasts, contrast polarities, spatial frequencies and motion, auditory detection or discrimination of pitch in pure tones, noise levels, timbre; there are also many studies involving olfaction, taste and touch sensation; proprioception, and pain. “Experimental Phenomenology” has been dealing with issues like figural complexity, symmetry, occlusion and the role of perception in action planning. Not only do psychophysicists and experimental phenomenologists know how to collect their data without being contaminated by things like response bias, importantly there are theoretical laws which can be applied to quantitatively predict these data: signal detection (SD) theory [Wickens 2002], adaptation level theory [Helson 1964] and, for structure and shape: Structural Information Theory [van der Helm 2012].

17No scientist, not even a radical behaviorist, would deny the reliability of these data. Several would, perhaps, want to question their “experiential” status. Such critique seems far off the mark: psychophysics asks observers to report on aspects of what they see, hear or feel. By fitting theoretical curves to these data, such as the ROC curve in signal detection theory, qualia are being turned into quantia. This, however, without losing the transparent relation to first-person experience: would your experience have been different, so would your ROC curve.

18The reliability of scientific observations in objective experience notwithstanding, objective experience is not the same as reliable experience. How reliable objective experience is may vary from one case to another. Observers may be remarkably accurate in localizing a sound source by using the difference in phases between the inputs of both ears, but are otherwise remarkably bad at localizing a sound in space. As a result, they will only rely on visual rather than on auditory localization, except when visual resolution is poor (see the ventriloquist effect, [Alais & Burr 2004]).

19Understanding objective experience still belongs to what David Chalmers [Chalmers 1996] qualified as “the hard problem”. Like the pre-theoretical category of “subjective experience”, objective experience is individual and essentially so. It is your decision and yours alone, whether a stimulus in a signal-detection experiment is actually there, or not. You could, of course, be faking your participation. But this would easily be found out: no ROC curve would fit your data. Since they would have to fake their participation, Chalmers’ philosophical zombies would fail this test [Chalmers 1996]. More generally: we can dispute the status of objective experience, when necessary, by holding up observed data against established theories and methods of scientific observation.

20Note that the concept of “objective experience” allows various degrees of skepticism about the veracity of people’s subjective experience, as it is expressed in introspective reports, in literature. Objective experience does not necessarily overlap onto subjective experiences. The former sometimes allows us to “correct” the latter. Let us consider the psychophysics of pain [Gracely 2012], [Lloyd & Appel 1976]. Based on sound experimentation and rigorous application of signal detection theory, the phrase: “You think you are in pain, but in fact you are not” or likewise: you are denying your pain”, would both be meaningful and truth-valuable scientific statements.

21It may seem odd to deny such quintessential a first-person experience, but consider what Adam Swenson writes on his painblog:

• 3 http://dolor.blogspot.com/2004/05/thoughts-about-c-fibers-firing.html downloaded on 27-04-2012.

Many writers—at least those working in ethics and axiology—seem to assume that pains are essentially phenomenological (where ‘phenomenology’ refers to the hurting of the pain) and only accidentally associated with emotions, affect, expectation, etc. [...] They assume that there is some discrete neural phenomenon corresponding to the discrete phenomenological phenomenon—the pain. [...] I, of course, think this picture of what pains are is a mistake. I think that pains are best understood as having certain emotional, desiderative, conative, and affective components essentially.3

In other words, what we experience as pain may have an entrenched, complex and dynamical structure that is hard to be univocally expressed.

22To what extent efforts to “correct” subjective experience are effective is another matter. The simple adage that holds for psychotherapeutic practice, should equally be held up against thought experiments suggesting a scientific reform program promoting something like “my C-fibers are firing” to replace: “I feel a pain”. Let us ignore for the sake of the argument that, for pain and other complex, dynamic phenomena, a correspondence to a highly static brain component such as implied in “C-fibers firing” is most unlikely. Given how theory-laden our folk language for describing our experience is, the argument goes; adopting a new language will change, and perhaps even allow more differentiation in our description of experience. The proposal fails to take into account how much the way we describe our experience is culturally entrenched; for this reason, even if we would prefer such a new vocabulary, experience will stubbornly refuse to be affected by it.

23This stubbornness may resemble the resistance to change in visual illusions. For instance, observers stubbornly mistake the lengths of the familiar Müller-Lyer figure. The concept of objective experience motivates us to take this result seriously. Rather than as an error in need of correction, a more productive way of treating them would be, for instance, as an adaptation to what life requires to an observer, such as having to estimate lengths from a single 2-dimensional projection of a 3-dimensional world. We should, perhaps, principally consider experiences as adaptations, and how well individuals manage their worlds with them, rather than merely dismiss their expressions as illusion. This will certainly apply to pain. Yet, because pain is such a complex, dynamic phenomenon, the analogy with the Müller-Lyer illusion breaks down beyond this point. Compared to the Müller-Lyer illusion, complex sensations such as pain may be more liable to theory-ladenness of observation. Thus, whereas the Müller-Lyer is resistant against cognitive penetration because of the way it is evolutionarily entrenched, our experience of pain may be resistant because of the way it is culturally entrenched.

4 Case studies in mind–brain relationship

24I claim that, once phrased in its scientific form—that is as a relation between objective consciousness and brain, we can begin to consider the mind–brain problem as solvable, at least in principle. Let it be understood that the mind-body problem is unlikely to be conquered wholesale, by some magnificent masterstroke. In what follows I will report on my own efforts to solve the problem in a piecemeal fashion. I will provide two examples, both of which involve ambiguous figures. These are stimuli that can give rise to at least two alternative interpretations.

25A certain degree of ambiguity is characteristic of every visual pattern. Although several interpretations are principally possible, one or a few are typically preferred. We adopted a paradigm from psychophysics, in which the perceived grouping of dot lattices is being studied [Kubovy, Holcombe & Wagemans 1998]. In these experiments, a single stimulus parameter theoretically determines the degree of ambiguity: proximity (see Figure 1A). Proximity determines perceived grouping through a simple relationship called Aspect Ratio (AR). The larger AR, the stronger is the preference for grouping according to the smallest distance (a); the more AR approaches 1, the weaker the preference, i.e., the more ambiguous the dot pattern.

26In a recent study [Nikolaev, Gepshtein, Gong et al. 2010], we recorded electrocortical activity (EEG) at the scalp, evoked by the onset of the presentation of a dot lattice. In such visually evoked brain activity, we studied the oscillatory activity that typically accompanies the enhancement of a focus of mental activity in dedicated brain regions. These oscillations synchronize from time to time and, after some time interval, desynchronization occurs. Sometimes, the synchronization spreads to encompass an entire brain region. We called this phenomenon of emerging macroscopic order a coherence interval [van Leeuwen, Steyvers & Nooter 1997], [van Leeuwen, Gong & Nikolaev 2002]. We had prior determined the location on the scalp where evoked activity was sensitive to AR [Nikolaev, Gepshtein, Kubovy et al. 2008]. At this location we now measured the durations of coherence intervals (Figure 2). We considered these durations in relation to the aspect ratio of the dot lattice. We found a simple, linear relationship (Figure 1B).

27The relationship could be understood if we realize that aspect ratio reflects ambiguity of the lattice, and that ambiguity means absence of information. Thus the less ambiguity there is, the more information. Therefore, the more information contained in the pattern, the longer the coherence interval lasts. In addition, the steepness of the slope in coherence interval length as a function of aspect ratio in individual observers strongly correlated across observers to how sensitive they are to AR. In sum, we may therefore conclude that coherence intervals directly reflect the amount of stimulus information detected.

28The observed coherence intervals have a clearly established meaning in terms of a brain mechanism. Coherence of activity is known to facilitate transfer of information [Fries 2005]. The length of the coherence interval thus reflects the time the particular brain area sensitive to AR needs to transmit the information it has computed to the rest of the brain. But transmission of information in the brain occurs with different delays, depending on facts such as path length, axonal cable length and white matter myelination, synaptic efficacy, etc. It is always the slowest signal that determines the completion of transmission. The more channels are involved, the slower the slowest one tends to be. For the systematic differences to be substantial (approx. 40 ms differences were observed), the transmission has to occur at the scale of the entire brain. Coherence intervals, thus, represent global broadcasting of visual information. Global broadcasting has been proposed as a function of conscious awareness [Dehaene, Kerszberg & Changeux 1998], [Sergent & Dehaene 2004].

29The correspondence between visual experience and brain mechanism would go beyond a simple correlation of mind and brain activity. First, the observation was made in an experimental setting, which excludes other factors. Therefore, the effect of the independent variable, AR, can be interpreted as causal. Second, the independent variable was not a subjective experience, but an objective one: AR is an established and quantitatively specific theoretical predictor of ambiguity. Third, the brain activity is understood theoretically as having a function associated with consciousness. For these reasons, we may propose the coherence intervals as having an identity relation to conscious experience.

30The identity is constitutive of the psychological present [Stroud 1955]. The psychological present is the time window in which events are perceived as temporally contiguous. A psychological present is necessary, physiologically speaking, to accommodate differences in transmission delays between neural cables [Pöppel 1970] as in particular for perceiving movement [Mortensen 2012].

The Reichardt motion detector (M) receives input from photoreceptors R1 and R2, which respond to luminance change. Because there is a longer delay from R1 to M than from R2 to M, the detector will respond to an object moving to the right. [Kline, Holcombe & Eagleman 2004, 2655]

But the detector will have to accept the delay when it attributes the two inputs to the same source.

31Note that the identity considered here is between a mental phenomenon and a dynamically assembled, self-organized brain activity. This is not unimportant. As with the waves mentioned earlier, coherence intervals are caused by the collective movements of particles. But once set in motion, these movements determine whether a visual pattern is consciously registered [Nakatani, Ito, Nikolaev et al. 2005]. Thus, it means that a phenomenon at this high-level of organization, once brought into existence, is perfectly well capable of causing other events at that level. In other words the brain has, through its own activity, produced the conditions for mental causation. In turn, the activity at the pattern level modulates that of the different particles that collectively produce it. This means, specifically, that differences in coherence interval lengths will result in differences in perception and behavior. For these, see [van Leeuwen & Smit 2012].

32We just observed a scientific identity of a mental and a brain type, albeit within the narrow confines of an experiment. Before we turn too optimistic and start thinking about mind–brain identity and dual-aspect theories, allow me to present another piece of research to demonstrate that such identity cannot be taken for granted. This is true, in particular, if we turn from evoked to spontaneous activity. Let us consider another case of perceptual ambiguity. Observing ambiguous figures typically results in perceptual switching; the perception of the figure reverses without any changes to the figure itself [Attneave 1971], [Leopold & Logothetis 1999]. In other words, perceivers experience these changes as spontaneous. This, of course, has to be an illusion, as nothing happens without a cause. We experience these reversals, moreover, as sudden, instantaneous events. This, too may be an illusion, what is experienced as sudden may be considered as a product of an underlying, continuous dynamics [Spivey 2007]. Nakatani and van Leeuwen studied the brain events leading up to perceptual switching and found that several brain activity patterns occur in the run up to a switching response [Nakatani & van Leeuwen 2006]; these patterns typically are transient episodes of synchronized activity in the gamma range in higher brain scalp regions, some of which occurred as early as 1 s prior to the switching response. Ito et al. showed that 1,100 ms prior to the button press switching-related eye events occur, suggesting that already at this time the switching process may have begun [Ito, Nikolaev, Luman et al. 2003]. Responding to an external change the same stimulus takes, on average, only 574 ms [Nakatani, Orlandi & van Leeuwen 2011]. So we may safely assume that if a certain brain or oculomotor event occurs more 1,000 ms prior to a button-press response with which an observer indicates having experienced a switch, that event will have preceded the conscious experience of the switch. Brain and oculomotor activity leading to switching occurs at least approx. 500 ms before the conscious experience. Thus, the sudden and instantaneous character of switching may be considered an illusion.

33Following my own previous suggestion, I consider the adaptive significance of these illusions. If we would perceive the actual time course of the change, it would involve us in an intricate process of visual destabilization and re-stabilization that carries no relevant information whatsoever about the world. The visual system, in other words, protects us against such an irrelevant visual experience. The mechanism of this is not known, but may be similar to what happens during fast eye-movements, or saccades. During these movements, our eyes continue to receive stimulation; the extent and duration of the stimulation is such that we might expect it to be highly salient. We might have expected a horrendous blur of our visual experience. Yet, this does not happen. A mechanism called transient on sustained inhibition protects us against it. As a result, we experience our world to be stable across saccades. Likewise, if we were to envisage the causes of our voluntary decision as the process it is, instead of a unique moment of volition, we would probably be evoking a slew of thoughts, associations, and fears. This shows that volition, as switching, is a construct of the brain aimed to protect ourselves from information overload.

34Recent studies stress in perceptual switching the importance of both low-level (sensory) and high-level (cognitive) processes. In other words, here in the case of pain, the situation is complicated and involves many different brain systems. In a series of experimental studies [Nakatani, Ito, Nikolaev et al. 2005], [Nakatani & van Leeuwen 2006], [Nakatani, Orlandi & van Leeuwen 2011, 2012], we showed that several of these processes lead to switching independently over time, even in the same individual. We took these results to indicate that perceptual switching is a radically multiply realizable process, in that various neurological states can instantiate it in a single individual at different times.

35When first introduced by Putnam, multiple realizability, the idea that mental states may be variously instantiated by neurological states, was widely accepted [Putnam 1967/1975]. The idea was first intended to apply primarily to differences in instantiation across species. If we are correct, perceptual switching is more radically multiply realizable than that. Within the same species and, in fact, even within the same individual, the sequence of eye and brain events that instantiates reversing varies from one time to another. Kim has argued that multiple realizability is an obstacle to mental causation [Kim 2003]. It is either empty (the underlying brain mechanisms do the causing) or, if not, it would impose an extra cause on an already physically determined system. But in a system with both upward (particles to pattern) and downward (pattern to particles) causality, multiple realizability is merely a complication, not an obstacle.

36Radical multiple instantiation stands in the way of a quick and wholesale mind–brain identification program. These results are perfectly consistent, however, with a contextualized type identity: in some specific situations, when certain activity emerges within the brain, that activity is type-identical with mental activity: brain-in-context equals mind. But we need to be able to identify the correspondence one by one.

5 Conclusions

37According to science, nothing in principle stands in the way of free will. But not much speaks for it either. At the same time, there are important obstacles to a scientific resolution of the problem of free will. When we move away from such a contested issue, and are willing to step aside from established information processing theories, we find that the mind–brain problem may actually be closer to a solution than we think. Piecewise solutions are already on offer, as is exemplified in the two cases with ambiguous stimuli I discussed, but these also illustrate that solutions will probably not be achieved wholesale.