Trends in Cognitive Sciences
Special Issue: Probabilistic models of cognitionBayesian decision theory in sensorimotor control
Introduction
The central nervous system (CNS) constantly sends motor commands to our muscles. Determining the appropriate motor command is fundamentally a decision process. At each point in time we must select one particular motor command from the set of possible motor commands. Two components jointly define the decision problem: knowledge of the state of the world (including our own body) and knowledge of our objectives.
The sensory inputs of humans are plagued by noise 1, 2 which means that we will always have uncertainty about our hand's true location (Figure 1a). This uncertainty depends on the modality of the sensory input: when we use proprioception to locate our hand we may have more uncertainty about its position compared to when we can see it. Moreover, our muscles produce noisy outputs 3, 4 and when we quickly move to a target location (shown as a red × in Figure 1a) our final hand position will typically deviate from the intended target. Even if our sensors were perfect they would only tell us about the part of the world that we can currently sense. This uncertainty places the problem of estimating the state of the world and the control of our motor system within a statistical framework. Bayesian statistics 5, 6, 7, 8 provides a systematic way of solving problems in the presence of uncertainty (see the online article by Griffiths and Yuille associated with this issue: Supplementary material online). The approach of Bayesian statistics is characterized by assigning probabilities to any degree of belief about the state of the world (see also Conceptual Foundations editorial by Chater, Tenenbaum and Yuille).
Bayesian statistics defines how new information should be combined with prior beliefs and how information from several modalities should be integrated. Bayesian decision theory 9, 10, 11 defines how our beliefs should be combined with our objectives to make optimal decisions. Understanding the way the CNS deals with uncertainty might be key to understanding its normal mode of operation.
The cost of each movement (such as energy consumed) must be weighed against the potential rewards that can be obtained by moving. In the framework of decision theory a utility function should quantify the overall desirability of the outcome of a movement decision. We should choose a movement so that as to maximize utility. Several recent papers have addressed what functions people optimize with their movements. Understanding what human subjects try to optimize is a necessary step towards a rational theory of movement selection.
The selection of a movement can be described as the rational choice of the movement that maximizes utility according to decision theory (see Box 1). This approach thus asks why people behave the way they do. An increasing number of laboratories have addressed this question within this framework. Here we review recent studies that find human movement performance to be close to the predictions obtained from optimally combining probability estimates with movement costs and rewards. The approach has the potential to embed human behaviour into a coherent mathematical framework.
Section snippets
Estimation using Bayes rule
We need to estimate the variables that are relevant for our choice of movement. For example, when playing tennis we may want to estimate where the ball will bounce. Because vision does not provide perfect information about the ball's velocity there is uncertainty as to the bounce location. However, if we know about the noise in our sensory system then the sensory input can be used to compute the likelihood – the probability of getting the particular sensory inputs for different possible bounce
Bayesian integration in motor control
Bayes rule makes it clear that to perform optimally we must combine prior knowledge of the statistic of the task with the likelihood obtained from the sensory input. In a recent experiment [12], it was tested whether people use such a strategy. Instead of the bounce location of a tennis ball subjects had to estimate the position of a cursor relative to their hand (Figure 1b). Subjects could use two sources of information: The distribution of displacements over the course of many trials (prior),
Costs and rewards
To put movement into a rational framework it is necessary to define a function that measures how good or bad the outcome of a particular movement is. This function, often termed cost may for example be related to the energy consumed during a movement. In general people should prefer less demanding movements – movements that put less strain on the muscles or movements that can be executed using less energy. We are thus faced with the problem of selecting among the infinite set of possible
Models of optimal control: using online feedback
Understanding task statistics, the noise on our sensors and actuators and the utility function allows us to predict optimal behaviour. So far we have discussed these processes applied to discrete decisions chosen from a small number of possible decisions. However, in general we produce a continuous trajectory of movement in response to a contiguous stream of sensory input. The system will thus constantly use feedback to update its movements (Figure 3).
Future directions
The approach of formalizing human decision making as being based on partial uncertainty and utility functions formalizes the problems that are solved by the CNS. There is converging evidence from various communities that Bayesian approaches can serve as a coherent description of human decision making.
The optimal statistical approach to sensorimotor control raises many important questions (see Box 4). However, many of our movements are in the context of complicated tasks such as social
Acknowledgements
We like to thank the German Science Foundation Heisenberg Program for support (KK) as well as the Wellcome grant and the HFSP for financial support.
References (68)
Statistical analysis of motor unit firing patterns in a human skeletal muscle
Biophys. J.
(1969)- et al.
Bayesian models of object perception
Curr. Opin. Neurobiol.
(2003) - et al.
The ventriloquist effect results from near-optimal bimodal integration
Curr. Biol.
(2004) Optimal integration of texture and motion cues to depth
Vision Res.
(1999)- et al.
Do humans optimally integrate stereo and texture information for judgments of surface slant?
Vision Res.
(2003) - et al.
Human altruism: economic, neural, and evolutionary perspectives
Curr. Opin. Neurobiol.
(2004) - et al.
Uncertainty, neuromodulation, and attention
Neuron
(2005) Human contrast discrimination and the threshold of cortical neurons
J. Opt. Soc. Am. A
(1987)- et al.
The computation of position sense from spindles in mono- and multiarticular muscles
J. Neurosci.
(1994) Relationship of firing intervals of human motor units to the trajectory of post-spike after-hyperpolarization and synaptic noise
J. Physiol.
(1996)