Contents 1. Introduction 2. Reward-Guided Decision Making 3. Content in the Model 4. How to Deflate a Metarepresentational Reading Proust and Carruthers on metacognitive feelings 5. A Deflationary Treatment of RPEs? 5.1 Dispensing with prediction errors 5.2 What is use of the RPE focused on? 5.3 Alternative explanations—worldly correlates 5.4 Contrast cases 6. Conclusion Appendix: Temporal Difference Learning Algorithms.
The Human Genome Project (HGP) is regarded by many as one of the major scientific achievements in recent science history, a large-scale endeavour that is changing the way in which biomedical research is done and expected, moreover, to yield considerable benefit for society. Thus, since the completion of the human genome sequencing effort, a debate has emerged over the question whether this effort merits to be awarded a Nobel Prize and if so, who should be the one(s) to receive it, (...) as (according to current procedures) no more than three individuals can be selected. In this article, the HGP is taken as a case study to consider the ethical question to what extent it is still possible, in an era of big science, of large-scale consortia and global team work, to acknowledge and reward individual contributions to important breakthroughs in biomedical fields. Is it still viable to single out individuals for their decisive contributions in order to reward them in a fair and convincing way? Whereas the concept of the Nobel prize as such seems to reflect an archetypical view of scientists as solitary researchers who, at a certain point in their careers, make their one decisive discovery, this vision has proven to be problematic from the very outset. Already during the first decade of the Nobel era, Ivan Pavlov was denied the Prize several times before finally receiving it, on the basis of the argument that he had been active as a research manager (a designer and supervisor of research projects) rather than as a researcher himself. The question then is whether, in the case of the HGP, a research effort that involved the contributions of hundreds or even thousands of researchers worldwide, it is still possible to individualise the Prize? The HGP Nobel Prize problem is regarded as an exemplary issue in current research ethics, highlighting a number of quandaries and trends involved in contemporary life science research practices more broadly. (shrink)
Several psychological theories assume that there are two basic brain mechanisms that guide behavior: an avoidance or inhibition system, which is responsive to signals of punishment, and an approach or activation system, which is sensitive to signals of reward. Several self-report scales have been developed to assess the sensitivity to punishment and reward, and these instruments have been shown to be useful in research on personality, psychopathology, and underlying biological substrates. However, it is also true that in particular (...) scales for measuring reward responsiveness suffer from various inadequacies. Therefore, a new Reward Responsiveness (RR) scale was developed and subjected to an extensive psychometric evaluation. The results show that this scale measures a single factor, reward responsiveness that is clearly independent of punishment sensitivity. Further, the data indicated that the internal consistency, convergent validity, discriminant validity, test-retest reliability, and predictive properties of the new scale were all adequate. It can be concluded that the RR scale is a psychometrically sound instrument that may be useful for researchers with interest in the personality construct of reward responsiveness. (shrink)
Jouissance is a Lacanian concept, infamous for being impervious to understanding and which expresses the paradoxical satisfaction that a subject may derive from his symptom. On the basis of Freud’s “experience of satisfaction” we have proposed a first working definition of jouissance as the (benefit gained from) the motor tension underlying the action which was [once] adequate in bringing relief to the drive and, on the basis of their striking reciprocal resonances, we have proposed that central dopaminergic systems could embody (...) the physiological architecture of Freud’s concept of the drive. We have then distinguished two constitutive axes to jouissance: one concerns the subject’s body and the other the subject’s history. Four distinctive aspects of these axes are discussed both from a metapsychological and from a neuroscience point of view. We conclude that jouissance could be described as an accumulation of body tension, fuelling for action, but continuously balancing between reward and anxiety, and both marking the physiology of the body with the history of its commemoration and arising from this inscription as a constant push to act and to repeat. Moroever, it seems that the mesolimbic accumbens dopaminergic pathway is a reasonable candidate for its underlying physiological architecture. (shrink)
The mesocorticolimbic dopamine (DA) system linking the dopaminergic midbrain to the prefrontal cortex and subcortical striatum has been shown to be sensitive to reinforcement in animals and humans. Within this system, coexistent segregated striato-frontal circuits have been linked to different functions. In the present study, we tested patients with Parkinson’s disease (PD), a neurodegenerative disorder characterised by dopaminergic cell loss, on two reward-based learning tasks assumed to differentially involve dorsal and ventral striato-frontal circuits. 15 non-depressed and non-demented PD patients (...) on levodopa monotherapy were tested both on and off medication. Levodopa had beneficial effects on the performance on an instrumental learning task with constant stimulus-reward associations, hypothesized to rely on dorsal striato-frontal circuits. In contrast, performance on a reversal learning task with changing reward contingencies, relying on ventral striato-frontal structures, was better in the unmedicated state. These results are in line with the “overdose hypothesis” which assumes detrimental effects of dopaminergic medication on functions relying upon less affected regions in PD. This study demonstrates, in a within-subject design, a double dissociation of dopaminergic medication and performance on two reward-based learning tasks differing in regard to whether reward contingencies are constant or dynamic. There was no evidence for a dose effect of levodopa on reward-based behaviour with the patients’ actual levodopa dose being uncorrelated to their performance on the reward-based learning tasks. (shrink)
Although the multimodal stimulation provided by modern audiovisual video games is pleasing by itself, the rewarding nature of video game playing depends critically also on the players’ active engagement in the gameplay. The extent to which active engagement influences dopaminergic brain reward circuit responses remains unsettled. Here we show that striatal reward circuit responses elicited by successes (wins) and failures (losses) in a video game are stronger during active than vicarious gameplay. Eleven healthy males both played a competitive (...) first-person tank shooter game (active playing) and watched a pre-recorded gameplay video (vicarious playing) while their hemodynamic brain activation was measured with 3-tesla functional magnetic resonance imaging (fMRI). Wins and losses were paired with symmetrical monetary rewards and punishments during active and vicarious playing so that the external reward context remained identical during both conditions. Brain activation was stronger in the orbitomedial prefrontal cortex (omPFC) during winning than losing, both during active and vicarious playing conditions. In contrast, both wins and losses suppressed activations in the midbrain and striatum during active playing; however, the striatal suppression, particularly in the anterior putamen, was more pronounced during loss than win events. Sensorimotor confounds related to joystick movements did not account for the results. Self-ratings indicated losing to be more unpleasant during active than vicarious playing. Our findings demonstrate striatum to be selectively sensitive to self-acquired rewards, in contrast to frontal components of the reward circuit that process both self-acquired and passively received rewards. We propose that the striatal responses to repeated acquisition of rewards that are contingent on game related successes contribute to the motivational pull of video-game playing. (shrink)
Stress may promote the onset of psychopathology by disrupting reward processing. However, the extent to which stress impairs reward processing, rather than incentive processing more generally, is unclear. To evaluate the specificity of stress-induced reward processing disruption, 100 psychiatrically healthy females were administered a probabilistic stimulus selection task enabling comparison of sensitivity to reward-driven (Go) and punishment-driven (NoGo) learning under either ‘no stress’ or ‘stress’ (threat-of-shock) conditions. Cortisol samples and self-report measures were collected. Contrary to hypotheses, (...) the groups did not differ significantly in task performance or cortisol reactivity. However, further analyses focusing only on individuals under ‘stress’ who were high responders with regard to both cortisol reactivity and self-reported negative affect revealed reduced reward sensitivity relative to individuals tested in the ‘no stress’ condition; importantly, these deficits were reward-specific. Overall, findings provide preliminary evidence that stress-reactive individuals show diminished sensitivity to reward but not punishment under stress. While such results highlight the possibility that stress-induced anhedonia might be an important mechanism linking stress to affective disorders, future studies are necessary to confirm this conjecture. (shrink)
Controversial results have been reported concerning the neural mechanisms involved in the processing of rewards and punishments. On the one hand, there is evidence suggesting that monetary gains and losses activate a similar fronto-subcortical network. On the other hand, results of recent studies imply that reward and punishment may engage distinct neural mechanisms. Using functional magnetic resonance imaging we investigated both regional and interregional functional connectivity patterns while participants performed a gambling task featuring unexpectedly high monetary gains and losses. (...) Classical univariate statistical analysis showed that monetary gains and losses activated a similar fronto-striatal-limbic network, in which main activation peaks were observed bilaterally in the ventral striatum. Functional connectivity analysis showed similar responses for gain and loss conditions in the insular cortex, the amygdala, and the hippocampus that correlated with the activity observed in the seed region ventral striatum, with the connectivity to the amygdale appearing more pronounced after losses. Larger functional connectivity was found to the medial OFC for negative outcomes. The fact that different functional patterns were obtained with both analyses suggests that the brain activations observed in the classical univariate approach identifies the involvement of different functional networks in a current task. These results stress the importance of studying functional connectivity in addition to standard fMRI analysis in reward-related studies. (shrink)
Adolescence is a developmental period characterized by increased reward-seeking behavior. Investigators have used functional magnetic resonance imaging (fMRI) in conjunction with reward paradigms to test two opposing hypotheses about adolescent developmental changes in the striatum, a region implicated in reward processing. One hypothesis posits that the striatum is relatively hypo-responsive to rewards during adolescence, such that heightened reward-seeking behavior is necessary to achieve the same activation as adults. Another view suggests that during adolescence the striatal (...) class='Hi'>reward system is hyper-responsive, which subsequently results in greater reward-seeking. While evidence for both hypotheses has been reported, the field has generally converged on this latter hypothesis based on compelling evidence. In this review, I describe the evidence to support this notion, speculate on the disparate fMRI findings and conclude with future areas of inquiry to this fascinating question. (shrink)
Both emotion and reward are primary modulators of cognition: Emotional word content enhances word processing, and reward expectancy similarly amplifies cognitive processing from the perceptual up to the executive control level. Here, we investigate how these primary regulators of cognition interact. We studied how the anticipation of gain or loss modulates the neural time course (event-related potentials, ERPs) related to processing of emotional words. Participants performed a semantic categorization task on emotional and neutral words, which were preceded by (...) a cue indicating that performance could lead to monetary gain or loss. Emotion-related and reward-related effects occurred in different time windows, did not interact statistically, and showed different topographies. This speaks for an independence of reward expectancy and the processing of emotional word content. Therefore, privileged processing given to emotionally valenced words seems immune to short-term modulation of reward. Models of language comprehension should be able to incorporate effects of reward and emotion on language processing, and the current study argues for an architecture in which reward and emotion do not share a common neurobiological mechanism. (shrink)
This paper reports lab data from four games in order to analyze and compare the motivations behind monetary punishment and reward and their non-monetary counterparts, disapproval and approval, an important question given that both types of punishment/rewards affect cooperation and norm compliance. The results in our games support the hypothesis that a motivation akin to reciprocity plays the key role for approval and disapproval whereas payoff comparisons play the key role for monetary rewards and punishment.
The present paper aims to advance the understanding of the control of human behavior by integrating two lines of literature that so far have led separate lives. First, one line of literature is concerned with the ideomotor principle of human behavior, according to which actions are represented in terms of their outcomes. The second line of literature mainly considers the role of reward signals in adaptive control. Here, we offer a combined perspective on how outcome representations and reward (...) signals work together to modulate adaptive control processes. We propose that reward signals signify the value of outcome representations and facilitate the recruitment of control resources in situations where behavior needs to be maintained or adapted to attain the represented outcome. We discuss recent research demonstrating how adaptive control of goal-directed behavior may emerge when outcome representations are co-activated with positive reward signals. (shrink)
Recently, the subthalamic nucleus (STN) has been shown to be critically involved in decision-making, action selection, and motor control. Here we investigate the effect of deep brain stimulation (DBS) of the STN on reward-based decision-learning in patients diagnosed with Parkinson’s disease (PD). We determined computational measures of outcome evaluation and reward prediction from PD patients who performed a probabilistic reward-based decision-learning task. In previous work, these measures covaried with activation in the nucleus caudatus (outcome evaluation during the (...) early phases of learning) and the putamen (reward prediction during later phases of learning). We observed that stimulation of the STN motor regions in PD patients served to improve reward-based decision-learning, probably through its effect on activity in frontostriatal motor loops (prominently involving the putamen and, hence, reward prediction). In a subset of relatively younger patients with relatively shorter disease duration, the effects of DBS appeared to spread to more cognitive regions of the STN, benefitting loops that connect the caudate to various prefrontal areas important for outcome evaluation. These results highlight positive effects of STN stimulation on cognitive functions that may benefit PD patients in daily-life association-learning situations. (shrink)
Normal aging is associated with a decline in different cognitive domains and local structural atrophy as well as decreases in dopamine concentration and receptor density. To date, it is largely unknown how these reductions in dopaminergic neurotransmission affect human brain regions responsible for reward-based decision making in older adults. Using a learning criterion in a probabilistic object reversal task, we found a learning stage by age interaction in the dorsolateral prefrontal cortex (dlPFC) during decision making. While young adults recruited (...) the dlPFC in an early stage of learning reward associations, older adults recruited the dlPFC when reward associations had already been learned. Furthermore, we found a reduced change in ventral striatal BOLD signal in older as compared to younger adults in response to high probability rewards. Our data are in line with behavioral evidence that older adults show altered stimulus reward learning and support the view of an altered fronto-striatal interaction during reward-based decision making in old age, which contributes to prolonged learning of reward associations. (shrink)
It is becoming increasingly appreciated that affective influences can contribute strongly to goal-oriented cognition and behaviour. However, much work is still needed to properly characterize these influences and the mechanisms by which they contribute to cognitive processing. An important question concerns the nature of emotional manipulations (i.e., direct induction of affectively-valenced subjective experience) versus motivational manipulations (e.g., delivery of performance-contingent rewards and punishments) and their impact on cognitive control. Empirical evidence suggests that both kinds of manipulations can influence cognitive control (...) in a systematic fashion, but investigations of both have largely been conducted independently of one another. Likewise, some theoretical accounts suggest that emotion and motivation may modulate cognitive control via common neural mechanisms, while others suggest the possibility of dissociable influences. Here, we provide an analysis and synthesis of these various accounts, suggesting potentially fruitful new research directions to test competing hypotheses. (shrink)
Background/Aims: Trait binge eating has been proposed as a ‘hedonic subtype’ of obesity characterised by enhanced food liking and wanting, and a preference for high-fat sweet foods in the laboratory. The current study examined the influence of trait binge eating in overweight or obese women on eating behaviour under laboratory and free-living conditions over a 48-hour period. Methods: In a matched pairs design, 24 overweight or obese females (BMI: 30.30 ± 2.60kg/m2; Age: 25.42 ± 3.65yrs) with high or low scores (...) on the Binge Eating Scale were divided into one of two groups; Obese Binge (O-B) and Obese Non-binge (O-NB). Energy intake was assessed using combined laboratory energy intake measures and 24-hour dietary recall procedures. Liking and wanting were assessed using the Leeds Food Preference Questionnaire. Results: There was a significant association between overall energy consumed, and energy consumed from snack foods under laboratory and free-living conditions. O-B exhibited a greater preference for sweet snack foods in their laboratory and free-living eating behaviour. These findings were supported by greater laboratory-based measures of wanting and craving for this food type in O-B. In addition, O-B consumed significantly more energy than their estimated daily energy requirements in the laboratory suggesting that they over-consumed compared to O-NB. Conclusions: The measurement concordance between laboratory and free-living based energy intake supports the validity of laboratory-based test meal methodologies Variation in trait binge eating was associated with increased craving and wanting for high-fat sweet foods and overconsumption in the laboratory. These findings support the use of trait binge eating as a common hedonic subtype of obesity and extend the relevance of this subtype to habitual patterns of energy intake. (shrink)