Philipp Schönegger
University of St. Andrews
A key challenge in experimental social science research is the incentivisation of subjects such that they take the tasks presented to them seriously and answer honestly. If subject responses can be evaluated against an objective baseline, a standard way of incentivising participants is by rewarding them monetarily as a function of their performance. However, the subject area of experimental philosophy is such that this mode of incentivisation is not applicable as participant responses cannot easily be scored along a true-false spectrum by the experimenters. We claim that experimental philosophers’ neglect of and claims of unimportance about incentivisation mechanisms in their surveys and experiments has plausibly led to poorer data quality and worse conclusions drawn overall, potentially threatening the research programme of experimental philosophy in the long run. As a solution to this, we propose the adoption of the Bayesian Truth Serum, an incentive-compatible mechanism used in economics and marketing, designed for eliciting honest responding in subjective data designs by rewarding participant answers that are surprisingly common. We argue that the Bayesian Truth Serum adequately addresses the issue of incentive compatibility in subjective data research designs and that it should be applied to the vast majority of research in experimental philosophy. Further, we provide an empirical application of the method, demonstrating its qualified impact on the distribution of answers on a number of standard experimental philosophy items and outline guidance for researchers aiming to apply this mechanism in future research by specifying the additional costs and design steps involved.
Keywords No keywords specified (fix it)
Categories (categorize this paper)
DOI 10.1007/s13164-021-00571-4
Edit this record
Mark as duplicate
Export citation
Find it on Scholar
Request removal from index
Translate to english
Revision history

Download options

PhilArchive copy

Upload a copy of this paper     Check publisher's policy     Papers currently archived: 63,319
External links

Setup an account with your affiliations in order to access resources via your University's proxy server
Configure custom proxy (use this if your affiliation does not provide a proxy)
Through your library

References found in this work BETA

No references found.

Add more references

Citations of this work BETA

No citations found.

Add more citations

Similar books and articles

Oracles, Aesthetics, and Bayesian Consensus.Jeffrey A. Barrett - 1996 - Philosophy of Science 63 (Supplement):273-280.
Bayesian Confirmation: A Means with No End.Peter Brössel & Franz Huber - 2015 - British Journal for the Philosophy of Science 66 (4):737-749.
Many Irrelevant Evils: A Response to the Bayesian Problem of Evil.Jamie Carlin Watson - 2014 - International Journal of Philosophy and Theology 75 (4):365-378.
Armchair-Friendly Experimental Philosophy.Jennifer Nagel & Kaija Mortensen - 2016 - In Justin Sytsma & Wesley Buckwalter (eds.), A Companion to Experimental Philosophy. Malden, MA, USA: Wiley-Blackwell. pp. 53-70.
Bayesian Inference and its Application to Meta-Analysis of Epidemiologic Data.Pamela Anne Trudeau - 1991 - Dissertation, The University of Texas Health Sciences Center at Houston School of Public Health
Epistemic Irrationality in the Bayesian Brain.Daniel Williams - forthcoming - British Journal for the Philosophy of Science:000-000.
Bayesianism and Language Change.Jon Williamson - 2003 - Journal of Logic, Language and Information 12 (1):53-97.
On Doing Better, Experimental-Style. [REVIEW]Jonathan M. Weinberg - 2009 - Philosophical Studies 145 (3):455 - 464.
Bayesian Orgulity.Gordon Belot - 2013 - Philosophy of Science 80 (4):483-503.


Added to PP index

Total views
4 ( #1,245,002 of 2,448,697 )

Recent downloads (6 months)
4 ( #178,140 of 2,448,697 )

How can I increase my downloads?


My notes