by David Bourget and David Chalmers

This is a very brief discussion of the results of the PhilPapers Survey and Metasurvey. At a later point we will write up a much fuller analysis.

We were pleased with the response rate to the Survey. There were 3226 respondents altogether, including 931 from the target group of 1974 philosophers from 99 leading departments, 872 other philosophy faculty and/or PhDs, 829 graduate students in philosophy, 217 undergraduates in philosophy, and 377 with no listed affiliation.

The response rate from the target group was 47%. This high rate allows reasonable inferences to the distribution of views among the target group as a whole. Of course there may be some selection bias among the respondents: perhaps the responding group has some bias toward philosophers working in M&E areas, or younger philosophers, or analytic philosophers, or philosophers sympathetic with the editors' views. At some point we may attempt to analyze certain demographic features of the responding group compared to nonrespondents, but we have not done this so far.

The basic results of the Survey speak for themselves. So far, we have allowed users to view results for groups divided by population (faculty, graduate, etc) and area of specialization. There are many interesting patterns here, but we will save comment until we have performed a fuller analysis. At a later point (probably sometime in January) we will issue further results concerning answers to the main questions broken down by chronological features (age and year of PhD), geographical features (country of nationality, Ph,D, affiliation), gender, and other features. We will also issue interquestion correlations and a factor analysis, as well as other statistical analyses.

On the Metasurvey, we had 727 respondents, including 216 from the target group, 221 other philosophy faculty or PhDs, and 210 philosophy graduate students. Around 23% of Survey respondents completed the Metasurvey. The lower rate is understandable as the cognitive load of the Metasurvey is much higher than that of the Survey. (We thought about offering a prize for the Metasurvey respondent who got closest to the true figures, but decided against in order not to overly encourage respondents to guess in cases where they had no idea.) We will also analyze Metasurvey respondents for selection bias. But again we think that prima facie, there are enough respondents to give some guidance about the sociological beliefs of the target population as a whole as well as about those of other relevant groups. We will focus here on predictions among the population of target faculty, which is also the population whose Survey results are being predicted.

Among the Metasurvey results, it is especially striking that for many questions, the target population's mean estimates of that population's views are off by 20% or more. The normalized results are perhaps most useful here, as errors in estimating "other" options may not reflect errors concerning philosophical views. The biggest errors concern aesthetic value (estimate 68:32 for subjective:objective, actual 45:54) and the analytic-synthetic distinction (estimate 50:50 for yes-no, actual 71:29). Respondents also underestimate the strong support for scientific realism, for not switching on the trolley problem, for moral cognitivism, for non-Humeanism about laws, and for a priori knowledge by close to 20% each. The case of laws is perhaps the most striking, with a 50:50 estimate and a 70:30 result.

The Metasurvey results on the thirty questions break down into five types.

In four cases, the population gets the leading view wrong: predicting subjectivism rather than objectivism about aesthetic value, invariantism instead of contextualism, consequentialism instead of deontology, nominalism instead of Platonism.

In three cases, respondents predict a fairly close result when in fact things are not close: analytic-synthetic distinction, non-Humeanism, moral realism.

In two cases, a minority view is underestimated by 4-11%: rationalism, non-physicalism.

In sixteen cases, significant support for a majority view is predicted but its degree is underestimated by 4-21%: scientific realism, switching on trolley problem, cognitivism, compatibilism, non-skeptical realism, a priori knowledge, representationalism, correspondence theory, egalitarianism, content and epistemic externalism, atheism, psychological view, B-theory, classical logic, conceivability and metaphysical impossibility of zombies.

In five cases, the estimates are within 1.2% of the actual result: naturalism, moral motivation, Newcomb's problem, proper names, teletransporter.

There is of course much to analyze here. Do respondents' sociological predictions tend to favor their own views? Do predictions better track views among philosophers working in the AOS of the question than in the population as a whole? Do predictions better track results in one's own geographical area? Are Metasurvey respondents in an AOS better or worse at predicting the views of the population as a whole? And so on. We will analyze these questions and others in coming months.

Discussion

PhilPapers Surveys (threads)

Other forums | There are 19 threads in this forum | Start a new thread

First post

Latest post

Total

Discussions of Survey results elsewhere on the web

David Chalmers 2009-12-10

Carlo Penco 2010-01-10

8

Surveys of experts in other fields (economics, physics, biology, etc.)

Luke Muehlhauser 2010-01-03

No follow-ups

1

Missing Questions?

Brent Allsop 2010-01-03

No follow-ups

1

Overestimating or Underestimating the Proportion Who Agree with You

Eric Schwitzgebel 2009-12-11

Eric Schwitzgebel 2009-12-30

5

Effects of specialization

David Chalmers 2009-12-13

Mark Povich 2009-12-29

6

Eye-openers in the Survey results?

David Bourget 2009-12-08

Panu Raatikainen 2009-12-18

23

Switching the Trolley

Eric Schwitzgebel 2009-12-09

Mark van Roojen 2009-12-17

22

Eastern Philosophers

Pohlan Song 2009-12-16

David Chalmers 2009-12-16

2

Is it possible to ask the survey team for further correlations/regressions?

Vlastimil Vohánka 2009-12-09

Vlastimil Vohánka 2009-12-16

8

Theism, Atheism, Agnosticism?

Noam Zohar 2009-12-13

Derek Allan 2009-12-15

20

Comments

Anderson Brown 2009-12-14

David Chalmers 2009-12-14

2

Graduate students vs. faculty

David Chalmers 2009-12-09

David Cheely 2009-12-14

6

How did you do on the Metasurvey?

David Chalmers 2009-12-08

John Maier 2009-12-14

18

Survey questions sorted by degree of consensus

Geoff Milburn 2009-12-11

No follow-ups

1

Will you survey again?

Anderson Brown 2009-12-09

Michael Bishop 2009-12-10

3

Discuss PhilPapers Survey results here

David Chalmers 2009-12-08

Cliff Landesman 2009-12-10

5

Political Affiliation and the Existence of an External World

Eric Schwitzgebel 2009-12-09

No follow-ups

1

Normative Ethics

Andrew Kunsak 2009-12-08

Ignasi Llobera 2009-12-09

3

Metaphysics

Raul Saucedo 2009-12-09

No follow-ups

1

Back to contents