PhilPapers Surveys |
| 2009-12-08 |
How did you do on the Metasurvey?
|
|
David Chalmers
Australian National University |
All Survey and Metasurvey respondents will receive an email giving a link to a page with their responses, including an assessment of how well they did on the Metasurvey. We can't post those results publically, as participants did not consent to that, but people should feel free to post about their own Metasurvey results.
I took the Metasurvey unofficially by making predictions at the start of the Survey. I didn't take it officially, as even by that point I'd seen results from beta testing the Survey. Even so, a few of my predictions were off by a long way. For example, I wrongly predicted a substantial majority for Humeanism, aesthetic subjectivism, Platonism, and invariantism. I did better on the physicalism and analytic-synthetic distinction questions, predicting 60-20-20 in both cases (compared to 56-27-17 and 65-27-18), and was reasonable close on the zombie question, predicting 40-20-20-20 for CMI, MP, IC, other as opposed to 36-23-16-25. I also accidentally predicted majorities for theism and scientific anti-realism by putting the predictions in the wrong order (I think this was an artifact of having worked so much with the questions that a particular order was fixed in my mind). Overall, my average absolute error was 11%, placing me (unofficially) 31st out of 728 respondents. It would be interesting to hear from others.
Permanent link: http://philpapers.org/post/2284
Reply
|
| 2009-12-09 |
How did you do on the Metasurvey?
Reply to David Chalmers |
|
Berit Brogaard
University of Missouri St. Louis |
I wrongly predicted a majority for Humeanism, non-classical logic, moral anti-realism, aesthetic value (subjective), analytic-synthetic distinction, proper names (Fregeanism), time (A-theory), and the trolley problem (don't switch). My best scores were on: truth, God and politics (!). I did reasonably well on a priori knowledge, personal identity, Newcomb's problem and mental content. But overall I was way off. My average absolute error per answer option was 18.6%.
Permanent link: http://philpapers.org/post/2293
Reply
|
| 2009-12-09 |
How did you do on the Metasurvey?
Reply to David Chalmers |
|
Robert Williams
University of Leeds |
Fun stuff! My worst prediction was on Newcomb---I thought that 2-boxing was far more widespread. I was surprised it only beat out 1-boxing 3:2. It would be interesting to see the predictions matched against results ignoring both predicted and actual "other" and scaling appropriately----basically, accuracy conditional on venturing an opinion. Anyway, I was 140---in the top quarter, I think....
Permanent link: http://philpapers.org/post/2297
Reply
|
| 2009-12-09 |
How did you do on the Metasurvey?
Reply to David Chalmers |
|
Jah Bil
University of Victoria |
Average absolute error per answer option: 13.5%
Standard deviation: 10.9% Your rank by average absolute error: 212 of 728 I wrongly predicted a high percentage of political libertarians, a-theorists, relativists, consequentialists, trolly non-switchers, and analytic-synthetic conflaters.
Permanent link: http://philpapers.org/post/2299
Reply
|
| 2009-12-09 |
How did you do on the Metasurvey?
Reply to David Chalmers |
|
Andrea Weisberger
University of North Florida |
Well I did not do so well on the metasurvey -- average absolute error per answer option: 17.8%
I made a conscious effort to answer what I thought other people might select, and had difficulty when that started to coincide with what I thought. So I erred on the side of caution by presuming that if I selected a position, it might be due to my own preference, not necessarily what the majority might opt for, and in many cases deliberately skewed in favor of an opposite approach. For example, as an avowed atheist, I assumed that there are more theistic philosophers than the survey indicated. At least it seems that way in the normal world, atheists being the minority everywhere and few people, even in academia, admitting to that culturally unpopular view. I see now my approach to the metasurvey didn't work out so well. Happily there are more identifiable atheists than I thought (at least private ones). I must admit that I also got a bit frustrated with the sliders and how difficult it was to get an exact rendering of the desired percentages. After a while, I jettisoned trying to include the "other" category and split results between positive and negative responses. Overall, it was an enlightening exercise. Thanks to all who put in the hard work in presenting this to the philosophical community.
Permanent link: http://philpapers.org/post/2302
Reply
|
| 2009-12-09 |
How did you do on the Metasurvey?
Reply to David Chalmers |
|
Eric T. Kerr
University of Edinburgh |
I thought there would be more support for sense-datum theory and was surprised by the number of compatibalists. Overall I had an AAE of 11.6%. It would be interesting to see what the average AAE of respondents was as many so far seem to be reasonably accurate. I wouldn't find this surprising as philosophers perhaps spend more time asking each other where they stand on X and making claims about consensus than many other disciplines. I only took a cursory look at the breakdown by AOS but it is interesting that some groups of specialists are much better at predicting results of respondents in their own AOS than others.
Permanent link: http://philpapers.org/post/2300
Reply
|
| 2009-12-09 |
How did you do on the Metasurvey?
Reply to David Chalmers |
|
Helen Daly
Colorado College |
I did better on the Metasurvey than I expected, so my MetaMetasurvey score would be quite low. I think this surprise is due to the fact that I skipped a few questions. I don't have a very good idea about where people stand on some issues, so I opted not to respond about those. Having skipped them appears not to have counted against me, and so my average absolute error per answer is probably higher than it would have been had I answered all of the questions.
Average absolute error per answer option: 11.5% Standard deviation: 9.6% Your rank by average absolute error: 52 of 728
Permanent link: http://philpapers.org/post/2301
Reply
|
| 2009-12-09 |
How did you do on the Metasurvey?
Reply to David Chalmers |
|
Average absolute error per answer option: 11.4%
Standard deviation: 8.4% Your rank by average absolute error: 46 of 728 I did worst on
Permanent link: http://philpapers.org/post/2304
Reply
|
| 2009-12-09 |
How did you do on the Metasurvey?
Reply to David Chalmers |
|
Antony Eagle
University of Adelaide |
My metasurvey results were:
Average absolute error per answer option: 14.2% Standard deviation: 12.1% Your rank by average absolute error: 266 of 728 I did okay on a few. I did worst on the trolley problem: don't switch switch Predicted: 70% 25% Actual: 7.6% 68.2% But my main problem seemed to be that I systematically underestimated 'other', in every debate. I thought that I was representative of philosophers in having firm opinions about nearly every topic. But maybe there are lots of philosopher who aren't quite so opinionated? (The other alternative, of course, is that I have one of two crude views on every topic, while many others have sophisticated views that don't fall neatly into the categories offered. I'm not so persuaded this alternative is correct.)
Permanent link: http://philpapers.org/post/2309
Reply
|
| 2009-12-10 |
How did you do on the Metasurvey?
Reply to Antony Eagle |
|
Richard Corry
University of Tasmania |
My results:
Average absolute error per answer option: 15.5% Standard deviation: 11.1% Your rank by average absolute error: 377 of 728 I had the same problem as Antony: consistently underestimating 'other'. I guess we shouldn't underestimate the ability of philosophers to make a straightforward question complex!
Permanent link: http://philpapers.org/post/2329
Reply
|
| 2009-12-10 |
How did you do on the Metasurvey?
Reply to David Chalmers |
|
I viewed the survey as asking one general question divided into 30 sub-questions, classical positions against newly developed positions. For example, he who denies a priori knowledge, Nominalist in abstract objects, externalist in epistemic justification, realist in external world, Humean in laws of nature, classical in logic, naturalist in metaphilosopy, physicalist in Mind, externalist in moral motivation, ...etc subscribe in general to the classical position. Those who take the counter positions subscribe to the newly developed positions. Naturally, things are much more complicated, many may advance answers that include elements of both positions. However, my view is that the philosophical community is split relatively equal between the two positions. Hence, except in some specifically clear cases I predicted a 45 : 40 :5. Responses, in general were closer to 60 : 30 : 10 which means that the classical position is still significantly prevailing. As a result my metasurvey results were: Average absolute error per answer
option: 18.6% As I see it, the classical
position is not so prevailing. Possible reason may be that the "new
positions" is flourishing within specific communities (younger professors,
non-leading institutes, or else) who are not sufficiently presented in the
survey.
Permanent link: http://philpapers.org/post/2326
Reply
|
| 2009-12-10 |
How did you do on the Metasurvey?
Reply to David Chalmers |
|
Michael Bishop
University of Chicago |
@Someone working with the data:I hope you be analyzing the relationship between score on the metasurvey and philosophical views?
Permanent link: http://philpapers.org/post/2333
Reply
|
| 2009-12-11 |
How did you do on the Metasurvey?
Reply to David Chalmers |
|
Eric Schwitzgebel
University of California, Riverside |
I am inordinately and completely inappropriately proud of the fact that I was ranked number 3 overall in the metasurvey, though I'm not sure I'd want to do a rematch with Dave.
But I wonder what it indicates about someone that they're ranked well. In Dave's case, I suspect it's largely attunement to the Zeitgeist and lots of experience with informal polling. But I spend half my time reading psychology and the weirder corners of the history of philosophy, so I doubt my Zeitgeist-attunement is above average for a faculty member at at PhD-granting institution. My guess -- this could checked -- is that most people oversteered, while I played it flatter except in obvious cases like atheism-theism, including always setting aside a healthy chunk for "other".
Permanent link: http://philpapers.org/post/2346
Reply
|
| 2009-12-12 |
How did you do on the Metasurvey?
Reply to Eric Schwitzgebel |
|
David Chalmers
Australian National University |
Eric: You're really #2, as the official #2 answered only 12 questions, while you and #1 answered all thirty. We should have had a separate ranking for no-skippers or a penalty factor for unanswered questions, but that seemed too complicated.
The "winner" has given me permission to reveal his identity: it's Alex Byrne of MIT. Congrats, champ! You have a fine sociological nose, indeed. Alex and Eric were a long way ahead of the pack, with average absolute errors of 8.07 and 8.62, respectively, compared to 9.82 for #3 on the no-skip list (then a cluster down to 10.45 for #10). I do note that both Alex and Eric were beta testers for the Survey, as were #5 and #6. It's possible that the testing process in effect leaked some information about the distribution of answers. I didn't give specific figures to beta testers, but a few bits of qualitative information were passed along. So I ought also to give special congrats to #3 (a well-known philosopher of language) and #4 (an ex-ANU graduate student), both of whom answered all thirty and neither of whom were beta testers.
Permanent link: http://philpapers.org/post/2351
Reply
|
| 2009-12-12 |
How did you do on the Metasurvey?
Reply to David Chalmers |
|
Richard Brown
LaGuardia Community College, CUNY |
The metasurvey was costly to me as I lost a bet on how many philosophers would self-identify as dualists. I bet Dave $100.00 that it would be less than 10% and it actually turned out to be something like 27%! It urns out the only category where I got it right was among people who explicitly identify Philosophy of Cognitive Science as their AOS…coincidentally these are just the people that I usually associate with…I wonder if other people who took the meta-survey noticed that their meta-survey guesses reflected the numbers filtered for their (main) AOS/friends in philosophy? If so then the mistake we might be making may lie in (wrongly) thinking that the group of philosophers one interacts with regularly is a representative sample of the landscape whereas people who did well will probably tend to be those who interact with a large group of philosophers from many different areas.
Permanent link: http://philpapers.org/post/2356
Reply
|
| 2009-12-12 |
How did you do on the Metasurvey?
Reply to Richard Brown |
|
Berit Brogaard
University of Missouri St. Louis |
I don't think interacting with a large number of people will necessarily improve your results. I take myself to be interacting with a large number of people in the target group. I also was part of the beta-testing. I still misfired. Why? Here is one reason. When you talk to people, they will tend to lean towards certain views. So, it's impossible to predict how many people will say "other". I highly suspect it was pure puck who got this right and who got it wrong (sorry, Alex). What we should be looking at is how many people got the majorities wrong. That's interesting!
Permanent link: http://philpapers.org/post/2359
Reply
|
| 2009-12-14 |
How did you do on the Metasurvey?
Reply to David Chalmers |
|
John Maier
University of Sydney |
Like many others, I underestimated the extent of "other." (My results: 13.0% absolute error, rank 163 of 728).
Here is the case that puzzles me most: the actual for Newcomb's problem was 31.4% two-boxers and 21.3% one-boxers. So 47.3% of respondents fell into the "other" category here! I don't understand this. A response that I made several times, that the question is too unclear to answer, is simply not plausible here. Nor is it plausible that many professional philosophers are ignorant of the issue. There are some sophisticated views one could have that would make one reject both options (some sort of non-factualism about rationality, for example) but I find it very hard to believe that such a large number of philosophers hold this sort of view. Is there a breakdown of the data for the "others" available? Alternate explanations?
Permanent link: http://philpapers.org/post/2376
Reply
|
| 2009-12-14 |
How did you do on the Metasurvey?
Reply to David Chalmers |
|
John Maier
University of Sydney |
Someone pointed me to the "fine-grained" data, and that answers my question, though it remains surprising. The most popular answer was unfamiliarity, while the third-most popular was agnosticism. Based on a quick look through the fine-grained data, that gives Newcomb's problem the highest percentage of agnosticism and the second-highest of unfamiliarity (after time).
Permanent link: http://philpapers.org/post/2377
Reply
|
