David Bourget (Western Ontario)
David Chalmers (ANU, NYU)
Rafael De Clercq
Ezio Di Nucci
Jack Alan Reynolds
Learn more about PhilPapers
Stochastic forecasts in complex environments can beneﬁt from combining the estimates of large groups of forecasters (“judges”). But aggregating multiple opinions faces several challenges. First, human judges are notoriously incoherent when their forecasts involve logically complex events. Second, individual judges may have specialized knowledge, so diﬀerent judges may produce forecasts for diﬀerent events. Third, the credibility of individual judges might vary, and one would like to pay greater attention to more trustworthy forecasts. These considerations limit the value of simple aggregation methods like linear averaging. In this paper, a new algorithm is proposed for combining probabilistic assessments from a large pool of judges. Two measures of a judge’s likely credibility are introduced and used in the algorithm to determine the judge’s weight in aggregation. The algorithm was tested on a data set of nearly half a million probability estimates of events related to the 2008 U.S. presidential election (∼ 16000 judges).
|Keywords||No keywords specified (fix it)|
|Categories||categorize this paper)|
Setup an account with your affiliations in order to access resources via your University's proxy server
Configure custom proxy (use this if your affiliation does not provide a proxy)
|Through your library||
References found in this work BETA
No references found.
Citations of this work BETA
Julia Staffel (2015). Measuring the Overall Incoherence of Credence Functions. Synthese 192 (5):1467-1493.
Similar books and articles
Guanchun Wang, Sanjeev Kulkarni & Daniel N. Osherson, Improving Aggregated Forecasts of Probability.
Teddy Seidenfeld (1985). Calibration, Coherence, and Scoring Rules. Philosophy of Science 52 (2):274-294.
Joel Predd, Robert Seiringer, Elliott Lieb, Daniel Osherson, H. Vincent Poor & Sanjeev Kulkarni (2009). Probabilistic Coherence and Proper Scoring Rules. IEEE Transactions on Information Theory 55 (10):4786-4792.
Robert S. Goldfarb, H. O. Stekler & Joel David (2005). Methodological Issues in Forecasting: Insights From the Egregious Business Forecast Errors of Late 1930. Journal of Economic Methodology 12 (4):517-542.
Richard Bradley, Franz Dietrich & Christian List (2014). Aggregating Causal Judgments. Philosophy of Science 81 (4):491-515.
Peter Ayton, Alice Pott & Najat Elwakili (2007). Affective Forecasting: Why Can't People Predict Their Emotions? Thinking and Reasoning 13 (1):62 – 80.
Keith Lehrer (1983). Rationality as Weighted Averaging. Synthese 57 (3):283 - 295.
Added to index2010-12-22
Total downloads36 ( #112,942 of 1,907,520 )
Recent downloads (6 months)4 ( #197,471 of 1,907,520 )
How can I increase my downloads?