The problem of evaluating automated large-scale evidence aggregators

Synthese:1-28 (forthcoming)
Nicolas Wuethrich
London School of Economics
Katie Steele
Australian National University
In the biomedical context, policy makers face a large amount of potentially discordant evidence from different sources. This prompts the question of how this evidence should be aggregated in the interests of best-informed policy recommendations. The starting point of our discussion is Hunter and Williams’ recent work on an automated aggregation method for medical evidence. Our negative claim is that it is far from clear what the relevant criteria for evaluating an evidence aggregator of this sort are. What is the appropriate balance between explicitly coded algorithms and implicit reasoning involved, for instance, in the packaging of input evidence? In short: What is the optimal degree of ‘automation’? On the positive side: We propose the ability to perform an adequate robustness analysis (which depends on the nature of the input variables and parameters of the aggregator) as the focal criterion, primarily because it directs efforts to what is most important, namely, the structure of the algorithm and the appropriate extent of automation. Moreover, where there are resource constraints on the aggregation process, one must also consider what balance between volume of evidence and accuracy in the treatment of individual evidence best facilitates inference. There is no prerogative to aggregate the total evidence available if this would in fact reduce overall accuracy.
Keywords Evidence Aggregation  Evidence-based medicine  Statistical Meta-Analysis  Robustness Analysis
Categories (categorize this paper)
Reprint years 2017
Edit this record
Mark as duplicate
Export citation
Find it on Scholar
Request removal from index
Revision history

Download options

Our Archive
External links

Setup an account with your affiliations in order to access resources via your University's proxy server
Configure custom proxy (use this if your affiliation does not provide a proxy)
Through your library

References found in this work BETA

No references found.

Add more references

Citations of this work BETA

No citations found.

Add more citations

Similar books and articles

Is Meta-Analysis the Platinum Standard of Evidence?Jacob Stegenga - 2011 - Studies in History and Philosophy of Science Part C 42 (4):497-507.
Evidence-Based Policy: Where is Our Theory of Evidence?N. Cartwright, A. Goldfinch & J. Howick - 2009 - Journal of Children’s Services 4 (4):6--14.
Robust Evidence and Secure Evidence Claims.Kent W. Staley - 2004 - Philosophy of Science 71 (4):467-488.
Corroborating Evidence-Based Medicine.Alexander Mebius - 2014 - Journal of Evaluation in Clinical Practice 20 (6):915-920.
Down with the Hierarchies.Jacob Stegenga - 2014 - Topoi 33 (2):313-322.


Added to PP index

Total views
76 ( #90,873 of 2,313,615 )

Recent downloads (6 months)
19 ( #28,752 of 2,313,615 )

How can I increase my downloads?

Monthly downloads

My notes

Sign in to use this feature