The limitations of randomized controlled trials in predicting effectiveness

Journal of Evaluation in Clinical Practice 16 (2):260-266 (2010)
  Copy   BIBTEX

Abstract

What kinds of evidence reliably support predictions of effectiveness for health and social care interventions? There is increasing reliance, not only for health care policy and practice but also for more general social and economic policy deliberation, on evidence that comes from studies whose basic logic is that of JS Mill's method of difference. These include randomized controlled trials, case–control studies, cohort studies, and some uses of causal Bayes nets and counterfactual-licensing models like ones commonly developed in econometrics. The topic of this paper is the 'external validity' of causal conclusions from these kinds of studies. We shall argue two claims. Claim, negative: external validity is the wrong idea; claim, positive: 'capacities' are almost always the right idea, if there is a right idea to be had. If we are right about these claims, it makes big problems for policy decisions. Many advice guides for grading policy predictions give top grades to a proposed policy if it has two good Mill's-method-of difference studies that support it. But if capacities are to serve as the conduit for support from a method-of-difference study to an effectiveness prediction, much more evidence, and much different in kind, is required. We will illustrate the complexities involved with the case of multisystemic therapy, an internationally adopted intervention to try to diminish antisocial behaviour in young people

Links

PhilArchive



    Upload a copy of this work     Papers currently archived: 74,662

External links

Setup an account with your affiliations in order to access resources via your University's proxy server

Through your library

Analytics

Added to PP
2010-07-25

Downloads
262 (#45,029)

6 months
5 (#144,933)

Historical graph of downloads
How can I increase my downloads?

Author's Profile

Eileen Munro
London School of Economics

Citations of this work

Fact-Value Entanglement in Positive Economics.Julian Reiss - 2017 - Journal of Economic Methodology 24 (2):134-149.
Why Do Experts Disagree?Julian Reiss - 2020 - Critical Review: A Journal of Politics and Society 32 (1-3):218-241.
Rhetoric and Argumentation: How Clinical Practice Guidelines Think.Jonathan Fuller - 2013 - Journal of Evaluation in Clinical Practice 19 (3):433-441.

View all 35 citations / Add more citations