Evidence-Based Medicine (EBM) developed from the work of clinical epidemiologists at McMaster University and Oxford University in the 1970s and 1980s and self-consciously presented itself as a "new paradigm" called "evidence-based medicine" in the early 1990s. The techniques of the randomized controlled trial, systematic review and meta-analysis have produced an extensive and powerful body of research. They have also generated a critical literature that raises general concerns about its methods. This paper is a systematic review of the critical literature. It (...) finds the description of EBM as a Kuhnian paradigm helpful and worth taking further. Three kinds of criticism are evaluated in detail: criticisms of procedural aspects of EBM (especially from Cartwright, Worrall and Howick), data showing the greater than expected fallibility of EBM (Ioaanidis and others), and concerns that EBM is incomplete as a philosophy of science (Ashcroft and others). The paper recommends a more instrumental or pragmatic approach to EBM, in which any ranking of evidence is done by reference to the actual, rather than the theoretically expected, reliability of results. Emphasis on EBM has eclipsed other necessary research methods in medicine. With the recent emphasis on translational medicine, we are seeing a restoration of the recognition that clinical research requires an engagement with basic theory (e.g. physiological, genetic, biochemical) and a range of empirical techniques such as bedside observation, laboratory and animal studies. EBM works best when used in this context. (shrink)

The claim that randomized controlled trials can improve policy-making in developing countries seemed to be severely challenged by the widespread belief that RCTs lack external validity. This belief led to the articulation of strategies to alleviate this problem which developed under the assumption that policy-making in developing countries can be best understood as a problem of prediction. The causal structures that characterize developing contexts, however, render this framing inadequate. I propose to understand policy-making in developing contexts as a planning process. (...) In this new framework, the relevance of several kinds of evidence is highlighted and the role of causal effects refocused. Interestingly, also the concern for the external validity of causal effects seems now less forceful than the critics of RCTs suggest. (shrink)

This paper looks at an appeal to the authority of biomedical research that has recently been used by empirical economists to motivate and justify their methods. I argue that those who make this appeal mistake the nature of biomedical research. Randomised trials, which are said to have revolutionised biomedical research, are a central methodology, but according to only one paradigm. There is another paradigm at work in biomedical research, the inferentialist paradigm, in which randomised trials play no special role. I (...) outline the inferentialist alternative in broad strokes, apply it to a recent controversy in econometrics and draw some general conclusions concerning econometric methodology. (shrink)

Francesco Guala once wrote that ‘The problem of extrapolation is a minor scandal in the philosophy of science’. This paper agrees with the statement, but for reasons different from Guala’s. The scandal is not, or not any longer, that the problem has been ignored in the philosophy of science. The scandal is that framing the problem as one of external validity encourages poor evidential reasoning. The aim of this paper is to propose an alternative—an alternative which constitutes much better evidential (...) reasoning about target systems of interest, and which makes do without consideration of external validity. (shrink)

Philosophical discussions on causal inference in medicine are stuck in dyadic camps, each defending one kind of evidence or method rather than another as best support for causal hypotheses. Whereas Evidence Based Medicine advocates the use of Randomised Controlled Trials and systematic reviews of RCTs as gold standard, philosophers of science emphasise the importance of mechanisms and their distinctive informational contribution to causal inference and assessment. Some have suggested the adoption of a pluralistic approach to causal inference, and an inductive (...) rather than hypothetico-deductive inferential paradigm. However, these proposals deliver no clear guidelines about how such plurality of evidence sources should jointly justify hypotheses of causal associations. We here develop such guidelines by first giving a philosophical analysis of the underpinnings of Hill’s viewpoints on causality. We then put forward an evidence-amalgamation framework adopting a Bayesian net approach to model causal inference in pharmacology for the assessment of harms. Our framework accommodates a number of intuitions already expressed in the literature concerning the EBM vs. pluralist debate on causal inference, evidence hierarchies, causal holism, relevance, and reliability. (shrink)

Evidence-based policy has support in many areas of government and in public affairs more generally. In this paper we outline what evidence-based policy is, then we discuss its strengths and weaknesses. In particular, we argue that it faces a serious challenge to provide a plausible, over-arching account of evidence. We contrast evidence-based policy with evidence-based medicine, especially the role of evidence in assessing the effectiveness of medicines. The evidence required for policy decisions does not easily lend itself to randomized controlled (...) trials, nor, for that matter, being listed in a single all-purpose hierarchy. (shrink)

Next SectionThe current focus on early intervention trials in Alzheimer's disease research raises particular ethical issues. These arise out of problems of validating study results and translating them into general practice for one thing and out of unwanted effects of an uncertain diagnosis for diagnosed people for another. The first addresses the demands of scientific research compared to those of medical practice, questioning how the medical value of clinical trials is evaluated. The second relates the scientific and medical value of (...) early intervention trials to the normative value of an uncertain diagnosis. Are people who are diagnosed with a potential early form of AD in clinical studies patients proper—although they would not have been diagnosed with the given “disease” in non-research-oriented medical settings? The very problem of framing this question in terms of diagnostic misconceptions connects conceptual with ethical issues of research into preclinical stages of neurodegenerative diseases. (shrink)

Semmelweis’s work predates the discovery of the power of randomization in medicine by almost a century. Although Semmelweis would not have consciously used a randomized controlled trial (RCT), some features of his material—the allocation of patients to the first and second clinics—did involve what was in fact a randomization, though this was not realised at the time. This article begins by explaining why Semmelweis’s methodology, nevertheless, did not amount to the use of a RCT. It then shows why it is (...) descriptively and normatively interesting to compare what he did with the modern approach using RCTs. The argumentation centres on causal inferences and the contrast between Semmelweis’s causal concept and that deployed by many advocates of RCTs. It is argued that Semmelweis’s approach has implications for matters of explanation and medical practice. (shrink)

This paper examines the scientific controversy over the yields of genetically modified [GM] crops as a case study in epistemologically deep disagreements. Appeals to “the evidence” are inadequate to resolve such disagreements; not because the interlocutors have radically different metaphysical views (as in cases of incommensurability), but instead because they assume rival epistemological frameworks and so have incompatible views about what kinds of research methods and claims count as evidence. Specifically, I show that, in the yield debate, proponents and opponents (...) of GM crops cite two different sets of claims as evidence, which correspond to two rival epistemological frameworks, classical experimental epistemology and Nancy Cartwright's evidence for use. I go on to argue that, even if both sides of the debate accepted Cartwright's view, they might still disagree over what counts as evidence, because evidence for use ties standards of evidence to what is sometimes called the “context of application.”. (shrink)

:Proponents of behavioural policies seek to justify them as ‘evidence-based’. Yet they typically fail to show through which mechanisms these policies operate. This paper shows – at the hand of examples from economics and psychology – that without sufficient mechanistic evidence, one often cannot determine whether a given policy in its target environment will be effective, robust, persistent or welfare-improving. Because these properties are important for justification, policies that lack sufficient support from mechanistic evidence should not be called ‘evidence-based’.

The disconnect problem arises wherever there is ongoing and severe discordance between the scientific assessment of a politically relevant issue, and the politics and legislation of said issue. Here, we focus on the disconnect problem as it arises in the case of climate change, diagnosing a failure to respect the necessary tradeoff between authority and autonomy within a public institution like science. After assessing the problematic deployment of scientific authority in this arena, we offer suggestions for how to mitigate climate (...) change’s particular disconnect problem, as well as more general proposals for reforming science advising. (shrink)

The goal of this paper is to provide an interpretation of Feyerabend's metaphysics of science as found in late works like Conquest of Abundance and Tyranny of Science. Feyerabend's late metaphysics consists of an attempt to criticize and provide a systematic alternative to traditional scientific realism, a package of views he sometimes referred to as “scientific materialism.” Scientific materialism is objectionable not only on metaphysical grounds, nor because it provides a poor ground for understanding science, but because it implies problematic (...) claims about the epistemic and cultural authority of science, claims incompatible with situating science properly in democratic societies. I show how Feyerabend's metaphysical view, which I call “the abundant world” or “abundant realism,” constitute a sophisticated and challenging form of ontological pluralism that makes interesting connections with contemporary philosophy of science and issues of the political and policy role of science in a democratic society. (shrink)

Evidence-based policy is gaining support in many areas of government and in public affairs more generally. In this paper we outline what evidence—based policy is then discuss its strengths and weaknesses. In particular, we argue that it faces a serious challenge to provide a plausible account of evidence. This account needs to be at least in the spirit of the hierarchy of evidence subscribed to by evidence-based medicine (from which evidence—based policy derives its name and inspiration). Yet evidence-based policy’s hierarchy (...) needs to be tailored to the kinds of evidence relevant and available to the policy arena. The evidence required for policy decisions does not easily lend itself to randomised controlled trials (the "gold standard" in evidence-based medicine), nor, for that matter, being listed in a single all—purpose hierarchy. (shrink)