Abstract

Intuitively, Berger [1], commenting on Penston [2,3], must be right that we should not abandon RCTs. But here, intuition is no guide. Penston's strength is comprehensively marshalling the evidence, clarity of exposition, intellectual rigour and depth of analysis [4]. Dr Berger suggests some solutions: "needing better trials, more careful consideration of what can go wrong in trials, better reporting of trial results and better statistical input, we also need a more educated and involved general public". This has the ring of a political manifesto. In 1997 Britain, Tony Blair's New Labour government's emblematic theme was "Things can only get better". They didn't. With over fifty years of failure and tens of billions of dollars spent annually on medical research, is not suggesting "better" anything, too late? And how is better to be achieved? Is what exists now the best we will ever get? Berger attributes the "serious problems that occur with randomized trials" to how "they are currently conducted in practice". Penston, however shows that the problems are fundamental [4]. Large scale RCT's claiming treatment effects in small sub-groups are systematic, but not scientific. In this context, RCTs alone only ever demonstrate correlation; never causation. The answer to Berger's question "Can we attribute this failure to the involvement of statisticians, without even considering other possible explanations" is, therefore, "No". Berger asks "Is the answer, then, to dismiss statistical involvement altogether?" No. Berger is right that "statistical input is an asset, not a hindrance", but only in its rightful context and attributed an appropriate level of evidential reliability. I have little option but to agree with Penston. Large scale RCTs for small treatment effects are weak evidence of correlation and not scientific. Fraud, corruption and incompetence aside, they probably rank with early reports calling for detailed clinical investigations. The latter in turn call for real scientific peer review, rigorous replication or falsification. If that is right, are large scale RCTs to find small treatment effects worth the money or the effort? If not, what do we do? The textbooks, so far as I am aware, are yet to be written. The larger issue is the reliability of the medical evidence base. A recent paper in the IJPCM addresses the failure of EBM [5]. Its working title was Evidence Based Medicine or Ignorance Based Evidence. The medical evidence base is contradictory, conflicting and a source of a substantial body of ignorance believed to be truth, which is misdirecting medical practice daily. It is corrupt in all senses beyond tainted or adulterated [4]. As a body of evidence it, like large scale RCTs, is neither science nor scientific. Berger readily concedes "much statistical work … is grossly incompetent", but says "This does not justify condemning all statisticians with a broad brush ….. nor does it justify condemning all statistics-based research." This is the I'm only a little bit pregnant approach. It does not work. When a practitioner turns to the published medical evidence base, which papers does he or she trust? This goes well beyond statistics. James Penston has shown EBM has no roof. Neither does it possess foundations [5]. So, where to now? Given the opportunity in an academic environment, that is a question I am willing to attempt to address, if there is an institution brave enough to undertake and resource that journey.