Universidad Francisco de Vitoria Madrid 2014 Medicine is not science: guessing the future, predicting the past Universidad Francisco de Vitoria Madrid 2014 Copyright © Clifford Miller 2014 Email: cliffordmiller {at} cliffordmiller.com UK Tel: +44 208 663 0044 Knowledge-based Medicine Clifford Miller ABSTRACT There is a middle ground of imperfect knowledge in fields like medicine and the social sciences. It stands between our day-to-day relatively certain knowledge obtained from ordinary basic observation of regularities in our world and our knowledge from well-validated theories in the physical sciences. The latter enable reliable prediction a great deal of the time of the happening of events never before experienced. The former enable prediction only of what has happened before and beyond that of educated guesses which may sometimes prove right and others not when we test them. The imperfection of our knowledge between those limits is a consequence of complexity. Reductionist empiricism fails when faced with complexity but we all still have to live in a complex world where reductionist science cannot help us devise reliable theories from which we can predict the behaviour of the world around us. We cannot predict reliably. We can only monitor actively and manage actively the world around us if we want to try to control our environment. We have a limited prediction horizon. Science and empiricism work well if we want to send Voyager I and II over 3 billion miles away but not for most other things. In medicine how and when to apply probabilistic, conjectural, incomplete medical theories and explanations requires professional expertise, intuition and judgement (non scientific knowledge) which is essential. Medical diagnosis is a skill of predicting from knowledge of what has happened before that the past will recur in the current patient applying expertise and intuition from knowledge and experience of prior cases and probabilistic medical research and theory. The physician is left to an educated guess as to how the future will develop for the particular individual patient the focus of his or her current attention. Saturday, 26 July 2014 Universidad Francisco de Vitoria Madrid 2014 What is the nature of medical knowledge? 2 Copyright © Clifford Miller 2014 • 2 papers MINS* and GTF** 15,000 words • objective today to cover main points in 20 minutes • conventional analyses but unexpected outcomes • MINS co-author Donald W. Miller, Jr., MD, Emeritus Professor of Surgery, Division of Cardiothoracic Surgery, University of Washington School of Medicine, Seattle, USA • GTF introduces Irregularity Theory * ["MINS"] Medicine is Not Science. Miller, C., Miller, D. (2014) European Journal for Person Centered Healthcare 2(2) in press publication 30th June 2014 ** ["GTF"] Medicine is Not Science: Guessing the Future, Predicting the Past. Miller, C. (2014) Journal of Evaluation in Clinical Practice in press & online early view 21 June 2014 Universidad Francisco de Vitoria Madrid 2014 Knowledge-based medicine • there is no "hierarchy of evidence" in evidence * • how do you "know" and how reliable is it? • scientific, unscientific and non scientific knowledge • physical sciences (experimental) • purely observational knowledge (unscientific) • observational "sciences" (non experimental) • "soft" science (experimental but "scientific" experiments generally impossible) • professional expertise, intuition, judgement (non scientific) 3 Copyright © Clifford Miller 2014 * Miller, C., Miller, D. (2011) The Real World Failure of Evidence-Based Medicine. International Journal of Person Centered Medicine 1 (2) 295-300. Universidad Francisco de Vitoria Madrid 2014 "Soft" vs "hard" science 4 Copyright © Clifford Miller 2014 MINS Fig. 1: Scientific Validation of Theories A) Psychology: In psychology experiments the participants are an heterogeneous group. Each is a singular irregular "experiment". The outcome may be that 70% respond in the expected manner but 30% do not. A theory cannot predict which participant will respond as expected or when but only what the probability might be in any case. A scientific theory will be falsified by any one of the 30% of irregular outcomes. Whether such a theory should be applied to any future case requires professional judgement, experience and all the evidence relevant to the case. B) Space Exploration: If theories were established as valid 70% of the time, the crew and funders of a space mission would have to accept that for nearly one launch in three the craft could fail to launch, complete its mission and return. A successful mission also relies on numerous theories all being right and not just one. If each theory were right 70% of the time, for a combination of such theories the probability of their being right and hence the spacecraft completing its mission and returning safely could be an exceptionally small one and space exploration might be unlikely to be undertaken. Universidad Francisco de Vitoria Madrid 2014 Is medicine "science"? • we do not know what the word "science" means • multiplicity of meanings, contradictory, vague, meaningless, misleading 5 Copyright © Clifford Miller 2014 "The whole of science is nothing more than a refinement of everyday thinking." Albert Einstein, Physics and Reality [1936] "science" includes marketing. Anderson, P. (1983) Marketing, Scientific Progress, And Scientific Method Journal of Marketing 47: 18-31. Universidad Francisco de Vitoria Madrid 2014 6 The word "science" • "science" =/= "reliable" knowledge • it includes much unreliable "knowledge" • myth "science" = "reliable" is harmful • majority of knowledge is not "science" • "unscientific" and "non scientific" knowledge =/= "unreliable" Copyright © Clifford Miller 2014 Universidad Francisco de Vitoria Madrid 2014 7 "science" and "scientific" • here I use the "conventional conception" of science from the physical sciences as a benchmark for "science" • taught to generations of schoolchildren and undergraduates • serves to illustrate the issues • does not represent how all physical science is done • "scientific method" idealised representation • however, experimental verification is fundamental to science and consistent with Kuhnian "normal science" Copyright © Clifford Miller 2014 Universidad Francisco de Vitoria Madrid 2014 8 Physical sciences • markedly successful in applied science and technology • high standard of proof • verification by strict regularity of outcome every time in repeated reproducible experiments • theory falsified by a single irregular outcome • successful validation leads to inference theories are of universal general applicability • predictive success in new situations [eg. space travel] • not perfect eg. anomalies ignored & theories still used; some still searching for a ToE [Theory of Everything] Copyright © Clifford Miller 2014 Universidad Francisco de Vitoria Madrid 2014 9 Experimental intervention + strict regularity in science • drives direction of causation showing X causes Y • observation alone can only show if X and Y are associated • establishing causal associations requires more [Slide 24] • Kuhnian "normal science" • repeated reproducible experimental verifications • aims at elimination of bias, subjectivity and judgement • narrow evidence base • marked eventual wide consensus Copyright © Clifford Miller 2014 Universidad Francisco de Vitoria Madrid 2014 10 Observational knowledge (unscientific) • fundamental knowledge • "Reliable predictive empirical knowledge of the physical world has always been obtained by observation of regularities without needing science or theory." [GTF] • routine day-to-day observation of regularity sunrise, sunset, grass grows, water runs downhill • no science or scientific theories eg cultivation of crop from seed but mishaps & disease caused by witchcraft? • common, uncomplicated, reliable [but not as generalisation] • permits reliable prediction of what is already observed and known to happen [but not generally new outcomes or in situations not already known] Copyright © Clifford Miller 2014 Universidad Francisco de Vitoria Madrid 2014 11 Observational knowledge in medicine [GTF] • thousands of years of sophisticated medical systems based on therapeutic plants [Newman, D., Cragga, G. & Snader, K. (2000) The influence of natural products upon drug discovery. Natural Product Reports, 17 (3), 215–234.] • 400 medicinal herbs tested: 250 were recommended by former German health agency [Schulz, V., Hänsel, R., Blumenthal, M. & Tyler, V. (2004) Rational Phytotherapy A Reference Guide for Physicians and Pharmacists, Telger, T (translator), 5th edn, p. viii. Berlin, Heidelberg, New York: Springer-Verlag. Preface.]. • 25% of drugs ".. used in high tech medicine are derived from the natural world and our ancestors' experiments". [Newton, P. & Wolfe, N. (1992) Can animals teach us medicine? Over the counter and into the forest. British Medical Journal, 305, 1517–1518.]. • by 1990, about 80% of conventional drugs were either natural products or analogues inspired by them [Li, J. & Vederas, H. (2009) Drug discovery and natural products: end of an era or an endless frontier? Science, 325, 161–165.]. Copyright © Clifford Miller 2014 Universidad Francisco de Vitoria Madrid 2014 12 Observational "sciences" • experiment impossible scales of time and space: eg. geology, space physics, astronomy, anthropology • theories conjectural • validation inference from wide evidence base scientific, unscientific and non scientific clue seeking • educated guessing judgemental, subjective, biases, but can be reliable 1929 Wegener's continental drift theories dismissed • incidental experiment plate tectonics theory demonstrated data from late 1960's US seismometry intended to monitor Soviet nuclear tests Copyright © Clifford Miller 2014 Universidad Francisco de Vitoria Madrid 2014 13 "Soft" science research not "scientific" • "scientific" experiments impossible eg. medicine, psychology, economics, sociology, political science • heterogeneity makes strict regularity impossible each experiment is irregular cannot test one variable holding all else invariant independent identical experimental repetition impossible • no single causes multifactorial nature of causality in medicine • prediction from theory is uncertain = not reliable & not scientific • theories probabilistic, conjectural & incomplete explanations at best Copyright © Clifford Miller 2014 Universidad Francisco de Vitoria Madrid 2014 14 "Soft" & "hard" science part company • experimental validation of "soft" and purely observational science theories impossible • conceptually wholly different means of assessing validity "requiring expert judgement applying all available evidence in the relevant available factual matrix" [MINS] • but the application of judgement is an anathema to "science" Copyright © Clifford Miller 2014 Universidad Francisco de Vitoria Madrid 2014 Unreliable science incomplete explanations • '.. cardinal aims of science are prediction, control, and explanation; .. greatest .. is explanation. ... most inscrutable: prediction aims at truth .... Explanation .. aims at scientific understanding' * • incomplete theories are unreliable • incomplete explanations do not explain ** • misguided belief we understand when we do not • reliable prediction every time is impossible 15 Copyright © Clifford Miller 2014 * Strevens, M. (2006) Scientific explanation. In D. M. Borchert (ed.), Encyclopedia of Philosophy, second edition. Macmillan Reference USA, Detroit. ** Reiss, J. The explanation paradox Journal of Economic Methodology Vol. 19, No. 1, March 2012, 43–62 Universidad Francisco de Vitoria Madrid 2014 Where does unreliable science leave medicine? • beyond the limited extent of physical science, the most reliable knowledge is of what is known to happen • prior well documented cases and observations • "prediction from observational knowledge can remain reliable despite some theories based on it proving false" [GTF] • how and when to apply probabilistic, conjectural, incomplete medical theories and explanations • requires professional expertise, intuition, judgement (non scientific knowledge) which is essential 16 Copyright © Clifford Miller 2014 Universidad Francisco de Vitoria Madrid 2014 Conclusion: essential medical knowledge • professional expertise, intuition, judgement • needed to apply observational knowledge, medical theory and its incomplete explanations • long experience & specialisation: ".. expert intuition .. reliable when practiced in an environment of strong regularity [high validity] .. necessary .. for the development of skilled intuitions, with adequate opportunities for .. prolonged practice and rapid unequivocal feedback .." * • knowledge systems of appropriately documented analysed defined and classified cases & histories • training in expert decision making 17 Copyright © Clifford Miller 2014 *Kahneman, D. Klein, G. (2009) Conditions for Intuitive Expertise. American Psychologist 64(6), 515-526. Universidad Francisco de Vitoria Madrid 2014 Complexity Death of Clockwork Science • Henri Poincaré 1908 explained failure of reductionist science when faced with complexity • complex systems: human body, companies, economies, countries ["non linear dynamic systems"] • developing reliable predictive theories is fraught • prediction requires infinite an impossible absolute accuracy infinitesimal errors cause prediction and behaviour to diverge rapidly • limited prediction horizon so complex systems require active monitoring & management 18 Copyright © Clifford Miller 2014 Universidad Francisco de Vitoria Madrid 2014 Consequences of complexity for medicine 19 Copyright © Clifford Miller 2014 • human body is a complex system • cannot predict reliably using "science" • same consequences for medicine as unreliable science [slides 16-17] • professional expertise, intuition and judgement essential • long experience and specialisation • knowledge systems of prior cases [demonstrate regularities of the human body as a complex system to predict behaviour and manage it] • training in expert decision making Universidad Francisco de Vitoria Madrid 2014 Guessing the future, predicting the past • generalisation in GTF (consequent upon MINS): "medical diagnosis is predicting the past will recur in the current patient applying expertise and intuition from knowledge and experience of prior cases and probabilistic medical theory" [GTF] • "the past" what is already known to happen • not scientific theory alone predicting reliably a certain new previously unexperienced outcome • "treatment decisions are an educated guess about the future [prognosis]" [GTF] 20 Copyright © Clifford Miller 2014 Universidad Francisco de Vitoria Madrid 2014 Randomised controlled trials • RCTs not scientific zero predictive certainty • no knowledge to predict reliably the outcome of treatment or its risks in an individual patient • provide no mechanism for prediction beyond a probability of how often we might expect X to cause Y • RCTs do not tell us: • when factor X can be predicted to cause outcome Y • X is the only cause or factor in the cause of Y • when X does not cause Y • why or when Y will appear when X is not present 21 Copyright © Clifford Miller 2014 Universidad Francisco de Vitoria Madrid 2014 Randomised controlled trials cont'd /1 • RCTs cannot verify hypotheses about when or why X does or does not cause Y • provide no information to predict new outcomes in previously unexperienced circumstances • provide no knowledge to predict the outcome of a treatment involving two or more drugs 22 Copyright © Clifford Miller 2014 Universidad Francisco de Vitoria Madrid 2014 Randomised controlled trials cont'd /2 • treatment effect in 10% should medicine concentrate on the 10% or 90%? • why effect in 10% but not in the 90%? • what are characteristics of 10% reduce risk to 90% and improve the numbers needed to treat? • if no benefit to 90% should ADRs be more carefully investigated? 23 Copyright © Clifford Miller 2014 Universidad Francisco de Vitoria Madrid 2014 Observational statistical studies are not science • Only establish associations X is associated with Y • do not establish causation that X causes or is caused by Y • tell us nothing to predict when X and Y will present in a particular individual • establishing general causality is a matter of judgement • applying US Surgeon General's and Bradford Hill's criteria to evaluation of clinical information from the cases • professional expertise, intuition and judgement essential 24 Copyright © Clifford Miller 2014 Universidad Francisco de Vitoria Madrid 2014 Reliable unscientific medical knowledge • case reports and case series "... Permit discovery of new diseases and unexpected effects (adverse or beneficial) as well as the study of mechanisms, and they play an important role in medical education" * 25 Copyright © Clifford Miller 2014 * Vandenbroucke J. P. (2001) In Defense of Case Reports and Case Series Annals of Internal Medicine 134(4) 330-334 Universidad Francisco de Vitoria Madrid 2014 Reliable unscientific medical knowledge /2 • Single well documented spontaneous report of rechallenge or three of dechallenge prove causality: drug X causes effect Y • specialised subset of case series • highly probative equivalent to "similar fact evidence" in law R v Smith: Brides in the Bath case causation established by challenge case series of three facts which are so similar and unique • challenge case series 75% of 47 spontaneous anecdotal adverse drug reaction reports proved reliable. Remaining 12 unclear because no action taken to verify in prior 18 years* 26 Copyright © Clifford Miller 2014 * Venning GR. Validity of anecdotal reports of suspected adverse drug reactions: the problem of false alarms. Br Med J (Clin Res Ed). 1982;284:249-52. Universidad Francisco de Vitoria Madrid 2014 Reliable unscientific medical knowledge /3 • general, theoretical knowledge is not more valuable than knowledge of individual cases * • generalisation on the basis of individual cases is possible * • case studies can contribute to scientific development * • case studies are suitable for hypothesis testing and theory building * • case studies can be used for verification and falsification * • do not tend to confirm preconceived notions * • case studies can be used to summarize and develop general propositions and theories * 27 Copyright © Clifford Miller 2014 * Flyvbjerg B. (2011) "Case Study" Chapter 17 Denzin, NK. Lincoln, Y.S. eds., The Sage Handbook of Qualitative Research, 4th Edition (Thousand Oaks, CA: Sage, 2011) pp. 301-316.