On Evidence, Medical and Legal Introduction Legal Standards of Proof Medicine, like law, is a pragmatic, probabilistic activity. Both require that decisions be made on the basis of available evidence, within a limited time. In contrast to law, medicine, particularly evidence-based medicine as it is currently practiced, aspires to a scientific standard of proof, one that is more certain than the standards of proof courts apply in civil and criminal proceedings. But medicine, as Dr. William Osler put it, is an "art of probabilities," or at best, a "science of uncertainty." One can better practice medicine by using other evidentiary standards in addition to the "scientific." To employ only the scientific standard of proof is inappropriate, if not impossible; furthermore, as this review will show, its application in medicine is fraught with bias. Evidence is information. It supports or undermines a proposition, whether a hypothesis in science, a diagnosis in medicine, or a fact or point in question in a legal investigation. In medicine, physicians marshal evidence to make decisions on how to best prevent, diagnose, and treat disease, and improve health. In law, courts decide the facts and render justice. Judges and juries assess evidence to establish liability, to settle custody and medical issues, and to determine a defendant's guilt or innocence. Law applies well-defined evidentiary standards. In British and U.S. common law systems, differential standards of proof are set according to the consequences of the decision, with life and liberty prized most highly. Legal standards of proof range from the lowest, the Precautionary Principle, to the criminal standard (see Table 1). In 38 States, the highest, criminal legal standard of "beyond a reasonable doubt" can result in the defendant being put to death. Where criminal penalties are not in issue, courts resolve disputes at a lower standard. Civil cases that follow an evidentiary standard of "more likely than not" require only that the balance of probability be greater than 50 percent to support, or undermine, a disputed proposition. U.S. courts, unlike those of the UK at present, use a higher civil standard of "clear and convincing." This standard is applied when settling disputes involving child custody, involuntary commitment, withdrawal of life support in comatose patients, and determination of a "punishable frame of mind" driven by malice, oppression, or fraud. It is also used in some administrative disciplinary proceedings for attorneys, medical professionals, and other cases. 1 2 The Scientific Standard of Proof The Precautionary Principle Science prizes objective certainty. For a hypothesis to be proved, or a theory to become theorem, the evidence supporting it must be irrefutable. But science does not uniformly adhere to this standard. Subjective opinions and consensus among scientists often supersede the stricture of irrefutability. Hence, scientific standards of proof are not uniform and well defined, in contrast to legal standards. Standards of measurement, ways of reporting and evaluating results, and particular types of experimental practices vary. As a result, there is no simple and reducible algorithm against which "good" science can be evaluated. There is another aspect of the scientific standard of proof that particularly impacts medicine. Science's quest for objective certainty admits only a narrow range of evidence. The Precautionary Principle is derived from the 1990 Bergen Declaration, which states, "Where there are threats of serious or irreversible damage, lack of full scientific certainty should not be used as a reason for postponing measures to prevent environmental degradation." As currently practiced, governments implement policies and regulations based on what "might" cause harm, even if there is little or no evidence that a hazard exists. This principle increasingly governs state regulatory policy and international environmental law; and regulators employ it to ban DDT, reduce supposedly harmful CO emissions, and bar planting of genetically engineered crops. It is broadly analogous to "probable cause," and thus a lower standard than for a prima facie case. In the European Union, under this standard a decision is taken on the "available" evidence: "the real risk alleged for public health appears sufficiently established on the basis of the latest scientific data available." Like requiring medical evidence to meet a scientific standard of proof, rendering regulatory decisions based on the Precautionary Principle, without requiring any evidence on their risk and benefits, 3 4 5 6 7 8 9 2 Donald W. Miller, Jr., M.D. Clifford G. Miller, Esq. Table 1. Legal and scientific proof 70 Journal of American Physicians and Surgeons Volume 10 Number 3 Fall 2005 must be questioned. The benefits achieved from having banned DDT are disputed; and, not having access to this pesticide, 50 million people have died from DDT-preventable malaria. Evidence-based medicine (EBM) promotes the "...use of current best evidence in making decisions about the care of individual patients." Only well-designed, randomized controlled trials (RCTs) produce medical evidence that can meet the scientific standard of proof. Systematic reviews ("meta-analyses") of multiple RCTs are even better. Meta-analyses are the "gold standard" of scientific medical evidence, and EBM proponents put them at the top of the EBM evidence pyramid. Investigators have carried out seven randomized controlled trials on transmyocardial laser revascularization (TMR). In this procedure the surgeon burns 1-mm full-thickness holes through the heart muscle with a laser, 1 cm apart in a line from the base to the apex, and then in other lines 1 cm from each other, for a total of 20 to 40 channels. These channels and the capillaries that grow out from them provide a way for blood being pumped out of the left ventricle to nourish the myocardium. The channels seal over on the epicardial side and mimic the sinusoids in a reptile's heart, which has no coronary arteries. These RCTs prove beyond a reasonable doubt, if not irrefutably, that TMR relieves angina, improves myocardial perfusion, and reduces the need for subsequent angina-related hospitalizations. Accordingly, ACC/AHA guidelines now recommend TMR as a "Class IIA" therapy for intractable angina, which means the "weight of evidence is in favor of usefulness/efficacy," with a "Level of Evidence: A," i.e., "data derived from multiple randomized clinical trials." These ACC/AHA guidelines apply to the average patient in the population with intractable angina. Medicine endeavors to make decisions for individual patients in the context of population-based information like this. Data from these TMR trials do not provide information the surgeon needs to treat a specific patient. Some patients with small coronary arteries might benefit from TMR done in conjunction with coronary bypass surgery, since graft patency rates are low in these patients. The operative mortality for TMR is higher in patients with poor ventricular function. Should the surgeon use an intra-aortic balloon pump in these patients? These trials do not provide answers to treatment questions like this. With regard toAlzheimer's disease, Saver and Kalafut calculate that 127 RCTs would have to be done in 63,500 patients over a 286year period to determine the optimal combination of agents to treat this disease. Systematic reviews combine trials that address similar questions, like whether albumin or crystalloid is better for volume expansion, in order to achieve a statistically more certain conclusion. The Cochrane Injuries Group Albumin Reviewers in Britain performed a meta-analysis in 1998 of 30 RCTs on volume replacement in critically ill trauma victims, and they found that the risk of death was 6 percent higher in patients given albumin rather 10 11 12 13,14 15-21 22 23 Randomized Controlled Trials Meta-Analyses than crystalloid. It is notable that none of the study's seven analysts had experience working in an intensive care unit. When the study was published, the (London) reported that it "suggests that up to 30,000 patients in Britain alone have died because they were treated with human albumin solution." The director of the Cochrane Centre in Oxford said that he would sue any doctor who gave him an infusion of albumin and that patients should seek redress in the courts for clinical negligence if the guidelines based on this analysis were transgressed. Another systematic review on this subject, published in 2001, analyzed 55 RCTs, including ones that had a lower mortality with albumin that the first meta-analysis left out. This 2001 study concluded that albumin has no adverse effect on mortality. Analysts employ statistical techniques in their systematic reviews that include a numerical scale for weighting the quality of each trial. Juni and colleagues show how analysts can obtain diametrically opposing results depending on which of the more than 25 scales they use to distinguish between highand lowquality RCTs. Another source of bias is the study's sponsor. The UK's National Health Service (NHS), which stocks albumin and crystalloid in its hospitals, funded the 1998 albumin meta-analysis. Albumin is 30 times more expensive than crystalloid, and the study's sponsor would save a lot of money if it only had to purchase crystalloid. Other meta-analyses suffer similar flaws, such as a recently published one claiming that high-dose vitamin E supplements increase mortality. Critics have exposed the methodological flaws in this study. Randomized trials provide epidemiologic evidence framed in terms of statistical significance. Epidemiology examines the incidence of disease and the effects of therapeutic interventions at the population level. It cannot answer the question of whether causes in a specific individual. The U.S. Federal Judicial Center's states: "Epidemiology...does not address the question of the cause of an individual's disease. This question...[of]...specific causation is beyond the domain of the science of epidemiology.... [It] addresses whether an agent can cause a disease, not whether an agent did cause a specific disease" Epidemiology can show that an association exists between the agent in question and a given toxicity or disease, at the population level. Epidemiologic evidence cannot establish a causal association unless other biological evidence backs it up. The Bradford Hill criteria spell out what that evidence needs to be. Regardless of these criteria, some U.S. courts will admit epidemiology as evidence justifying an inference of causation in toxic tort litigation on a "balance of probability" when the relative risk is shown to exceed 2.0. U.S. courts also admit studies with a lower relative risk while recognizing that such studies may be insufficient proof of specific causation. Evidence from epidemiologic RCTs does not necessarily meet a scientific standard of proof. Indeed, biases in methodology can generate evidence that does not even meet the lowest legal-civil standard of proof. These include faulty trial protocols, reporting outcomes in terms of relative risk without giving absolute risk of all24 25 26 27 28 29,30 2, p 381 30,31 2, p 384 Times x y Reference Manual on Scientific Evidence Epidemiologic Evidence Journal of American Physicians and Surgeons Volume 10 Number 3 Fall 2005 71 cause deaths, and justifying interventions on surrogate outcomes (e.g., cholesterol level) when the primary outcome (freedom from myocardial infarction and survival) is not improved. The investigator's interpretation of the trial's results is especially prone to bias. And, as seen in the NHS albumin meta-analysis, a study's source of funding can affect its results. Als-Nielson and colleagues found that RCTs funded by pharmaceutical companies are significantly more likely to recommend the experimental drug as the treatment of choice than are studies funded by organizations that have no financial stake in the outcome. Chan and Altman reviewed 519 RCTs that were published in December 2000 and indexed in PubMed. They found that incomplete reporting of outcomes (described in the methods section but not in the results section) was common, and conclude that the medical literature of randomized trials represents a selective and biased subset of study outcomes. As one observer put it, "Epidemiological analysis is notoriously susceptible to misinterpretation, and even manipulation. Two sets of researchers can extract diametrically opposed results from the same data." The pharmaceutical and biotech industries now fund more than 60 percent of the RCTs that medical journals publish, which raises the concern that supposedly objective science is being turned into a marketing tool. EBM protagonists place case reports near the bottom of the medical evidence pyramid alongside editorials and opinions. They call this eyewitness-like testimony "anecdotal." Nevertheless, like witness testimony in the courtroom, the most essential evidence in medicine is the patient's story. In a court of law, eyewitness testimony is often the primary source of information that the court must use to reach a verdict. Prosecutors and defense attorneys cross-examine witnesses to plumb the evidentiary reliability of their testimony and introduce, when available, more scientific, "hard" evidence, such as DNAhair analysis, that can corroborate it. Most medical evidence does not meet the scientific standard of proof; and, as in law, it should be judged by a standard of proof appropriate to the fact or point in question. An "anecdotal" case report can provide evidence of probative value, just like eyewitness testimony in a murder trial.And it can be similarly tested, by second opinions, re-examination, laboratory tests, and follow-up. A single case report can prove that a drug causes an adverse reaction. Three events related to administration of the drug prove specific causation: 1) -the reaction occurs after the drug is given; 2) -it resolves when the drug is discontinued; and 3) -the adverse event recurs when the drug is given a second time. Causation is judged to be certain owing to this "double hit" of challenge and re-challenge. The U.S. Food and Drug Administration (FDA) and pharmaceutical companies acknowledge that just one 33 34 35,36 37 38 39 40 41 2 42 43 Eyewitness Testimony Case Reports Specific Causation challenge de-challenge re-challenge challenge/de-challenge/re-challenge (CDR) case proves causality. The FDA states, "Even a single well-documented case report can be viewed as a signal [of causation], particularly if the report describes a positive re-challenge." In another report, the FDA notes that determining causality includes "assessment of temporal relationships [and] de-challenge/re-challenge information...which is usually considered your strongest evidence of a causal association." And as puts it, a positive re-challenge is "probably the strongest proof of a causal relationship." If giving the drug a second time is not done, owing to ethical considerations, three cases of challenge/de-challenge (CD) can prove causality. Heparin causes thrombocytopenia in a small percentage of patients (2-3 percent). In one patient, after a 10-day course of heparin the platelet count dropped from 200,000/mm to 60,000. Over the next 20 days, off heparin, it returned to normal (179,000). Asecond bolus of heparin was then given, which promptly dropped the platelet count to 49,000. No other causes for thrombocytopenia were evident, and the presence of heparin/platelet factor 4 antibodies provides biologic plausibility on how heparin can cause this adverse effect. This single case proves that heparin causes life-threatening thrombocytopenia in some people. Likewise, one CDR case of suicide ideation after taking flouxetine (Prozac) is sufficient to prove that the drug causes this reaction. With regard to drugs and vaccines, the Institute of Medicine (IOM) acknowledges that "[t]he recurrence or nonrecurrence of the adverse event will often have a major impact on the causality assessment." The judiciary follows well-developed rules on admissibility of evidence. Hearsay evidence is not admissible (except for civil cases in the UK and certain well-defined areas such as business records in criminal cases) nor is opinion evidence (except for expert opinion on technical and scientific matters). "Similar fact evidence" is normally inadmissible in English law criminal proceedings unless its value as proof outweighs its prejudicial effect. In the case, the defendant, George Smith, was accused of drowning his bride in the bathtub. No physical evidence implicated him in her death, but she had signed over her estate to him on their betrothal. Evidence was admitted at trial that this person, using different names, had married two other women who also drowned in their bathtubs. They too had made financial arrangements from which he would benefit. This evidence was strong proof that outweighed its prejudicial effect. It was sufficient to find Smith guilty as charged, and he was executed in 1915. This early English law example shows that similar fact evidence is analogous to challenge/de-challenge/re-challenge evidence in medicine. Both are capable of demonstrating causality to the highest standards of proof. In , their deaths precluded a de-challenge, but such evidence is essentially the same as three CD cases in proving causation. This also demonstrates that the plausibility of a single case report can be reinforced by each subsequent report, whereby a case series taken together can provide a substantially higher degree of proof than each report taken individually or isolated spontaneous reports of adverse 44 45 46 47 48 49 50 51 Stephens' Detection of New Adverse Drug Reactions Brides in the Bath Brides in the Bath 3 Similar Fact Evidence vs. Case Series and De-Challenges 72 Journal of American Physicians and Surgeons Volume 10 Number 3 Fall 2005 events. One such case series is that of Wakefield et al., which shows a possible association of autistic regression, intestinal complaints, and ileal lymphoid-nodular hyperplasia following MMR vaccination in 12 children. An epidemic of autism afflicts children today. Fifty years ago fewer than one in 10,000 children had this devastating malady, but today, with a prevalence of one in 166, one in every 68 American families has an autistic child. A number of parents with autistic children and some investigators believe that the measles-mumpsrubella (MMR) vaccine and/or vaccines that contain thimerosal, especially in combination, can cause autism. Indeed, the director of the Autism Research Institute states, "Thousands of parents report-and demonstrate with home videos-that their children were normal and responsive until suffering an adverse vaccine reaction." Medical practitioners first inject the MMR vaccine into American children at age 12-15 months, and then a second time when they are ages 4 to 6. Injecting two widely separated doses of this vaccine constitutes a challenge/de-challenge/re-challenge in susceptible children. A valid way to test the hypothesis that MMR vaccine causes autism is to adopt the methodology that the FDAand pharmaceutical companies use to show that a particular drug causes an adverse reaction-a CDR case report or CD case series. One well documented case of a normally developing child who becomes autistic after being given the MMR vaccine, improves with therapy, and then regresses following the second dose (re-challenge) would be strong proof that this hypothesis is true. Public health officials and their respective medical establishments in the United States and United Kingdom will not accept this kind of evidence with regard to vaccines, stating: "The weight of currently available scientific evidence does not support the hypothesis that vaccines cause autism." For them, only epidemiologic evidence is sufficiently "scientific." But epidemiologic evidence, as an application of statistics, is open to manipulation and bias. Since it does not meet the scientific standard of irrefutability, it is not per se "scientific." The chairman of the IOM Committee on Immunization Safety Review acknowledges that "[the Committee] does not exclude the possibility that MMR vaccine could in rare cases contribute to autistic spectrum disorders ... because epidemiological evidence lacks the precision to assess rare occurrence and the proposed biological models, although far from established, are nevertheless not disproved." Absence of evidence is not evidence of absence. Clinical importance is not equivalent to statistical significance. With rare and uncommonly occurring diseases, a nonsignificant finding in a randomized trial does not necessarily mean that there is no causal association between the agent in question and the disease. Such trials are subject to a false-negative Type II error, which incorrectly supports the null hypothesis that agent does not cause disease In a legal case, lawyers organize the evidence they obtain to create a "factual matrix." Elements of information are corroborated 52 53,54 55 56 57 58 59 Medical Evidence inAutism Commonality of Medical and Legal Evidence x y. and cross-correlated to present a consistent, linked set of facts. Law tests the reliability of the sources of information, in addition to testing the information the sources supply. It admits evidence from a broad range of sources, which include human witnesses, documents, and machine "witnesses" (material on computers, audio, and video). Courts tend to exclude information, like hearsay evidence, if the court lacks the means to test its reliability. Medical evidence is the same. It begins with admitting (of necessity) the patient's oral account. Labeling witness testimony "anecdotal" does not render it inherently unreliable. Oral or eyewitness evidence and "anecdote" are not synonymous. Eyewitness evidence can be tested In a medical case, physicians also marshal evidence from a variety of sources to create a factual matrix. They include, in addition to statistical epidemiologic evidence, case reports, case series, their own clinical experience and judgment, the opinions of others. And medicine, like law, has various means for testing and assessing evidence, which include reproducibility and predictability in addition to statistical significance. Medical evidence spans the gamut of proof, from "more likely than not" to "irrefutable." In writing "evidence-based" testing and treatment guidelines, EBM advocates make recommendations based only on evidence obtained from controlled trials and meta-analyses. Considered "best practices," such guidelines are now used by government agencies, third-party payers, and managed care organizations to decide coverage and track physicians' "quality of care." But the factual matrix in each patient, which includes genetic and biologic variations and coexisting diseases, renders application of these epidemiologically based guidelines problematic. There are often special circumstances in a particular patient, described by Welsby as "Type 3 complexity," that guidelines do not address. The commonality of medical and legal evidence helps expose the inherent flaw in these EBM two-dimensional, reductionist flowcharts. In scientific and technical matters, judges and juries rely on the testimony of individual experts. The U.S. legal system has rules of evidence that regulate the admission of such testimony. But these rules of evidence do not question or regulate the rules, methods, procedures, and evidence generally accepted in medicine. Expert testimony based on flawed medical evidentiary practices will continue to fail courts and litigants and result in unreliable and unjust court decisions. U.S. courts always have had power to exclude or admit medical, scientific, or other technical evidence. In 1923 the U.S. Circuit Court of Appeals laid down a "general acceptance" test for the admission of novel scientific opinion testimony in , which stated: "The thing from which the deduction is made must be sufficiently established to have gained general acceptance in the particular field to which it belongs." The Court affirmed the trial judge's refusal to admit evidence of the results of a "systolic blood pressure deception test" (a predecessor to the polygraph). Following the 1975 enactment of new Federal Rules of Evidence, and particularly Rule 702 dealing with scientific evidence, in 1993 the U.S. Supreme Court, in . Dealing with 60 61 62 p 1014 Daubert Frye v United States Daubert v. Dow 63, Journal of American Physicians and Surgeons Volume 10 Number 3 Fall 2005 73 Merrell Pharmaceuticals, Inc. Daubert Daubert Daubert Reference Manual on Scientific Evidence in vitro Daubert Daubert , for the first time obliged federal judges to be proactive and screen the medical scientific evidence of individual experts in toxic tort litigation to ensure it is relevant and reliable. makes judges "gatekeepers" of medical/ scientific expert testimony measured against the benchmarks of existing knowledge and practice. A judge must now ascertain whether scientific evidence is grounded "in the methods and procedures of science...." The Court emphasized that the "inquiry envisioned by Rule 702 is...a flexible one." It then identified four factors to consider when assessing whether a theory or technique is derived scientifically. These include its methodology, testability, subjection to peer review, and general acceptance by the scientific community. In practice, is vulnerable to manifold corruptions resulting in relevant reliable evidence being systematically excluded in favor of the less reliable. rules do not correct erroneous theories that have become accepted medical thinking, including theories about what evidence is reliable. Editors can subvert peer review by selecting only reviewers who will reject papers that run counter to-or praise papers that support-the interests of journal's advertisers or its owners. Lines of independent research contradicting conventional wisdom can systemically remain unpublished. Such hard-to-publish research may prove that what the scientific community generally accepts as correct is, in fact, wrong. Research follows the funding, resulting in a wealth of publications favoring the funding interests. This can have a disproportionate effect on the "weight" of evidence, especially for epidemiologic evidence in court. According to some leading trial lawyers, plaintiffs now have to demonstrate near certainty before a court will allow a novel scientific theory to prevail (Waters CAof Waters & Kraus, personal communication, 2004). Following the lead of evidence-based medicine, U.S. courts place a premium on epidemiologic data. Before a U.S. judge will allow the plaintiff to prove specific causation, epidemiologic evidence that a causal association exists between the agent in question and a given toxicity or disease must normally be presented first. The states: "[A]n agent cannot be considered to cause the illness of a specific person unless it is recognized as a cause of that disease in general." After jumping this hurdle, the judge will then admit other medical evidence for proving specific causation, such as CDR case reports and CD case series, pharmacological research on mechanisms of toxicity, and animal and tissue studies. In U.S. federal courts and in an increasing number of state courts that have adopted , this epidemiologic prerequisite has blocked litigation on harm done by mercury amalgams, thimerosal, and MMR vaccine. With regard to uncommonly occurring and rare events like adverse drug reactions and vaccine-induced autism, judges need to realize that a CDR case report and CD case series alone can prove causation to a very high standard. Courts will be informed of apposite evidence of this kind if, and only if, evidence in medicine and medical science does the informing. Moreover, aside, for this to happen, medicine needs to develop a better understanding of the nature of evidence and of evidentiary proof, by emulating law's approach to evidence. Law in turn needs a better understanding of the shortcomings of medicine's current approach to evidence. 58 59 66 2, p 385 67 Donald W. Miller, Jr., M.D., Clifford G. Miller, Esq. RENCES is a professor of surgery in the Division of Cardiothoracic Surgery at the University of Washington School of Medicine. E-mail: dwm@u.Washington.edu. is a commercial lawyer in private practice, London, England; BSc Hons (Physics) ARCS; Solicitor of the Supreme Court of England; and former examining lecturer in law, London University. E-mail: cgmiller@cliffordmiller.com. The authors have no conflicts of interest to disclose. Silverman ME, Murray TJ, Bryan CS, eds. . Philadelphia, Pa.: American College of Physicians; 2002. Federal Judicial Center. . 2 ed. Palm Beach Gardens, Fla.: LPR Publication; 2000. Available at: http://www.fjc.gov/public/pdf.nsf/lookup/sciman00.pdf/$file/sciman00. pdf. Accessed Jan 15, 2005. Feverabend P. . London: Verso; 1993. Collins H, Pinch T. . Cambridge, Mass.: Cambridge University Press; 1993. Gieryn T. Boundaries of science. In: Jasanoff S, Markle G, Petersen J, Pinch T, eds. . Thousand Oaks, Calif.: Sage; 1995:393-443. Chase A. Some cautionary remarks about the precautionary principle. Available at: http://www.apec.org.au/docs/chase.pdf. Accessed Feb 3, 2005. Whelan EM. Our stolen future: a reckless use of the precautionary principle. Committee for a Constructive Tomorrow; 1997. Available at: http://www.cfact.org/IssueArchive/stolen-future.497.txt. Accessed Feb 3, 2005. Goklany IM. . Washington, D.C.: Cato Institute; 2001. (2003) ECR I-9693, §48. Edwards JG. DDT: a case study in scientific fraud. 2004;9:83-88. Sackett DL, Straus SE, Richardson WS, et al. 2 ed. Edinburgh: Churchill Livingstone; 2000. Evidence-Based Medicine Working Group. Evidence-based medicine: a new approach to teaching the practice of medicine. 1992;268: 2420-2425. The Evidence-Based Medicine Working Group, Guyatt G, Rennie D, eds. . Chicago, Ill.: AMA Press; 2002. Sackett DL, Rosenberg WMC, Muir Gray JA, et al. Evidence-based medicine: what it is and what it isn't. 1996;312:71-72. Frazier OH, Tuzun E, Eichstadt H, et al. Transmyocardial laser revascularization as an adjunct to coronary artery bypass grafting: a randomized, multicenter study with 4-year follow-up. 2004;31:231-239. Horvath KA, Aranki SF, Cohn LH, et al. Sustained angina relief 5 years after transmyocardial laser revascularization with a CO(2) laser. 2001;104(Suppl I):81-84. Frazier OH, March RJ, Horvath KA. Transmyocardial revascularization with a carbon dioxide laser in patients with end-stage coronary artery disease. 1999;341:1021-1028. Horvath KA, Cohn LH, Colley DA, et al. Transmyocardial laser revascularization: results of a multicenter trial with transmyocardial laser revascularization used as sole therapy for end-stage coronary artery disease. 1997;113:645-653. Schofield PM, Sharples LD, Caine N, et al. Transmyocardial laser revascularization in patients with refractory angina: a randomised controlled trial. 1999;353:519-524. Allen KB, Dowling RD, Fudge TL, et al. Comparison of transmyocardial revascularization with medical therapy in patients with refractory angina. 1999;341:1021-1028. Burkhoff D, Schmidt S, Schulman SP, et al, for the ATLANTIC Investigators. Angina Treatments-Lasers and Normal Therapies in Comparison. Transmyocardial laser revascularization compared with continued medical therapy for treatment of refractory angina pectoris: a prospective randomised trial. 1999;354:885-890. REFE 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20. 21 The Quotable Osler Reference Manual on Scientific Evidence Against Method The Golem: What Everyone Should Know About Science Handbook of Science and Technology The Precautionary Principle: A Critical Appraisal of Environmental Risk Assessment Commission v. Denmark J Am Phys Surg Evidence-Based Medicine: How to Practice and Teach EBM. JAMA Users' Guide to the Medical Literature: Essentials of Evidence-Based Clinical Practice BMJ Tex Heart Inst J Circulation N Engl J Med J Thorac Cardiovasc Surg Lancet N Engl J Med Lancet nd nd 74 Journal of American Physicians and Surgeons Volume 10 Number 3 Fall 2005 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 ACC/AHA 2004 Guideline Update for Coronary Artery Bypass Graft Surgery: A report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (Committee to Update the 1999 Guidelines for Coronary Artery bypass Graft Surgery). Available at: http://www.acc.org/clinical/guidelines/cabg/index.pdf/. Accessed Jan 30, 2005. Saver JL, Kalafut M. Combination therapies and the theoretical limits of evidence-based medicine. 2001;20:57-64. Cochrane Injuries Group Albumin Reviewers. Human albumin administration in critically ill patients: systematic review of randomized controlled trials. 1998;317:235-240. Horsey PJ. The Cochrane 1998 albumin review-not all it was cracked up to be. 2002;19:701-704. Wilkes MM, Navickis RJ. Patient survival after human albumin administration: a meta-analysis of randomized, controlled trials. 2001;135:149-164. Juni P, Witsch JP, Bloch R, Egger M. The hazards of scoring the quality of clinical trials for meta-analysis. 1999;28:1054-1060. Miller ER, Pastor-Barriuso R, Dalal D, et al. Meta-analysis: high-dosage vitamin E supplementation may increase all-cause mortality. 2004;142:37-46. Graby AR. Study flaws cast doubt on claims of vitamin E danger. Nov 18, 2004. Available at: http://www.nnfa.org/images/vitamine_images/ Healthnotes.pdf. Accessed Jan 22, 2005. Council for Responsible Nutrition. Vitamin E meta-analysis in : what's wrong with this picture? Available at: http:// www.crnusa.org/vitaminEmetanalysis.html. Accessed Jan 22, 2005. Hill AB: The environment and disease: association or causation? 1965;58:295-300. Phillips CV, Goodman KJ. The missed lessons of Sir Austin Bradford Hill. 2004;1:3. Available at: http://www.epiperspectives.com/content/1/1/3. Accessed Apr 10, 2005. Kauffman JM. Bias in recent papers on diets and drugs in peer-reviewed medical journals. 2004;9:11-14. McCormack J, Greenhalgh T. Seeing what you want to see in randomised controlled trials: versions and perversions of the UKPDS [UK prospective diabetes study] data. 2000;320:1720-1723. Lexchin J, Bero LA, Djulbegovic B, Clark O. Pharmaceutical industry sponsorship and research outcome and quality: systematic review. 2003;326;1167-1170. Melander H, Ahlqvist-Rastad J, Meijer G, Beermann G. Evidence b(i)ased medicine-selective reporting from studies sponsored by pharmaceutical industry: review of studies in new drug applications. 2003;326:1171-1173. Als-Nielsen B, Chen W, Gluud C, Kjaergard LK. Association of funding and conclusions in randomized drug trials: a reflection of treatment effect or adverse events? 2000;290:921-928. Chan AW, Altman DG. Identifying outcome reporting bias in randomized trials on PubMed: review of publications and survey of authors. 2005;330(7494):753. Available at: http://bmj.bmjjournals.com/ cgi/content/short/bmj.38356.424606.8Fv1. Accessed Apr 10, 2005. Kirby, D. : . New York, N.Y.: St. Martin's Press; 2005. Brownlee S. Doctors without borders: why you can't trust medical journals anymore. , April 2004. Available at: http://www. washingtonmonthly.com/features/2004/0404.brownlee.html. Accessed May 29, 2005. Malterud K. Reflexivity and metapositions: strategies for appraisal of clinical evidence. 2002;8:121-126. Miller CG. Unreliability of scientific papers as evidence. BMJ.com Rapid Responses to: Mayor S. Authors reject interpretation linking autism and MMR vaccine. 2004;328:602. Available at: http://bmj.bmjjournals.com/cgi/ eletters/328/7440/602-c#52948. Accessed Jan 29, 2005. Cook County Bureau of Health Services. Reporting adverse events related to investigational drugs or devices. . Available at: http://www.cookcountyresearch.net/ade.html. Accessed Jan 30, 2005. Neuroepidemiology BMJ EJA Ann Intern Med JAMA Ann Intern Med Annals of Internal Medicine Proc R Soc Med Epidemiol Perspect Innovations J Am Phys Surg BMJ BMJ BMJ JAMA BMJ Evidence of Harm Mercury in Vaccines and the Autism Epidemic: A Medical Controversy Washington Monthly J Eval Clin Pract BMJ Grants Handbook 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 Novartis. ; 2002 Available at: http:// www.novartis-transplant.com/medpro/pdf/DrugInteractions2002.pdf. Accessed Jan 30, 2005. U.S. FDA Center for Biologics Evaluation and Research. Guidance for industry: good pharmacovigilance practices and pharmacoepidemiologic assessment, March 2005. Available at: http://www.fda.gov/ cber/gdlns/pharmacovig.htm. Accessed Jul 23, 2005. U.S. FDA. Postmarketing safety reporting for human drug and biological products including vaccines, March 2001. Available at: http://www. fda.gov/cber/gdlns/safety031201.pdf. Accessed Jan 29, 2005. Talbot J, Waller P, eds. . 5 ed. Hoboken, N.J.: Wiley; 2004. Warkentin TE, Kelton JG. Temporal aspects of heparin-induced thrombocytopenia. 2001;344:1286-1292. Rothschild AJ, Lock CA. Re-exposure to fluoxetine after serious suicide attempts by three patients: the role of akathisia. 1991;52:491493. Institute of Medicine, Vaccine Safety Committee, Stratton KR, Howe CJ, Johnson RB, eds. . Washington, D.C.: National Academy Press; 1994. 1915) 11 Cr. App. R. 229. Wakefield AJ, Murch SH, Anthony A, et al. Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children. 1998;351(9103):637-641. Yazbak FE. Autism in the United States: a perspective. 2003;8:103-107. CDC National Center of Birth Defects and Developmental Disability Autism Information Center. Available at: http://www.cdc.gov/ncbddd/ dd/aic/about/default.htm#common. Accessed Apr 11, 2005. Rimland B. The autism epidemic is real, and excessive vaccinations are the cause; 2003. Available at: http://www.taap.info/Rimland.htm. Accessed Feb 1, 2005. CDC Recommended Childhood Immunization Schedule. Available at: http://www2a.cdc.gov/nip/scheduler_le/schedule.asp. Accessed Feb 1, 2005. CDC National Immunization Program: vaccines and theory. Available at: http://www.cdc.gov/nip/vacsafe/concerns/autism/default.htm. Accessed Feb 2, 2005. Statement of Marie McCormick, M.D., Sc.D., before the Committee on Government Reform, U.S. House of Representatives. Apr 25, 2001. Available at: http://www7.nationalacademies.org/ocga/testimony/ Autism.asp. Accessed Feb 2, 2005. Altman DG, Bland M. Absence of evidence is not evidence of absence. 1995;311:485. Latov N. Evidence-based guidelines: not recommended. 2005;10:18-19. Brase T. How technocrats are taking over the practice of medicine. Citizens' Council on Health Care Policy Report, January 2005. Available at: http://www.cchconline.org/pr/pr121304.php. Accessed Jan 27, 2005. Welsby PD. Reductionism in medicine: some thoughts on medical education from the clinical front line. 1999;5:125-131. 293 F. 1013 (D.C. Cir. 1923). 509 U.S. 579 (1993) (No. 92-102). Bernstein DE. The admissibility of scientific evidence after 1993-1994;15:21392175. Edmond G, Mercer D. Recognising : what judges should know about falsificationism. 1996;5:29-42. Chirba-Martin MA, Welshhans CM. An uncertain risk and an uncertain future: assessing the legal implications of mercury amalgam fillings. 2004;14:293-324. Cyclosporine Drug Interaction Manual Stephens' Detection of New Adverse Drug Reactions N Engl J Med J Clin Psych Adverse Events Associated with Childhood Vaccines: Evidence Bearing on Causality R v. Smith ( Lancet J Am Phys Surg BMJ J Am Phys Surg J Eval Clin Pract Frye v. United States Daubert v. Merrell Dow Pharmaceuticals, Inc., Daubert v. Merrell Dow Pharmaceuticals, Inc. Cardozo Law Rev Daubert Expert Evidence Health Matrix: J Law Med th Journal of American Physicians and Surgeons Volume 10 Number 3 Fall 2005