Abstract
Clinical trials play a prominent role today in medicine, but are not without controversy. These issues start from the day physicians begin their specialization process in medical school and continues onto their day-to-day practice as attendings with referral patterns and resulting financial incentives. This combined with the lack of training in basic issues of epistemology and statistics, allows poor interpretations of clinical trials to reign free. A proposal to integrate the notion of severity to help remedy these issues are made by ensuring that studies are tested “As Severe As Reasonably Possible” (ASARP) will put P-values, multiplicity adjustments, and model checking in their rightful place. In light of this, multiple clinical trials are examined with an in-depth analysis of erroneous statistical inferences taken from phase III randomized trials, non-inferiority, early phase, as well as observational studies. Severe testing in the setting of medicine will help with alleviate some of the issues with the current misinterpretation of clinical studies.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
I am not accusing physicians of having nefarious motives, but want to point out the power financial incentives they face when making these decisions.
Overall survival refers to the patient being alive at that time point, whereas progression free survival indicates the patient is alive and has not had a recurrence of cancer.
Hierarchical order refers to the data passing the first step/endpoint of the hierarchy before moving on to the next level / endpoints. This can be seen in Figs. 5 and 6. For Fig. 5, one must meet an α = 0.025 for the primary endpoint of overall survival (OS). If one does not meet this criterion, then the other endpoints on the next level (OS in PD-L1% in CPS patients) cannot be tested (further discussed in Sect. 7.1).
Harrell notes there is “no principled recipe for how they [multiplicity issues] should be handled,” meaning no way to adjudicate, which multiplicity method is best (Harrell & Slaughter, 2021, pp. 5–11). As noted, as long as they are in the ASARP sample space, the differences in multiplicity may not make much difference.
Clinicaltirals.gov is a database of worldwide clinical studies provided by the U.S. National Library of Medicine.
There certainly is controversy regarding the use of different inductive methods, such as the use of Bayesian statistics and the causal inference in medicine, however, there is no doubt the frequentist methodology is the reigning paradigm in medicine. The error statistical school that uses severe testing rests firmly in the frequentist camp and so the philosophical basis is being noted here without regard to the bigger controversy of Bayesianism vs Frequentism, which is beyond the scope of this article. The reader is directed to the works of Judea Pearl, Frank Harrell, and Richard McElreath for their contribution to Bayesian statistics and causal inference.
The protocol for these trials can be found in the supplementary materials section of the referenced papers where the complete statistical information can be found, except where noted. First line treatment would be the initial treatment of a cancer. “Second line” refers to the second treatment a patient would receive after recurrence whether that is in the same site or with spread to another organ (i.e. metastatic disease). Curative treatment is not given for patients who are initially diagnosed with metastatic cancer, so the treatment for recurrent and metastatic cancer are often the same.
The other trials presented here have the same pattern. Early animal and human trials, provide the basis for the HR target. The HR ratio calculation is based on expert opinion. The 90% power and one sided α = 0.025 are simply standard clinical trial targets.
Food and Drug Administration guidance notes that “Type I error probabilities can also apply to one-sided hypothesis tests, in which case they refer to the probability of concluding specifically that there is a beneficial difference due to the drug when there is not. The most widely-used values for alpha are 0.05 for two-sided tests and 0.025 for one-sided tests” (Food and Drug Administration, 2017).
PD-L1 positivity denotes patients who will likely respond to immunotherapy, thus are tested in their own cohort. PFS = Progression Free Survival; ORR = Overall Response Rate; CPS = Combined Positive Score, which is defined as PD-L1 staining tumor cells, lymphocytes, and macrophages. Sometimes Tumor Proportion Score (TPS) is used in immunotherapy trials, which only look at PD-L1 positivity in the tumor cells.
Traditional cytotoxic chemotherapy, such as cisplatin, carboplatin, and 5-FU that were used in this trial, does not hone in on cancer cells directly, rather they take advantage of the fact that they effect tumor cells to a greater degree than normal cells. The other agents in the trial cetuximab (used in the EXTREME arm) and pembrolizumab have cell specific targets and are lumped into a class of treatments often called “targeted agents.” Cetuximab is an antibody that attaches to the epidermal-growth-factor-receptor (EGFR) expressed on cancer cells and pembrolizumab is also an antibody that hones in on PD-L1 of the T-cell, as such they are less toxic in general. The oncologic rationale behind the multiple subgroups in this trial is to see if pembrolizumab monotherapy, the treatment with the least amount of side effects, is as good as the other options.
These efforts generally revolve around substituting cytotoxic chemotherapy for targeted agents, reduction of radiation dose, the use of more precise radiation techniques, or less invasive surgeries.
A hematologist can be thought of as an oncologist who specializes in blood-based cancers such as lymphoma and leukemia.
Traditional imaging such as a CT scan or MRI can see changes in shape or size, whereas positron emission tomography (PET) scans can look at the change of the metabolic rate of a tumor via a radiolabeled analogue, most commonly glucose (18-flourodeoxyglucose). This is especially important in lymphomas.
An intention to treat analysis means that the patients will be analyzed in the group they were originally randomized to regardless of if they receive the treatment on that arm. For instance, if a patient was randomized to the no radiotherapy arm, but was concerned about failure without radiation and ultimately received it, the patient would still be counted on the no radiotherapy arm.
This shows that the investigators were willing to trade off 10% progression to avoid a modest dose of radiation therapy. In addition, this brings up the main issue of non-inferiority trials, which is the controversy around the acceptable lower limits. A 10% absolute difference in progression is not acceptable, but since this trial was not even able to meet its 7% goal this is a moot point. The issue of acceptable bounds, however will be discussed again in Sect. 7.4.
NRG is a combination of three major cooperative groups the National Surgical Adjuvant Breast and Bowel Project (NSABP), Radiation Therapy Oncology Group (RTOG), and Gynecologic Oncology Group (GOG). HN-005 is to denote the trial is the fifth head and neck cancer trial.
SBRT is also referred to as stereotactic ablative radiotherapy (SABR) in the literature interchangeably. It is a very conformal and short course (5 treatments or less) of radiation.
I have not personally seen one DAG in a major publication in oncology.
Anything over 5% survival advantage would be considerate a great success in oncology, so an 15% absolute difference is rarely seen in the field.
Anything less than a randomized phase III study.
Microsatellite instability (MSI) refers to a mutation characterized by continuous repetition of 1–9 DNA (nucleotide) bases. This can be caused by a loss of function of the DNA mismatch repair system (MMR), which as the name suggests, fixes errors in the DNA (Baudrin et al., 2018). When deficient in MMR, the abbreviation (dMMR) is used.
Pembrolizumab can now be used for any solid tumors that are microsatellite instability–high (MSI-H) or mismatch repair deficient (dMMR). The four other studies are KEYNOTE-016, 164, 012, and 028.
RECIST are national standards used to determine response criteria such as “At least a 30% decrease in the sum of diameters of target lesions” (Eisenahuer et al., 2009). Duration of response refers to the length of time, for patients who have a response, before progression.
For oropharyngeal cancers, gliomas (primary brain tumors), and lung cancer.
Pathological findings are those found under traditional microscopy techniques. The new markers now require genetic and molecular analysis. IDH = isocitrate dehydrogenase; 1p19q = Chromosome 1 short arm and chromosome 19 long arm; ATRX = α thalassemia/mental retardation syndrome X-linked; MGMT = O6-methylguanine DNA methyltransferase.
Driver mutations are those that “drive” cancer growth. Some can also act as targets for various therapies. ALK = anaplastic lymphoma kinase; ROS1 = c-ros oncogene 1; BRAF = v-raf murine sarcoma viral oncogene homolog B1; KRAS = Kirsten rat sarcoma viral oncogene homolog; HER2 = human epidermal growth factor receptor 2; MET = mesenchymal-epithelial transition; RET = rearranged during transfection; NTRK = Neurotrophic tropomyosin receptor kinase.
Some examples of guidelines are as follows: Prediction model Risk Of Bias ASsessment Tool (PROBAST) (Moons et al. 2019), International Council for Harmonization (ICH) guidelines, Cochrane Collaboration Tool, Consolidated Standards of Reporting Trials (CONSORT) (Turner et al. 2012), Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) (Chan et al. 2013), Checklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) (Moons et al. 2014).
Steven Goodman was an author in both the 1998 and 2017 papers.
There are certainly those who do critique the EBM movement and the superiority of randomized trials, but this is certainly not the norm and does not reflect the praxis of clinical trials today. The reader is referred to the works of Nancy Cartwright and John Worrall for further discussion.
Philosophy of science, which includes issues in epistemology and philosophy of statistics.
On the statistical side, in October of 2017, the American Statistical Association held the “Symposium on Statistical Inference” where ideas such as the use of s-values (Shannon information, surprisal, or binary logworth), the second generation p-value, analysis of credibility, false positivity risk, or Bayes factor bounds, etc. were proposed to tackle these issues (Wasserstein et al., 2019).
The USMLE Step I takes place after the second year of medical school and is a test of basic science knowledge, as the clinical curriculum starts only in the third year. This test’s score is the most important part of a student’s residency application. Now that is it pass/fail, time and energy can now be focused spent in other important areas such as statistics and philosophy of science.
Recall only 7% of National Cancer Comprehensive Network (NCCN) guidelines are level 1 evidence and that observational studies often make bold claims that are unwarranted.
The American Society of Clinical Oncology (ASCO) is the largest and most preeminent cancer society and holds an annual meeting where the latest cancer discoveries are presented. Its journal, the “Journal of Clinical Oncology,” is also the most prestigious dedicated oncology journal.
References
Altman, D. G. (1994). The scandal of poor medical research. BMJ, 308(6924), 283–284. https://doi.org/10.1136/bmj.308.6924.283
Anderson, C. M., Kimple, R. J., Lin, A., Karam, S. D., Margalit, D. N., & Chua, M. L. K. (2019). De-escalation strategies in HPV-associated oropharynx cancer—Are we putting the cart before the horse? International Journal of Radiation Oncology, Biology, Physics, 104(4), 705–709. https://doi.org/10.1016/j.ijrobp.2019.02.054
Ang, K. K., Harris, J., Wheeler, R., Weber, R., Rosenthal, D. I., Nguyen-Tân, P. F., Westra, W. H., Chung, C. H., Jordan, R. C., Lu, C., Kim, H., & Axelrod, R. (2010). Human papillomavirus and survival of patients with oropharyngeal cancer. New England Journal of Medicine, 363(1), 24–35. https://doi.org/10.1056/NEJMoa0912217
Ashcroft, R. E. (2004). Current epistemological problems in evidence based medicine. Journal of Medical Ethics, 30(2), 131–135. https://doi.org/10.1136/jme.2003.007039
Banerjee, R., & Prasad, V. (2020). Are observational, real-world studies suitable to make cancer treatment recommendations? JAMA Network Open, 3(7), e2012119. https://doi.org/10.1001/jamanetworkopen.2020.12119
Baudrin, L. G., Deleuze, J. F., & How-Kit, A. (2018). Molecular and computational methods for the detection of microsatellite instability in cancer. Frontiers in Oncology., 8, 621. https://doi.org/10.3389/fonc.2018.00621
Berger, J. (2006). The case for objective Bayesian analysis. Bayesian Analysis, 1(3), 385–402. https://doi.org/10.1214/06-BA115
Blakely, T., Lynch, J., Simons, K., Bentley, R., & Rose, S. (2019). Reflection on modern methods: When worlds collide—Prediction, machine learning and causal inference. International Journal of Epidemiology. https://doi.org/10.1093/ije/dyz132
Burtness, B., Harrington, K. J., Greil, R., Soulières, D., Tahara, M., de Castro, G., Psyrri, A., Basté, N., Neupane, P., Bratland, Å., Fuereder, T., Hughes, B. G. M., Mesía, R., Ngamphaiboon, N., Rordorf, T., Wan Ishak, W. Z., Hong, R. L., González Mendoza, R., Roy, A., … Rischin, D. (2019). Pembrolizumab alone or with chemotherapy versus cetuximab with chemotherapy for recurrent or metastatic squamous cell carcinoma of the head and neck (KEYNOTE-048): A randomised, open-label, phase 3 study. The Lancet, 394(10212), 1915–1928. https://doi.org/10.1016/S0140-6736(19)32591-7
Burtness, B., Zhang, Y., Harrington, K. J., & Rischin, D. (2020). Further clinical interpretation and implications of KEYNOTE-048 findings—Authors’ reply. The Lancet, 396(10248), 379–380. https://doi.org/10.1016/S0140-6736(20)30900-4
Cavallo, J. (2017). FDA’s first site-agnostic drug approval marks a paradigm shift in regulatory criteria—The ASCO post. Retrieved November 23, 2020, from https://ascopost.com/issues/august-10-2017/fda-s-first-site-agnostic-drug-approval-marks-a-paradigm-shift-in-regulatory-criteria/
Chan, A. W., Tetzlaff, J. M., Altman, D. G., Laupacis, A., Gøtzsche, P. C., Krleža-Jerić, K., Hróbjartsson, A., Mann, H., Dickersin, K., Berlin, J. A., Doré, C. J., Parulekar, W. R., Summerskill, W. S., Groves, T., Schulz, K. F., Sox, H. C., Rockhold, F. W., Rennie, D., & Moher, D. (2013). SPIRIT 2013 statement: Defining standard protocol items for clinical trials. Annals of Internal Medicine, 158(3), 200–207. https://doi.org/10.7326/0003-4819-158-3-201302050-00583
Chang, J. Y., Senan, S., Paul, M. A., Mehran, R. J., Louie, A. V., Balter, P., Groen, H. J. M., McRae, S. E., Widder, J., Feng, L., van den Borne, B. E. E. M., Munsell, M. F., Hurkmans, C., Berry, D. A., van Werkhoven, E., Kresl, J. J., Dingemans, A.-M., Dawood, O., Haasbeek, C. J. A., … Roth, J. A. (2015). Stereotactic ablative radiotherapy versus lobectomy for operable stage I non-small-cell lung cancer: A pooled analysis of two randomised trials. The Lancet Oncology, 16(6), 630–637. https://doi.org/10.1016/S1470-2045(15)70168-3
Chen, H., Laba, J. M., Boldt, R. G., Goodman, C. D., Palma, D. A., Senan, S., & Louie, A. V. (2018). Stereotactic ablative radiation therapy versus surgery in early lung cancer: A meta-analysis of propensity score studies. International Journal of Radiation Oncology, Biology, Physics, 101(1), 186–194. https://doi.org/10.1016/j.ijrobp.2018.01.064
Cohen, E. E. W., Soulières, D., Le Tourneau, C., Dinis, J., Licitra, L., Ahn, M. J., Soria, A., Machiels, J. P., Mach, N., Mehra, R., Burtness, B., Zhang, P., Cheng, J., Swaby, R. F., Harrington, K. J., Acosta-Rivera, M., Adkins, D. R., Aghmesheh, M., Ahn, M. J., … Zandberg, D. (2019). Pembrolizumab versus methotrexate, docetaxel, or cetuximab for recurrent or metastatic head-and-neck squamous cell carcinoma (KEYNOTE-040): A randomised, open-label, phase 3 study. The Lancet, 393(10167), 156–167. https://doi.org/10.1016/S0140-6736(18)31999-8
Desai, A., Go, R. S., & Poonacha, T. (2019). Level of scientific evidence supporting NCCN guidelines: Is there evidence of progress? Journal of Clinical Oncology, 37(27_suppl), 14–14. https://doi.org/10.1200/JCO.2019.37.27_suppl.14
Diamandis, P., & Aldape, K. D. (2017). Insights from molecular profiling of adult glioma. Journal of Clinical Oncology, 35(21), 2386–2393. https://doi.org/10.1200/JCO.2017.73.9516
Eisenhauer, E. A., Therasse, P., Bogaerts, J., Schwartz, L. H., Sargent, D., Ford, R., Dancey, J., Arbuck, S., Gwyther, S., Mooney, M., Rubinstein, L., Shankar, L., Dodd, L., Kaplan, R., Lacombe, D., & Verweij, J. (2009). New response evaluation criteria in solid tumours: Revised RECIST guideline (version 1.1). European Journal of Cancer, 45(2), 228–247. https://doi.org/10.1016/j.ejca.2008.10.026
ESMO. (2017). ESMO 2017 press release: KEYNOTE-040 evaluates pembrolizumab in head and neck cancer. Retrieved November 22, 2020, from https://www.esmo.org/newsroom/press-office/KEYNOTE-040-Evaluates-Pembrolizumab-in-Head-and-Neck-Cancer
European Agency for the Evaluation of Medicinal Products. (2002). Points to on multiplicity issues in clinical trials. Retrieved February 21, 2021, from https://www.ema.europa.eu/en/documents/scientific-guideline/points-consider-multiplicity-issues-clinical-trials_en.pdf
Food and Drug Administration. (2017). Multiple endpoints in clinical trials guidance for industry. Retrieved February 26, 2021, from https://www.fda.gov/media/102657/download.
Food and Drug Administration. (2020). FDA approves pembrolizumab for adults and children with TMB-H solid tumors. FDA. Retrieved November 23, 2020, from https://www.fda.gov/drugs/drug-approvals-and-databases/fda-approves-pembrolizumab-adults-and-children-tmb-h-solid-tumors
Gaissmaier, W., & Gigerenzer, G. (2020). When misinformed patients try to make informed health decision. In G. Gigerenzer & J. A. Muirgray (Eds.), Better doctors, better patients, better decisions: Envisioning healthcare 2020 (pp. 29–42). MIT Press.
Gamble, C., Krishan, A., Stocken, D., Lewis, S., Juszczak, E., Doré, C., Williamson, P. R., Altman, D. G., Montgomery, A., Lim, P., Berlin, J., Senn, S., Day, S., Barbachano, Y., & Loder, E. (2017). Guidelines for the content of statistical analysis plans in clinical trials. JAMA, 318(23), 2337–2343. https://doi.org/10.1001/jama.2017.18556
Hamdy, F. C., Donovan, J. L., Lane, J. A., Mason, M., Metcalfe, C., Holding, P., Davis, M., Peters, T. J., Turner, E. L., Martin, R. M., Oxley, J., Robinson, M., Staffurth, J., Walsh, E., Bollina, P., Catto, J., Doble, A., Doherty, A., Gillatt, D., … Neal, D. E. (2016). 10-Year outcomes after monitoring, surgery, or radiotherapy for localized prostate cancer. New England Journal of Medicine, 375(15), 1415–1424. https://doi.org/10.1056/NEJMoa1606220
Hardwicke, T. E., & Goodman, S. N. (2020). How often do leading biomedical journals use statistical experts to evaluate statistical methods? The results of a survey. PLoS ONE, 15(10), e0239598. https://doi.org/10.1371/journal.pone.0239598
Harrell, F. E. (2020). Bayesian sequential designs for COVID-19 clinical trials. Retrieved November 22, 2020, from https://www.youtube.com/watch?v=V-XTT8HPEpw&feature=youtu.be&t=5325
Harrell, F. E., & Slaughter, J. C. (2021). Biostatistics for biomedical research (Feburary 6. 2021 update). Retrieved February 21, 2021, from https://hbiostat.org/doc/bbr.pdf
Harrington, D., D’Agostino, R. B., Gatsonis, C., Hogan, J. W., Hunter, D. J., Normand, S. L. T., Drazen, J. M., & Hamel, M. B. (2019). New guidelines for statistical reporting in the journal. The New England Journal of Medicine, 381(3), 285–286. https://doi.org/10.1056/NEJMe1906559
ICH Official website: ICH. Retrieved June 22, 2021, from https://www.ich.org/
Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Medicine, 2(8), 6. https://doi.org/10.1371/journal.pmed.0020124
Johnson, A. L., Fladie, I., Anderson, J. M., Lewis, D. M., Mons, B. R., & Vassar, M. (2020). Rates of discontinuation and nonpublication of head and neck cancer randomized clinical trials. JAMA Otolaryngology-Head & Neck Surgery, 146(2), 176. https://doi.org/10.1001/jamaoto.2019.3967
Jones, D. R. (2015). Do we know bad science when we see it? The Journal of Thoracic and Cardiovascular Surgery, 150(3), 472–473. https://doi.org/10.1016/j.jtcvs.2015.07.032
Journal of the American Medical Association. (1924). The spirit of apprenticeship in medicine. JAMA, 83(17), 1337–1338. https://doi.org/10.1001/jama.1924.02660170053021
Khushf, G. (2013). A framework for understanding medical epistemologies. Journal of Medicine and Philosophy, 38(5), 461–486. https://doi.org/10.1093/jmp/jht044
King, G., & Nielsen, R. (2019). Why propensity scores should not be used for matching. Political Analysis, 27(4), 435–454. https://doi.org/10.1017/pan.2019.11
Kumar, A., Guss, Z. D., Courtney, P. T., Nalawade, V., Sheridan, P., Sarkar, R. R., Banegas, M. P., Rose, B. S., Xu, R., & Murphy, J. D. (2020). Evaluation of the use of cancer registry data for comparative effectiveness research. JAMA Network Open, 3(7), e2011985. https://doi.org/10.1001/jamanetworkopen.2020.11985
Li, G., Taljaard, M., Van den Heuvel, E. R., Levine, M. A., Cook, D. J., Wells, G. A., Devereaux, P. J., & Thabane, L. (2017). An introduction to multiplicity issues in clinical trials: The what, why, when and how. International Journal of Epidemiology, 46(2), 746–755. https://doi.org/10.1093/ije/dyw320
Marabelle, A., Fakih, M., Lopez, J., Shah, M., Shapira-Frommer, R., Nakagawa, K., Chung, H. C., Kindler, H. L., Lopez-Martin, J. A., Miller, W. H., Italiano, A., Kao, S., Piha-Paul, S. A., Delord, J. P., McWilliams, R. R., Fabrizio, D. A., Aurora-Garg, D., Xu, L., Jin, F., … Bang, Y. J. (2020a). Association of tumour mutational burden with outcomes in patients with advanced solid tumours treated with pembrolizumab: Prospective biomarker analysis of the multicohort, open-label, phase 2 KEYNOTE-158 study. The Lancet. Oncology, 21(10), 1353–1365. https://doi.org/10.1016/S1470-2045(20)30445-9
Marabelle, A., Le, D. T., Ascierto, P. A., Di Giacomo, A. M., De Jesus-Acosta, A., Delord, J.-P., Geva, R., Gottfried, M., Penel, N., Hansen, A. R., Piha-Paul, S. A., Doi, T., Gao, B., Chung, H. C., Lopez-Martin, J., Bang, Y. J., Frommer, R. S., Shah, M., Ghori, R., … Diaz, L. A. (2020b). Efficacy of Pembrolizumab in patients with Noncolorectal high microsatellite instability/mismatch repair-deficient cancer: Results from the phase II KEYNOTE-158 study. Journal of Clinical Oncology: Official Journal of the American Society of Clinical Oncology, 38(1), 1–10. https://doi.org/10.1200/JCO.19.02105
Mayo, D. G. (1996). Error and the growth of experimental knowledge. The University of Chicago Press.
Mayo, D. G. (2018). Statistical inference as severe testing: How to get beyond the statistics wars. Cambridge University Press.
Mayo, D. G. (2019). P-value thresholds: Forfeit at your peril. European Journal of Clinical Investigation, 49(10), e13170. https://doi.org/10.1111/eci.13170
Mayo, D. G. (2020). Data & Science with Glen Wright Colopy. (2020). Philosophy of Science & Statistics | Deborah Mayo | Keynote Talk. Retrieved June 22, 2021, from https://www.youtube.com/watch?v=9GGAXZ6htrA
McNeil, B. J., Pauker, S. G., Sox, H. C., & Tversky, A. (1982). On the elicitation of preferences for alternative therapies. New England Journal of Medicine, 306(21), 1259–1262. https://doi.org/10.1056/NEJM198205273062103
Meyer, R. M., & Hoppe, R. T. (2012). Point/counterpoint: Early-stage Hodgkin lymphoma and the role of radiation therapy. Blood, 120(23), 4488–4495. https://doi.org/10.1182/blood-2012-05-423236
Miccio, J. A., Talcott, W. J., Jairam, V., Park, H. S., Yu, J. B., Leapman, M. S., Johnson, S. B., King, M. T., Nguyen, P. L., & Kann, B. H. (2020). Quantifying treatment selection bias effect on survival in comparative effectiveness research: Findings from low-risk prostate cancer patients. Prostate Cancer and Prostatic Diseases. https://doi.org/10.1038/s41391-020-00291-3
Moons, K. G. M., de Groot, J. A. H., Bouwmeester, W., Vergouwe, Y., Mallett, S., Altman, D. G., Reitsma, J. B., & Collins, G. S. (2014). Critical appraisal and data extraction for systematic reviews of prediction modelling studies: The CHARMS checklist. PLoS Medicine, 11(10), e1001744. https://doi.org/10.1371/journal.pmed.1001744
Moons, K. G. M., Wolff, R. F., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., & Mallett, S. (2019). PROBAST: A tool to assess risk of bias and applicability of prediction model studies: Explanation and elaboration. Annals of Internal Medicine, 170(1), W1–W33. https://doi.org/10.7326/M18-1377
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E. J., Ware, J. J., & Ioannidis, J. P. A. (2017). A manifesto for reproducible science. Nature Human Behaviour, 1(1), 0021. https://doi.org/10.1038/s41562-016-0021
Poonacha, T. K., & Go, R. S. (2011). Level of scientific evidence underlying recommendations arising from the national comprehensive cancer network clinical practice guidelines. Journal of Clinical Oncology, 29(2), 186–191. https://doi.org/10.1200/JCO.2010.31.6414
Popper, K. R. (1962). Conjectures and refutations: The growth of scientific knowledge. Basic Books.
Prasad, V. (2013). Persistent reservations against the premedical and medical curriculum. Perspectives on Medical Education, 2(5–6), 335–339. https://doi.org/10.1007/s40037-013-0047-2
Prasad, V. (2020). Malignant. Johns Hopkins University Press Books.
Prasad, V., & Cifu, A. (2011). Medical reversal: Why we must raise the bar before adopting new technologies. The Yale Journal of Biology and Medicine, 84(4), 471–478.
Radford, J., Illidge, T., Counsell, N., Hancock, B., Pettengell, R., Johnson, P., Wimperis, J., Culligan, D., Popova, B., Smith, P., McMillan, A., Brownell, A., Kruger, A., Lister, A., Hoskin, P., O’Doherty, M., & Barrington, S. (2015). Results of a trial of PET-directed therapy for early-stage Hodgkin’s Lymphoma. New England Journal of Medicine, 372(17), 1598–1607. https://doi.org/10.1056/NEJMoa1408648
Razi, S. S., Kodia, K., Alnajar, A., Block, M. I., Tarrazzi, F., Nguyen, D., & Villamizar, N. (2020). Lobectomy versus stereotactic body radiotherapy in healthy octogenarians with stage I lung cancer. The Annals of Thoracic Surgery. https://doi.org/10.1016/j.athoracsur.2020.06.097
Rovin, L. (2017). 22 case studies where phase 2 and Phase 3 Trials had divergent results, 44. Retrieved November 23, 2020, from https://www.fda.gov/about-fda/reports/22-case-studies-where-phase-2-and-phase-3-trials-had-divergent-results
Smaldino, P. E., & McElreath, R. (2016). The natural selection of bad science. Royal Society Open Science, 3(9), 160384. https://doi.org/10.1098/rsos.160384
Soni, P. D., Hartman, H. E., Dess, R. T., Abugharib, A., Allen, S. G., Feng, F. Y., Zietman, A. L., Jagsi, R., Schipper, M. J., & Spratt, D. E. (2019). Comparison of population-based observational studies with randomized trials in oncology. Journal of Clinical Oncology, 37(14), 1209–1216. https://doi.org/10.1200/JCO.18.01074
Spanos, A. (2019). Probability theory and statistical inference: empirical modeling with observational data (2nd ed.). Cambridge University Press.
Thoma, A., & Eaves, F. F. (2015). A brief history of evidence-based medicine (EBM) and the contributions of Dr David Sackett. Aesthetic Surgery Journal, 35(8), 261–263.
Turner, L., Shamseer, L., Altman, D. G., Weeks, L., Peters, J., Kober, T., Dias, S., Schulz, K. F., Plint, A. C., & Moher, D. (2012). Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals. The Cochrane Database of Systematic Reviews. https://doi.org/10.1002/14651858.MR000030.pub2
Visvanathan, K., Levit, L. A., Raghavan, D., Hudis, C. A., Wong, S., Dueck, A., & Lyman, G. H. (2017). Untapped potential of observational research to inform clinical decision making: American Society of Clinical Oncology research statement. Journal of Clinical Oncology: Official Journal of the American Society of Clinical Oncology, 35(16), 1845–1854. https://doi.org/10.1200/JCO.2017.72.6414
Wagenmakers, E. J., Wetzels, R., Borsboom, D., van der Maas, H. L. J., & Kievit, R. A. (2012). An agenda for purely confirmatory research. Perspectives on Psychological Science: A Journal of the Association for Psychological Science, 7(6), 632–638. https://doi.org/10.1177/1745691612463078
Wasserstein, R. L., Schirm, A. L., & Lazar, N. A. (2019). Moving to a world beyond “p < 0.05.” The American Statistician, 73(sup1), 1–19. https://doi.org/10.1080/00031305.2019.1583913
Weller, M., van den Bent, M., Tonn, J. C., Stupp, R., Preusser, M., Cohen-Jonathan-Moyal, E., Henriksson, R., Rhun, E. L., Balana, C., Chinot, O., Bendszus, M., Reijneveld, J. C., Dhermain, F., French, P., Marosi, C., Watts, C., Oberg, I., Pilkington, G., Baumert, B. G., … Wick, W. (2017). European Association for Neuro-Oncology (EANO) guideline on the diagnosis and treatment of adult astrocytic and oligodendroglial gliomas. The Lancet Oncology, 18(6), e315–e329. https://doi.org/10.1016/S1470-2045(17)30194-8
Wilcken, H. (2017). Prostate cancer: Urologists fight back. InSight+. Retrieved November, 20, 2020 from https://insightplus.mja.com.au/2017/29/prostate-cancer-urologists-fight-back/
Wilt, T. J., Brawer, M. K., Jones, K. M., Barry, M. J., Aronson, W. J., Fox, S., Gingrich, J. R., Wei, J. T., Gilhooly, P., Grob, B. M., Nsouli, I., Iyer, P., Cartagena, R., Snider, G., Roehrborn, C., Sharifi, R., Blank, W., Pandya, P., Andriole, G. L., … Wheeler, T. (2012). Radical prostatectomy versus observation for localized prostate cancer. New England Journal of Medicine, 367(3), 203–213. https://doi.org/10.1056/NEJMoa1113162
Windish, D., Huot, S., & Green, M. (2007). Medicine residents’ understanding of the biostatistics and results in the medical literature. Journal of the American Medical Association, 298(9), 1010–1022. https://doi.org/10.1001/jama.298.9.1010
Yom, S. (2019). NRG-HN 005A randomized phase II/III trial of de-intensified radiation therapy for patients with early-stage, p16-positive, non-smoking associated oropharyngeal cancer protocol. Retrieved November 22, 2020, from https://www.nrgoncology.org/Clinical-Trials/Protocol/nrg-hn005?filter=nrg-hn005
Zia, M. I., Siu, L. L., Pond, G. R., & Chen, E. X. (2005). Comparison of outcomes of phase II studies and subsequent randomized control studies using identical chemotherapeutic regimens. Journal of Clinical Oncology, 23(28), 6982–6991. https://doi.org/10.1200/JCO.2005.06.679
Zimerman, A. L. (2013). Evidence-based medicine: A short history of a modern medical movement. AMA Journal of Ethics, 15(1), 71–76. https://doi.org/10.1001/virtualmentor.2013.15.1.mhst1-1301
Funding
No funding was used for this study.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The author has no conflicts of interest to declare.
Ethical approval
Not applicable.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Park, J.H. Current issues in medical epistemology and statistics: a view from the frontline of medicine. Synthese 200, 417 (2022). https://doi.org/10.1007/s11229-022-03765-0
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11229-022-03765-0