Sweeping in scope, penetrating in analysis, and generously illustrated with examples from the history of science, this new and original approach to familiar questions about scientific evidence and method tackles vital questions about science and its place in society. Avoiding the twin pitfalls of scientism and cynicism, noted philosopher Susan Haack argues that, fallible and flawed as they are, the natural sciences have been among the most successful of human enterprises-valuable not only for the vast, interlocking body of knowledge they (...) have discovered, and not only for the technological advances that have improved our lives, but as a manifestation of the human talent for inquiry at its imperfect but sometimes remarkable best. This wide-ranging, trenchant, and illuminating book explores the complexities of scientific evidence, and the multifarious ways in which the sciences have refined and amplified the methods of everyday empirical inquiry; articulates the ways in which the social sciences are like the natural sciences, and the ways in which they are different; disentangles the confusions of radical rhetoricians and cynical sociologists of science; exposes the evasions of apologists for religious resistance to scientific advances; weighs the benefits and the dangers of technology; tracks the efforts of the legal system to make the best use of scientific testimony; and tackles predictions of the eventual culmination, or annihilation, of the scientific enterprise. Writing with verve and wry humor, in a witty, direct, and accessible style, Haack takes readers beyond the "Science Wars" to a balanced understanding of the value, and the limitations, of the scientific enterprise. (shrink)

Most theories and hypotheses in psychology are verbal in nature, yet their evaluation overwhelmingly relies on inferential statistical procedures. The validity of the move from qualitative to quantitative analysis depends on the verbal and statistical expressions of a hypothesis being closely aligned – that is, that the two must refer to roughly the same set of hypothetical observations. Here, I argue that many applications of statistical inference in psychology fail to meet this basic condition. Focusing on the most widely used (...) class of model in psychology – the linear mixed model – I explore the consequences of failing to statistically operationalize verbal hypotheses in a way that respects researchers' actual generalization intentions. I demonstrate that although the “random effect” formalism is used pervasively in psychology to model intersubject variability, few researchers accord the same treatment to other variables they clearly intend to generalize over. The under-specification of random effects imposes far stronger constraints on the generalizability of results than most researchers appreciate. Ignoring these constraints can dramatically inflate false-positive rates, and often leads researchers to draw sweeping verbal generalizations that lack a meaningful connection to the statistical quantities they are putatively based on. I argue that failure to take the alignment between verbal and statistical expressions seriously lies at the heart of many of psychology's ongoing problems, and conclude with a discussion of several potential avenues for improvement. (shrink)

Science's priority rule rewards those who are first to make a discovery, at the expense of all other scientists working towards the same goal, no matter how close they may be to making the same discovery. I propose an explanation of the priority rule that, better than previous explanations, accounts for the distinctive features of the rule. My explanation treats the priority system, and more generally, any scheme of rewards for scientific endeavor, as a device for achieving an allocation of (...) resources among different research programs that provides as much benefit as possible to society. I show that the priority system is especially well suited to finding an efficient allocation of resources in those situations, characteristic of scientific inquiry, in which any success in an endeavor subsequent to the first success brings little additional benefit to society. (shrink)

A science war is in full swing which has taken science's reproducibility crisis as a battleground. While conservatives and corporate interests use the crisis to weaken regulations, their opponent deny the existence of a science's crisis altogether. Thus, for the conservative National Association of Scholars NAS the crisis is real and due to the progressive assault on higher education with ideologies such as neo-Marxism, radical feminism, historicism, post-colonialism, deconstructionism, post-modernism, liberation theology. In the opposite field, some commentators claim that there (...) is no crisis in science and that saying the opposite is irresponsible. These positions are to be seen in the context of the ongoing battle against regulation, of which the new rules proposed at the US Environmental Protection Agency are but the last chapter. In this optic, Naomi Oreskes writes on Nature that what constitutes the crisis is the conservatives' attack on science. This evident right-left divide in the reading of the crisis is unhelpful and dangerous to the survival of science itself. An alternative reading ignored by the contendents would suggest that structural contradictions have emerged in modern science, and that addressing these should be the focus of our attention. (shrink)

Advocates of the self-corrective thesis argue that scientific method will refute false theories and find closer approximations to the truth in the long run. I discuss a contemporary interpretation of this thesis in terms of frequentist statistics in the context of the behavioral sciences. First, I identify experimental replications and systematic aggregation of evidence (meta-analysis) as the self-corrective mechanism. Then, I present a computer simulation study of scientific communities that implement this mechanism to argue that frequentist statistics may converge upon (...) a correct estimate or not depending on the social structure of the community that uses it. Based on this study, I argue that methodological explanations of the “replicability crisis” in psychology are limited and propose an alternative explanation in terms of biases. Finally, I conclude suggesting that scientific self-correction should be understood as an interaction effect between inference methods and social structures. (shrink)

A growing body of literature has identified potential problems that can compromise the quality, fairness, and integrity of journal peer review, including inadequate review, inconsistent reviewer reports, reviewer biases, and ethical transgressions by reviewers. We examine the evidence concerning these problems and discuss proposed reforms, including double-blind and open review. Regardless of the outcome of additional research or attempts at reforming the system, it is clear that editors are the linchpin of peer review, since they make decisions that have a (...) significant impact on the process and its outcome. We consider some of the steps editors should take to promote quality, fairness and integrity in different stages of the peer review process and make some recommendations for editorial conduct and decision-making. (shrink)

Striving to boldly redirect the philosophy of science, this book by renowned philosopher Philip Kitcher examines the heated debate surrounding the role of science in shaping our lives. Kitcher explores the sharp divide between those who believe that the pursuit of scientific knowledge is always valuable and necessary--the purists--and those who believe that it invariably serves the interests of people in positions of power. In a daring turn, he rejects both perspectives, working out a more realistic image of the sciences--one (...) that allows for the possibility of scientific truth, but nonetheless permits social consensus to determine which avenues to investigate. He then proposes a democratic and deliberative framework for responsible scientists to follow. Controversial, powerful, yet engaging, this volume will appeal to a wide range of readers. Kitcher's nuanced analysis and authorititative conclusion will interest countless scientists as well as all readers of science--scholars and laypersons alike. (shrink)

Published research findings are sometimes refuted by subsequent evidence, says Ioannidis, with ensuing confusion and disappointment.

Recent philosophical work has praised the reward structure of science, while recent empirical work has shown that many scientific results may not be reproducible. I argue that the reward structure of science incentivizes scientists to focus on speed and impact at the expense of the reproducibility of their work, thus contributing to the so-called reproducibility crisis. I use a rational choice model to identify a set of sufficient conditions for this problem to arise, and I argue that these conditions plausibly (...) apply to a wide range of research situations. Currently proposed solutions will not fully address this problem. Philosophical commentators should temper their optimism about the reward structure of science. (shrink)

Objectivity has a history, and it is full of surprises. In Objectivity, Lorraine Daston and Peter Galison chart the emergence of objectivity in the mid-nineteenth-century sciences--and show how the concept differs from its alternatives, truth-to-nature and trained judgment. This is a story of lofty epistemic ideals fused with workaday practices in the making of scientific images. From the eighteenth through the early twenty-first centuries, the images that reveal the deepest commitments of the empirical sciences--from anatomy to crystallography--are those featured in (...) scientific atlases, the compendia that teach practitioners what is worth looking at and how to look at it. Galison and Daston use atlas images to uncover a hidden history of scientific objectivity and its rivals. Whether an atlas maker idealizes an image to capture the essentials in the name of truth-to-nature or refuses to erase even the most incidental detail in the name of objectivity or highlights patterns in the name of trained judgment is a decision enforced by an ethos as well as by an epistemology. As Daston and Galison argue, atlases shape the subjects as well as the objects of science. To pursue objectivity--or truth-to-nature or trained judgment--is simultaneously to cultivate a distinctive scientific self wherein knowing and knower converge. Moreover, the very point at which they visibly converge is in the very act of seeing not as a separate individual but as a member of a particular scientific community. Embedded in the atlas image, therefore, are the traces of consequential choices about knowledge, persona, and collective sight. Objectivity is a book addressed to anyone interested in the elusive and crucial notion of objectivity-- and in what it means to peer into the world scientifically. Lorraine Daston is Director at the Max Planck Institute for the History of Science in Berlin, Germany. She is the coauthor of Wonders and the Order of Nature, 1150-1750 and the editor of Things That Talk: Object Lessons from Art and Science. Peter Galison is Pellegrino University Professor of the History of Science and of Physics at Harvard University. He is the author of Einstein's Clocks, Poincaré's Maps: Empires of Time, How Experiments End, and Image and Logic: A Material Culture of Microphysics, and other books, and coeditor of The Architecture of Science. (shrink)

The Empire of Chance tells how quantitative ideas of chance transformed the natural and social sciences, as well as daily life over the last three centuries. A continuous narrative connects the earliest application of probability and statistics in gambling and insurance to the most recent forays into law, medicine, polling and baseball. Separate chapters explore the theoretical and methodological impact in biology, physics and psychology. Themes recur - determinism, inference, causality, free will, evidence, the shifting meaning of probability - but (...) in dramatically different disciplinary and historical contexts. In contrast to the literature on the mathematical development of probability and statistics, this book centres on how these technical innovations remade our conceptions of nature, mind and society. Written by an interdisciplinary team of historians and philosophers, this readable, lucid account keeps technical material to an absolute minimum. It is aimed not only at specialists in the history and philosophy of science, but also at the general reader and scholars in other disciplines. (shrink)

Scientists and 'anti-scientists' alike need a more realistic image of science. The traditional mode of research, academic science, is not just a 'method': it is a distinctive culture, whose members win esteem and employment by making public their findings. Fierce competition for credibility is strictly regulated by established practices such as peer review. Highly specialized international communities of independent experts form spontaneously and generate the type of knowledge we call 'scientific' - systematic, theoretical, empirically-tested, quantitative, and so on. Ziman shows (...) that these familiar 'philosophical' features of scientific knowledge are inseparable from the ordinary cognitive capabilities and peculiar social relationships of its producers. This wide-angled close-up of the natural and human sciences recognizes their unique value, whilst revealing the limits of their rationality, reliability, and universal applicability. It also shows how, for better or worse, the new 'post-academic' research culture of teamwork, accountability, etc. is changing these supposedly eternal philosophical characteristics. (shrink)

Claims that science should be more democratic than it is frequently arouse opposition. In this essay, I distinguish my own views about the democratization of science from the more ambitious theses defended by Paul Feyerabend. I argue that it is unlikely that the complexity of some scientific debates will allow for resolution according to the methodological principles of any formal confirmation theory, suggesting instead that major revolutions rest on conflicts of values. Yet these conflicts should not be dismissed as irresoluble.

Current fears of a “reproducibility crisis” have led researchers, sources of scientific funding, and the public to question both the efficacy and trustworthiness of science. Suggested policy changes have been focused on statistical problems, such as p-hacking, and issues of experimental design and execution. However, “reproducibility” is a broad concept that includes a number of issues. Furthermore, reproducibility failures occur even in fields such as mathematics or computer science that do not have statistical problems or issues with experimental design. Most (...) importantly, these proposed policy changes ignore a core feature of the process of scientific inquiry that occurs after reproducibility failures: the integration of conflicting observations and ideas into a coherent theory. (shrink)

Research on bias in peer review examines scholarly communication and funding processes to assess the epistemic and social legitimacy of the mechanisms by which knowledge communities vet and self-regulate their work. Despite vocal concerns, a closer look at the empirical and methodological limitations of research on bias raises questions about the existence and extent of many hypothesized forms of bias. In addition, the notion of bias is predicated on an implicit ideal that, once articulated, raises questions about the normative implications (...) of research on bias in peer review. This review provides a brief description of the function, history, and scope of peer review; articulates and critiques the conception of bias unifying research on bias in peer review; characterizes and examines the empirical, methodological, and normative claims of bias in peer review research; and assesses possible alternatives to the status quo. We close by identifying ways to expand conceptions and studies of bias to countenance the complexity of social interactions among actors involved directly and indirectly in peer review. (shrink)

A heated debate surrounds the significance of reproducibility as an indicator for research quality and reliability, with many commentators linking a "crisis of reproducibility" to the rise of fraudulent, careless and unreliable practices of knowledge production. Through the analysis of discourse and practices across research fields, I point out that reproducibility is not only interpreted in different ways, but also serves a variety of epistemic functions depending on the research at hand. Given such variation, I argue that the uncritical pursuit (...) of reproducibility as an overarching epistemic value is misleading and potentially damaging to scientific advancement. Requirements for reproducibility, however they are interpreted, are one of many available means to secure reliable research outcomes. Furthermore, there are cases where the focus on enhancing reproducibility turns out not to foster high-quality research. Scientific communities and Open Science advocates should learn from inferential reasoning from irreproducible data, and promote incentives for all researchers to explicitly and publicly discuss their methodological commitments, the ways in which they learn from mistakes and problems in everyday practice, and the strategies they use to choose which research component of any project needs to be preserved in the long term, and how. (shrink)

. Scientists, for the most part, want to get it right. However, the social structures that govern their work undermine that aim, and this leads to nonreplicable findings in many fields. Because the social structure of science is a decentralized system, it is difficult to intervene. In this article, I discuss how we might do so, focusing on self-corrective-labor schemes. First, I argue that we need to implement a scheme that makes replication work outcome independent, systematic, and sustainable. Second, I (...) use these three criteria to evaluate extant proposals, which place the responsibility for replication on original researchers, consumers of their research, students, or many labs. Third, on the basis of a philosophical analysis of the reward system of science and the benefits of the division of cognitive labor, I propose a scheme that satisfies the criteria better: the professional scheme. This scheme has two main components. First, the scientific community is organized into two groups: discovery researchers, who produce new findings, and confirmation researchers, whose primary function is to do confirmation work. Second, a distinct reward system is established for confirmation researchers so that their career advancement is separated from whether they obtain positive experimental results. (shrink)

Claims that science should be more democratic than it is frequently arouse opposition. In this essay, I distinguish my own views about the democratization of science from the more ambitious theses defended by Paul Feyerabend. I argue that it is unlikely that the complexity of some scientific debates will allow for resolution according to the methodological principles of any formal confirmation theory, suggesting instead that major revolutions rest on conflicts of values. Yet these conflicts should not be dismissed as irresoluble.