This paper argues for more randomised controlled trials in educational research. Educational researchers have largely abandoned the methodology they helped to pioneer. This gold-standard methodology should be more widely used as it is an appropriate and robust research technique. Without subjecting curriculum innovations to a RCT then potentially harmful educational initiatives could be visited upon the nation's children.

Randomized controlled trials in educational research tend to be small. Small trials can have large, chance, imbalances in important covariates. For studies with sample sizes greater than 50, chance imbalances can be corrected using analysis of covariance; for small trials, however, statistical power is maximized if the trial is balanced and analysis of covariance is used in the analysis. The aim of the present study was to discuss methods of improving covariate balance in trial design and to demonstrate the method (...) of minimization. Using an exemplar of a cluster or class‐randomized trial with 29 classes, we employed minimization to achieve covariate balance. Minimization achieved good balance on four prognostic variables. Many trialists in education use restricted forms of allocation, including pairing or stratified randomization. These approaches have disadvantages. Another approach rarely used in educational research is minimization. Minimization uses a simple arithmetic algorithm to produce balanced groups across a number of important covariates and should be more widely used in educational and psychological research. (shrink)

In response to concern over the numeracy skills deficit displayed by student nurses, an online computer programme, ?Authentic World??, which aims to simulate a real-life clinical environment and improve the medication dosage calculation skills of users, was developed (Founded in 2004 Authentic World Ltd is a spin out company of Glarmorgan and Cardiff Universities, Cardiff, Wales UK.). Two randomised controlled trials were conducted, each at a UK University, in order to investigate the impact of Authentic World? on student nurses? general (...) numeracy abilities. All first year nursing students who gave consent were randomised equally into an intervention or control group. The intervention group were given access to Authentic World?. The primary outcome measure was the students? scores on a general numeracy test. The Intention to Treat (ITT) analysis in both trials revealed a small negative effect of Authentic World? on general numeracy, which was statistically significant in one trial. However, compliance with the intervention was very low in both trials, with only 24 and 12% of students allocated to the intervention groups spending more than 15 minutes using the programme. Providing nursing students with access to Authentic World? is not an effective use of resources since use of the programme appears to be very low. (shrink)

The aim of unpaid volunteer classroom assistants is to give extra support to children learning to read. The impact of using volunteers to improve children's acquisition of reading skills is unknown. To assess whether volunteers are effective in improving children's reading, we undertook a systematic review of all relevant randomised controlled trials (RCTs). An exhaustive search of all the main electronic databases was carried out (i.e. BEI, PsycInfo, ASSIA, PAIS, SSCI, ERIC, SPECTR, SIGLE). We identified eight experimental studies, of which (...) seven were RCTs. One of the RCTs was excluded because it did not meet the inclusion criteria. One RCT randomised intact classes and the other six studies randomised individual children and could therefore be included in a meta-analysis. All of the trials were fairly small, with the largest including 99 pupils. Four of the trials showed a positive outcome, while three showed a negative effect and the remaining study was equivocal. We pooled the four most homogeneous trials. The pooled data indicated an effect size of 0.19, which was not statistically significant ( p = 0.54, 95% confidence interval = -0.31 to 0.68). Overall, volunteering appeared to have a small effect on reading outcomes. However, the confidence intervals were wide, which could conceal a potentially large benefit or a harmful effect. Thus, more good quality RCTs are required in order to provide more conclusive evidence. (shrink)

The term 'publication bias' usually refers to the tendency for a greater proportion of statistically significant positive results of experiments to be published and, conversely, a greater proportion of statistically significant negative or null results not to be published. It is widely accepted in the fields of healthcare and psychological research to be a major threat to the validity of systematic reviews and meta-analyses. Some methodological work has previously been undertaken, by the author and others, in the field of educational (...) research to investigate the extent of the problem. This paper describes the problem of publication bias with reference to its history in a number of fields, with special reference to the area of educational research. Informal methods for detecting publication bias in systematic reviews and meta-analyses of controlled trials are outlined and retrospective and prospective methods for dealing with the problem are suggested. (shrink)

ABSTRACT: The term ‘publication bias’ usually refers to the tendency for a greater proportion of statistically significant positive results of experiments to be published and, conversely, a greater proportion of statistically significant negative or null results not to be published. It is widely accepted in the fields of healthcare and psychological research to be a major threat to the validity of systematic reviews and meta-analyses. Some methodological work has previously been undertaken, by the author and others, in the field of (...) educational research to investigate the extent of the problem. This paper describes the problem of publication bias with reference to its history in a number of fields, with special reference to the area of educational research. Informal methods for detecting publication bias in systematic reviews and meta-analyses of controlled trials are outlined and retrospective and prospective methods for dealing with the problem are suggested. (shrink)

Randomised controlled trials (RCTs) are often seen as the 'gold standard' of evaluative research. However, whilst randomisation will ensure comparable groups, trials are still vulnerable to a range of biases that can undermine their internal validity. In this paper we describe a number of common threats to the internal validity of RCTs and methods of countering them. We highlight a number of examples from randomised trials in education and health care where problems of execution and analysis of the RCT has (...) undermined their internal validity. However, awareness of these potential biases can lead to careful planning to avoid or reduce their occurrence. If good quality randomised trials are to inform policy and practice in education then rigorous trials need to be designed that are the least susceptible to threats to their validity. (shrink)

The Carol J. Adams Reader gathers together Adams's foundational and recent articles in the fields of critical studies, animal studies, media studies, vegan studies, ecofeminism and feminism, as well as relevant interviews and conversations in which Adams identifies key concepts and new developments in her decades-long work. This volume, a companion to The Sexual Politics of Meat (Bloomsbury Revelations), offers insight into a variety of urgent issues for our contemporary world: Why do batterers harm animals? What is the relationship between (...) genocide and attitudes toward other animals? How do activism and theory feed each other? How do race, gender, and species categories interact in strengthening oppressive attitudes? In clear language, Adams identifies the often hidden aspects of cultural presumptions. The essays and conversations found here capture the decades-long energy and vision that continue to shape new ways of thinking about and responding to oppression. (shrink)

On the surface, developing a social psychology of science seems compelling as a way to understand how individual social cognition – in aggregate – contributes towards individual and group behavior within scientific communities (Kitcher, 2002). However, in cases where the functional input-output profile of psychological processes cannot be mapped directly onto the observed behavior of working scientists, it becomes clear that the relationship between psychological claims and normative philosophy of science should be refined. For example, a robust body of social (...) psychological research demonstrates implicit gender bias in the evaluation of others (e.g., Steinpreis, Anders, & Ritzke, 1999). Many expected implicit bias to be a major cause of women’s underrepresentation in math intensive fields of science; however, quantitative sociological research of hiring and manuscript and grant evaluation has discovered no gender disparity in outcomes (Ceci & Williams, 2011). Why might this be so? This paper will discuss methodological challenges that go beyond classic problems of external validity in extrapolating psychological effects and explanations to scientific communities. These problems include more complex external validity issues raised by the introduction of multi-process models of cognition (e.g., implicit versus explicit social cognition) as well as the reflexive role that folk and experimental theories of social psychology play in guiding the behavior of scientists at the individual and community level. (shrink)

New Tenth Anniversary edition of this classic text with a new preface by the author, compares myths about meat-eating with myths about manliness, and seeks to ...

Deeply rooted structures of racism, ableism, misogyny, ageism, and transphobia hurt great numbers of people, exposing them to intolerance, economic exclusion, and physical harm around the globe. Billions of land animals suffer and die annually in concentrated feeding operations and slaughterhouses. Our planet and all who live here are in perilous straights as the climate changes. In the face of such grievous problems, people who want to find positive ways to respond often grapple with difficult questions about how to make (...) a difference. Effective Altruism has arisen as an attempt to help answer these questions, encouraging people to give as much money as they can in the most effective ways to address human and animal suffering. But the Effective Altruists’ answers support some of the very social structures that cause suffering, undermining its efforts to “do the most good.” The Good It Promises, The Harm It Does is the first book-length critique of Effective Altruism. It brings together a diverse group of activists and scholars to explore Effective Altruism’s failure to meaningfully address forms of human and animal suffering. (shrink)

In just a few years, the book became an underground classic. Neither Man Nor Beast takes Adams' thought one step further.

In this essay, I will argue that contemporary ecofeminist discourse, while potentially adequate to deal with the issue of animals, is now inadequate because it fails to give consistent conceptual place to the domination of animals as a significant aspect of the domination of nature. I will examine six answers ecofeminists could give for not including animals explicitly in ecofeminist analyses and show how a persistent patriarchal ideology regarding animals as instruments has kept the experience of animals from being fully (...) incorporated within ecofeminism. (shrink)

In this essay, I will argue that contemporary ecofeminist discourse, while potentially adequate to deal with the issue of animals, is now inadequate because it fails to give consistent conceptual place to the domination of animals as a significant aspect of the domination of nature. I will examine six answers ecofeminists could give for not including animals explicitly in ecofeminist analyses and show how a persistent patriarchal ideology regarding animals as instruments has kept the experience of animals from being fully (...) incorporated within ecofeminism.2. (shrink)

Research on bias in peer review examines scholarly communication and funding processes to assess the epistemic and social legitimacy of the mechanisms by which knowledge communities vet and self-regulate their work. Despite vocal concerns, a closer look at the empirical and methodological limitations of research on bias raises questions about the existence and extent of many hypothesized forms of bias. In addition, the notion of bias is predicated on an implicit ideal that, once articulated, raises questions about the normative implications (...) of research on bias in peer review. This review provides a brief description of the function, history, and scope of peer review; articulates and critiques the conception of bias unifying research on bias in peer review; characterizes and examines the empirical, methodological, and normative claims of bias in peer review research; and assesses possible alternatives to the status quo. We close by identifying ways to expand conceptions and studies of bias to countenance the complexity of social interactions among actors involved directly and indirectly in peer review. (shrink)

In this essay, I connect the sexual victimization of women, children, and pet animals with the violence manifest in a patriarchal culture. After discussing these connections, I demonstrate the importance of taking seriously these connections because of their implications for conceptual analysis, epistemology, and political, environmental, and applied philosophy. My goal is to broaden our understanding of issues relevant to creating peace and to provide some suggestions about what must be included in any adequate feminist peace politics.

To arrive at their final evaluation of a manuscript or grant proposal, reviewers must convert a submission’s strengths and weaknesses for heterogeneous peer review criteria into a single metric of quality or merit. I identify this process of commensuration as the locus for a new kind of peer review bias. Commensuration bias illuminates how the systematic prioritization of some peer review criteria over others permits and facilitates problematic patterns of publication and funding in science. Commensuration bias also foregrounds a range (...) of structural strategies for realigning peer review practices and institutions with the aims of science. (shrink)

An empirically sensitive formulation of the norms of transformative criticism must recognize that even public and shared standards of evaluation can be implemented in ways that unintentionally perpetuate and reproduce forms of social bias that are epistemically detrimental. Helen Longino’s theory can explain and redress such social bias by treating peer evaluations as hypotheses based on data and by requiring a kind of perspectival diversity that bears, not on the content of the community’s knowledge claims, but on the beliefs and (...) norms of the culture of the knowledge community itself. To illustrate how socializing cognition can bias evaluations, we focus on peer-review practices, with some discussion of peer-review practices in philosophy. Data include responses to surveys by editors from general philosophy journals, as well as analyses of reviews and editorial decisions for the 2007 Cognitive Science Society Conference. (shrink)

Psychometrically oriented researchers construe low inter-rater reliability measures for expert peer reviewers as damning for the practice of peer review. I argue that this perspective overlooks different forms of normatively appropriate disagreement among reviewers. Of special interest are Kuhnian questions about the extent to which variance in reviewer ratings can be accounted for by normatively appropriate disagreements about how to interpret and apply evaluative criteria within disciplines during times of normal science. Until these empirical-cum-philosophical analyses are done, it will remain (...) unclear the extent to which low inter-rater reliability measures represent reasonable disagreement rather than arbitrary differences between reviewers. (shrink)

The existential analysis -- The death of dasein -- The timeliness of dasein -- The derivation of time -- The time of being.

Scholars belong to multiple communities of credit simultaneously. When these communities disagree about a scholarly achievement’s credit assignment, this raises a puzzle for decision and game theor...

Under the traditional system of peer-reviewed publication, the degree of prestige conferred to authors by successful publication is tied to the degree of the intellectual rigor of its peer review process: ambitious scientists do well professionally by doing well epistemically. As a result, we should expect journal editors, in their dual role as epistemic evaluators and prestige-allocators, to have the power to motivate improved author behavior through the tightening of publication requirements. Contrary to this expectation, I will argue that the (...) publication bias literature in academic medicine demonstrates that editor interventions have had limited effectiveness in improving the health of the publication and trial registration record, suggesting that much stronger interventions are needed. (shrink)

Sister Species: Women, Animals, and Social Justice addresses interconnections between speciesism, sexism, racism, and homophobia, clarifying why social justice activists in the twenty-first century must challenge intersecting forms of oppression. This anthology presents bold and gripping--sometimes horrifying--personal narratives from fourteen activists who have personally explored links of oppression between humans and animals, including such exploitative enterprises as cockfighting, factory farming, vivisection, and the bushmeat trade. Sister Species asks readers to rethink how they view "others," how they affect animals with their (...) daily choices, and how they might bring change for all who are abused. These essays remind readers that women have always been important to social justice and animal advocacy, and they urge each of us to recognize the links that continue to bind all oppressed individuals. The astonishing honesty of these contributors demonstrates with painful clarity why every woman should be an animal activist and why every animal activist should be a feminist. Contributors are Carol J. Adams, Tara Sophia Bahna-James, Karen Davis, Elizabeth Jane Farians, Hope Ferdowsian, Linda Fisher, Twyla François, Christine Garcia, A. Breeze Harper, Sangamithra Iyer, Pattrice Jones, Lisa Kemmerer, Allison Lance, Ingrid Newkirk, Lauren Ornelas, and Miyun Park. (shrink)

There is an increasing push by journals to ensure that data and products related to published papers are shared as part of a cultural move to promote transparency, reproducibility, and trust in the scientific literature. Yet few journals commit to evaluating their effectiveness in implementing reporting standards aimed at meeting those goals (1, 2). Similarly, though the vast majority of journals endorse peer review as an approach to ensure trust in the literature, few make their peer review data available to (...) evaluate effectiveness toward achieving concrete measures of quality, including consistency and completeness in meeting reporting standards. Remedying these apparent disconnects is critical for closing the gap between guidance recommendations and actual reporting behavior. We see this as a collective action problem requiring leadership and investment by publishers, who can be incentivized through mechanisms that allow them to manage reputational risk and through continued innovation in journal assessment. (shrink)

One of the central features of Western existence is the objectification and use of other beings in creating the subjectification of human beings. My argument is for a Christian veganism that rejects the dependence of the subject on the object status of other beings. The roadblocks to recognizing the necessity for Christian veganism I call the pedagogy of the oppressor. I propose that one way to change the subject-object relationship is a poetics of Christian engagement. Christian veganism may seem a (...) radical position theoretically and pragmatically, but I will offer suggestions for expanding Christian engagement with other animals and for the food and environmental justice movements of which veganism is a part. (shrink)

Merton envisioned his norms of science at a time when peer-reviewed journals controlled scientific communication. Technologies for sharing and finding content have since divorced the certification and amplification of science, generating systemic vulnerabilities. Certified amplification—a new Mertonian-styled norm—enjoins their recoupling and introduces a taxonomy of strategies adopted by institutions to close the certification-amplification gap, including the proportioning of the one to the other. Examples illustrating each taxonomic type collectively paint a picture of an ethos employing a rich range of certification (...) and amplification techniques and emerging in a decentralized fashion across heterogeneous objects, communication modalities, and institutions. (shrink)

Psychologists and philosophers have not yet resolved what they take implicit attitudes to be; and, some, concerned about limitations in the psychometric evidence, have even challenged the predictive and theoretical value of positing implicit attitudes in explanations for social behavior. In the midst of this debate, prominent stakeholders in science have called for scientific communities to recognize and countenance implicit bias in STEM fields. In this paper, I stake out a stakeholder conception of implicit bias that responds to these challenges (...) in ways that are responsive to the psychometric evidence, while also being resilient to the sorts of disagreements and scientific progress that would not undermine the soundness of this call. Along the way, my account advocates for attributing collective (group-level) implicit attitudes rather than individual-level implicit attitudes. This position raises new puzzles for future research on the relationship (metaphysical, epistemic, and ethical) between collective implicit attitudes and individual-level attitudes. (shrink)

Psychologists' work on conversational pragmatics and judgment suggests a refreshing approach to charitable interpretation and theorizing. This charitable approach—what I call Gricean charity —recognizes the role of conversational assumptions and norms in subject-experimenter communication. In this paper, I outline the methodological lessons Gricean charity gleans from psychologists' work in conversational pragmatics. In particular, Gricean charity imposes specific evidential standards requiring that researchers collect empirical information about (1) the conditions of successful and unsuccessful communication for specific experimental contexts, and (2) the (...) conversational norms governing communication in experimental contexts. More generally, the Gricean turn in psychological research shifts focus from attributional to reflexive, situational explanations. Gricean charity does not primarily seek to rationalize subject responses. Rather, it imposes evidential requirements on psychological studies for the purpose of gaining a more accurate picture of the surprising and muddled ways in which we weigh evidence and draw. Key Words: Gricean charity • methodological rationalism • interpretation • principle of charity • cognitive psychology • conversational pragmatics • heuristics and biases • reflexive analysis. (shrink)

In his debates with Daniel Kahneman and Amos Tversky, Gerd Gigerenzer puts forward a stricter standard for the proper representation of judgment heuristics. I argue that Gigerenzer’s stricter standard contributes to naturalized epistemology in two ways. First, Gigerenzer’s standard can be used to winnow away cognitive processes that are inappropriately characterized and should not be used in the epistemic evaluation of belief. Second, Gigerenzer’s critique helps to recast the generality problem in naturalized epistemology and cognitive psychology as the methodological problem (...) of identifying criteria for the appropriate specification and characterization of cognitive processes in psychological explanations. I conclude that naturalized epistemologists seeking to address the generality problem should turn their focus to methodological questions about the proper characterization of cognitive processes for the purposes of psychological explanation. (shrink)

The White Coats for Black Lives and #ShutDownSTEM movements have galvanised biomedical practitioners and researchers to eliminate institutional and systematic racism, including barriers faced by Black researchers in biomedicine and science, technology, engineering, and mathematics. In our study on Black–White funding gaps for National Institutes of Health Research Project grants, we found that the overall award rate for Black applicants is 55% of that for white applicants. How can systems for allocating research grant funding be made more fair while improving (...) their efficiency? (shrink)

What is the current status of Asian Americans in philosophy? How do Asian Americans fare in comparison to other minority groups? And, what professional strategies might they use (more or less successfully) in response to their counterstereotypical status in philosophy? In this piece, I will address these questions empirically by extrapolating from available demographic, survey, and experimental studies. This analysis will be too fast and loose, but I offer it in the spirit of constructing a broad-brushed sketch— painted from a (...) pallet of variegated data—for others to critique, improve, and displace. (shrink)

is normative in the sense that it aims to make recommendations for improving human judgment; it aims to have a practical impact on morally and politically significant human decisions and actions; and it studies normative, rational judgment qua rational judgment. These nonstandard ways of understanding ACP as normative collectively suggest a new interpretation of the strong replacement thesis that does not call for replacing normative epistemic concepts, relations, and inquiries with descriptive, causal ones. Rather, it calls for recognizing that the (...) aims and normative inquiries of epistemology and normative psychology have become intermutual in nature. Key Words: Heuristics and biases • applied cognitive psychology • normative psychology • rationality • naturalized epistemology • Epistemics • Applied Naturalized Epistemology • strong replacement • strategic reliabilism • ameliorative psychology. (shrink)

Arguments in favor of legalized assisted suicide often center on issues of personal privacy and freedom of choice over one's body. Many disability advocates assert, however, that autonomy arguments neglect the complex sociopolitical determinants of despair for people with disabilities. Specifically, they argue that social approval of suicide for individuals with irreversible conditions is discriminatory and that relaxing restrictions on assisted suicide would jeopardize, not advance, the freedom of persons with disabilities to direct the lives they choose. This paper examines (...) the idea promoted by some proponents of assisted suicide that it is reasonable to be depressed about one's diminished quality of life in cases of irreversible illness or disability and, therefore, such depression should not call into question the individual's competence to request assistance in dying. The concept of rational depression is defined and examined in the context of: four real-life cases involving individuals with disabilities who requested assistance in dying; a set of criteria commonly applied to decision-making to determine rationality; and research bearing on the emotional status of people with disabilities. It is concluded that although disability is associated with particular socially mediated stressors, there is no theoretical or empirical evidence to indicate that depression and its role in the right to die is dynamically different, more natural, or more reasonable for disabled people than for non-disabled people. (shrink)

I motivate and articulate a dispositional account of aversive racism. By conceptualizing and measuring attitudes in terms of their full distribution, rather than in terms of their mode or mean preference, my account of dispositional attitudes gives ambivalent attitudes (qua attitude) the ability to predict aggregate behavior. This account can be distinguished from other dispositional accounts of attitude by its ability to characterize ambivalent attitudes such as aversive racism at the attitudinal rather than the sub-attitudinal level and its deeper appreciation (...) of the analogy between traits and attitudes. (shrink)

In his early experimental work with Suppes, Davidson adopted rationality assumptions, not as necessary constraints on interpretation, but as practical conceits in addressing methodological problems faced by experimenters studying decision making under uncertainty. Although the content of their theory has since been undermined, their methodological approach—a Galilean form of methodological rationalism—lives on in contemporary psychological research. This article draws on Max Weber’s verstehen to articulate an account of Galilean methodological rationalism; explains how anomalies faced by Davidson’s early experimental work gave (...) rise to his later, canonical claims about rationality and interpretation; and reclaims this Galilean framework for use in contemporary psychological research. (shrink)

The expectations of governments, science students, and employers of science graduates seem to be reshaping science education and redefining science work to make them more relevant to industry’s needs. But the skills, attitudes, and values required for science work in industry have not been clearly articulated. As a result, science teaching innovations may not be adequately addressing the challenges of preparing science students for a socially significant role in industry. This article reports some qualitative research on the characteristics of innovators (...) and then explores the nature of the problem facing science educators interested in nurturing those characteristics in their students. It suggests the desirable characteristics of innovators might be inconsistent with conventional notions of scientific competence. It suggests critical engagement with science’s fundamental beliefs and values is called for before the problem of teaching scientists to be “in-competent” can be tackled. (shrink)