In the past few decades, peer review has come to dominate virtually all professionally respectable academic and scientific publications. However, despite its near-universal acceptance, no code of conduct has been developed to which peer reviewers and their editors are encouraged to adhere. This paper proposes such a code of conduct.
A growing interest in and concern about the adequacy and fairness of modern peer-review practices in publication and funding are apparent across a wide range of scientific disciplines. Although questions about reliability, accountability, reviewer bias, and competence have been raised, there has been very little direct research on these variables.
Abstract. This paper argues that the “Argument from Moral Peer Disagreement” fails to make a case for widespread moral skepticism. The main reason for this is that the argument rests on a too strong assumption about the normative significance of peer disagreement (and higher-order evidence more generally). In order to demonstrate this, I distinguish two competing ways in which one might explain higher-order defeat. According to what I call the “Objective Defeat Explanation” it is the mere possession of (...) higher-order evidence that explains defeat. I argue that this type of explanation is problematic and that it at best collapses into another explanation I call the “Subjective Defeat Explanation”. According to this explanation, it is coming to believe that one’s belief fails to be rational that explains defeat. Then I go on to argue that the Subjective Defeat Explanation is able to provide a straightforward explanation of higher-order defeat but that it entails that peer disagreement (and higher-order evidence more generally) only contingently gives rise to defeat, and importantly, that the condition it is contingent upon is very often not satisfied when it comes to moral peer disagreement specifically. As a result, it appears that moral knowledge is seldom threatened by moral peer disagreement. (shrink)
Research on bias in peer review examines scholarly communication and funding processes to assess the epistemic and social legitimacy of the mechanisms by which knowledge communities vet and self-regulate their work. Despite vocal concerns, a closer look at the empirical and methodological limitations of research on bias raises questions about the existence and extent of many hypothesized forms of bias. In addition, the notion of bias is predicated on an implicit ideal that, once articulated, raises questions about the normative (...) implications of research on bias in peer review. This review provides a brief description of the function, history, and scope of peer review; articulates and critiques the conception of bias unifying research on bias in peer review; characterizes and examines the empirical, methodological, and normative claims of bias in peer review research; and assesses possible alternatives to the status quo. We close by identifying ways to expand conceptions and studies of bias to countenance the complexity of social interactions among actors involved directly and indirectly in peer review. (shrink)
A growing body of literature has identified potential problems that can compromise the quality, fairness, and integrity of journal peer review, including inadequate review, inconsistent reviewer reports, reviewer biases, and ethical transgressions by reviewers. We examine the evidence concerning these problems and discuss proposed reforms, including double-blind and open review. Regardless of the outcome of additional research or attempts at reforming the system, it is clear that editors are the linchpin of peer review, since they make decisions that (...) have a significant impact on the process and its outcome. We consider some of the steps editors should take to promote quality, fairness and integrity in different stages of the peer review process and make some recommendations for editorial conduct and decision-making. (shrink)
Prepublication peer review should be abolished. We consider the effects that such a change will have on the social structure of science, paying particular attention to the changed incentive structure and the likely effects on the behaviour of individual scientists. We evaluate these changes from the perspective of epistemic consequentialism. We find that where the effects of abolishing prepublication peer review can be evaluated with a reasonable level of confidence based on presently available evidence, they are either positive (...) or neutral. We conclude that on present evidence abolishing peer review weakly dominates the status quo. (shrink)
Peer review process helps in evaluating and validating of research that is published in the journals. U.S. Office of Research Integrity reported that data fraudulence was found to be involved in 94% cases of misconduct from 228 identified articles between 1994–2012. If fraud in published article are significantly as high as reported, the question arise in mind, were these articles peer reviewed? Another report said that the reviewers failed to detect 16 cases of fabricated article of Jan Hendrick (...) Schon. Superficial peer reviewing process does not reveals suspicion of misconduct. Lack of knowledge of systemic review process not only demolish the academic integrity in publication but also loss the trust of the people of the institution, the nation, and the world. The aim of this review article is to aware stakeholders specially novice reviewers about the peer review system. Beginners will understand how to review an article and they can justify better action choices in dealing with reviewing an article. (shrink)
To arrive at their final evaluation of a manuscript or grant proposal, reviewers must convert a submission’s strengths and weaknesses for heterogeneous peer review criteria into a single metric of quality or merit. I identify this process of commensuration as the locus for a new kind of peer review bias. Commensuration bias illuminates how the systematic prioritization of some peer review criteria over others permits and facilitates problematic patterns of publication and funding in science. Commensuration bias also (...) foregrounds a range of structural strategies for realigning peer review practices and institutions with the aims of science. (shrink)
The relationship between unethical peer behavior and observers’ unethical behavior traditionally has been examined from a social learning perspective. We employ two additional theoretical lenses, social identity theory and social comparison theory, each of which offers additional insight into this relationship. Data from 600 undergraduate business students in two universities provide support for all the three perspectives, suggesting that unethical behavior is influenced by social learning, social identity, and social comparison processes. Implications for managers and future research are discussed.
With the advent of the “Clean India” campaign in India, a renewed focus on cleanliness has started, with a special focus on sanitation. There have been efforts in the past to provide sanitation related services. However, there were several challenges in provisioning. Provision of sanitation is a public health imperative given increased instances of antimicrobial resistance in India. This paper focuses on sanitation provisioning in the city of Mumbai, especially in the slums of Mumbai. The paper compares and contrasts different (...) models of sanitation provision including supply-led provisioning of sanitation by the Indian government to demand-led provisioning of sanitation through a World Bank funded “Slum Sanitation Program” (SSP). The paper also outlines a comparative perspective on the implementation and usage of toilet blocks. The author presents the theory of social networks and positive peer pressure and argues that these will amplify the effect of other incentives. With the help of an illustration, this paper concludes that the sustainable sanitation policy should look at facilitating and creating the infrastructure as a network and not strictly at building toilet blocks. (shrink)
In this paper we propose and analyze a game-theoretic model of the epistemology of peer disagreement. In this model, the peers' rationality is evaluated in terms of their probability of ending the disagreement with a true belief. We find that different strategies---in particular, one based on the Steadfast View and one based on the Conciliatory View---are rational depending on the truth-sensitivity of the individuals involved in the disagreement. Interestingly, the Steadfast and the Conciliatory Views can even be rational simultaneously (...) in some circumstances. We tentatively provide some reasons to favor the Conciliatory View in such cases. We argue that the game-theoretic perspective is a fruitful one in this debate, and this fruitfulness has not been exhausted by the present paper. (shrink)
One explanation of rational peer disagreement is that agents find themselves in an epistemically permissive situation. In fact, some authors have suggested that, while evidence could be impermissive at the intrapersonal level, it is permissive at the interpersonal level. In this paper, I challenge such a claim. I will argue that, at least in cases of rational disagreement under full disclosure, there cannot be more interpersonal epistemically permissive situations than there are intrapersonal epistemically permissive situations. In other words, with (...) respect to cases of disagreement under full disclosure, I will argue that there is a necessary connection between interpersonal permissiveness and its intrapersonal counterpart. Specifically, I claim that a plausible principle of correct argumentation supports such a bridge. (shrink)
This paper presents a new solution to the problem of peer disagreement that distinguishes two principles of rational belief, here called probability and autonomy. When we discover that we disagree with peers, there is one sense in which we rationally ought to suspend belief, and another in which we rationally ought to retain our original belief. In the first sense, we aim to believe what is most probably true according to our total evidence, including testimony from peers and authorities. (...) In the second, we aim to base our beliefs only on objective evidence and argumentation, even if that lowers the probability of their being true. The first principle of rational belief tends to serve the short-term epistemic interests of individuals, while the second tends to serve the long-term epistemic interests of both individuals and groups. The best way to reconcile these principles in cases of peer disagreement is to associate them with two corresponding species of belief, here called perception and opinion. (shrink)
An advertising firm''s ethical culture (as defined by the firm''s managerial and peer ethical behaviors) may affect the employees'' comfort levels and ethical behaviors. In this research, scenarios were used to describe advertising firms with various ethical cultures. Respondents'' perceived comfort levels in working for the firms described in the scenarios and the respondents'' behavioral intentions when faced with various advertising situations were assessed. Results of the study indicate that peer ethical behavior exerts a strong influence on the (...) comfort or discomfort level and the ethical behavioral intentions of potential advertising employees. Further, the strong influence exerted by peers seems to transcend the ethical behavior of the manager and carry over to the attitude toward the entire corporate advertising environment. This study provides insights for firms and researchers interested in assessing the impact of an advertising firm''s ethical culture on potential employees. (shrink)
Psychometrically oriented researchers construe low inter-rater reliability measures for expert peer reviewers as damning for the practice of peer review. I argue that this perspective overlooks different forms of normatively appropriate disagreement among reviewers. Of special interest are Kuhnian questions about the extent to which variance in reviewer ratings can be accounted for by normatively appropriate disagreements about how to interpret and apply evaluative criteria within disciplines during times of normal science. Until these empirical-cum-philosophical analyses are done, it (...) will remain unclear the extent to which low inter-rater reliability measures represent reasonable disagreement rather than arbitrary differences between reviewers. (shrink)
This chapter addresses an ambiguity in some of the literature on rational peer disagreement about the use of the term 'rational'. In the literature 'rational' is used to describe a variety of normative statuses related to reasons, justification, and reasoning. This chapter focuses most closely on the upshot of peer disagreement for what is rationally required of parties to a peer disagreement. This follows recent work in theoretical reason which treats rationality as a system of requirements among (...) an agent's mental states. It is argued that peer disagreement has either no, or a very limited, affect on what rationality requires of an agent in a given circumstance. This is in part because of difficulties generated by a novel example of evidence of evidence of p being evidence against p. This example calls into question the mechanisms whereby peer disagreement might affect what is rationally required of an agent. The chapter also reevaluates the importance of actual peer disagreement against the backdrop of prior expectations about whether disagreement is believed to be likely, arguing that peer disagreement is most likely to change what is rationally required of an agent when it is believed to be unlikely. (shrink)
Scientific communication takes place at two registers: first, interactions with colleagues in close proximity—members of a network, school of thought or circle; second, depersonalised transactions among a potentially unlimited number of scholars can be involved (e.g., author and readers). The interference between the two registers in the process of peer review produces a drift toward conflict of interest. Three particular cases of peer review are differentiated: journal submissions, grant applications and applications for tenure. The current conflict of interest (...) policies do not cover all these areas. Furthermore, they have a number of flaws, which involves an excessive reliance on scholars’ personal integrity. Conflicts of interest could be managed more efficiently if several elements and rules of the judicial process were accepted in science. The analysis relies on both primary and secondary data with a particular focus on Canada. (shrink)
Pre-publication certification through peer review stands in need of philosophical examination. In this paper, philosopher-psychologist Steven James Bartlett recalls the arguments marshalled four hundred years ago by English poet John Milton against restraint of publication by the "gatekeepers of publication," AKA today's peer reviewers.
The authors argue in favor of the “nonconciliation” (or “steadfast”) position concerning the problem of peer disagreement. Throughout the paper they place heavy emphasis on matters of phenomenology—on how things seem epistemically with respect to the net import of one’s available evidence vis-à-vis the disputed claim p, and on how such phenomenology is affected by the awareness that an interlocutor whom one initially regards as an epistemic peer disagrees with oneself about p. Central to the argument is a (...) nested goal/sub-goal hierarchy that the authors claim is inherent to the structure of epistemically responsible belief-formation: pursuing true beliefs by pursuing beliefs that are objectively likely given one’s total available evidence; pursuing this sub-goal by pursuing beliefs that are likely true (given that evidence) relative to one’s own deep epistemic sensibility; and pursuing this sub-sub-goal by forming beliefs in accordance with one’s own all-in, ultima facie, epistemic seemings. (shrink)
Peer Instruction is a simple and effective technique you can use to make lectures more interactive, more engaging, and more effective learning experiences. Although well known in science and mathematics, the technique appears to be little known in the humanities. In this paper, we explain how Peer Instruction can be applied in philosophy lectures. We report the results from our own experience of using Peer Instruction in undergraduate courses in philosophy, formal logic, and critical thinking. We have (...) consistently found it to be a highly effective method of improving the lecture experience for both students and the lecturer. (shrink)
The demand for greater public accountability is changing the nature of ex ante peer review at public science agencies worldwide. Based on a four year research project, this essay examines these changes through an analysis of the process of grant proposal review at two US public science agencies, the National Institutes of Health (NIH) and the National Science Foundation (NSF). Weaving historical and conceptual narratives with analytical accounts, we describe the ways in which these two agencies struggle with the (...) question of incorporating considerations of societal impact into the process of peer review. We use this comparative analysis to draw two main conclusions. First, evaluation of broader societal impacts is not different in kind from evaluation of intellectual merit. Second, the scientific community may actually bolster its autonomy by taking a broader range of considerations into its peer review processes. (shrink)
This article reports the results of an anonymous survey of researchers at a government research institution concerning their perceptions about ethical problems with journal peer review. Incompetent review was the most common ethical problem reported by the respondents, with 61.8% (SE = 3.3%) claiming to have experienced this at some point during peer review. Bias (50.5%, SE = 3.4%) was the next most common problem. About 22.7% (SE = 2.8%) of respondents said that a reviewer had required them (...) to include unnecessary references to his/her publication(s), 17.7% (SE = 2.6%) said that comments from reviewers had included personal attacks, and 9.6% (SE = 2.0%) stated that reviewers had delayed publication to publish a paper on the same topic. Two of the most serious violations of peer review ethics, breach of confidentiality (6.8%, SE = 1.7%) and using ideas, data, or methods without permission (5%, SE = 1.5%) were perceived less often than the other problems. We recommend that other investigators follow up on our exploratory research with additional studies on the ethics of peer review. (shrink)
In a situation of peer disagreement, peers are usually assumed to share the same evidence. However they might not share the same evidence for the epistemic system used to process the evidence. This synchronic complication of the peer disagreement debate suggested by Goldman (In Feldman R, Warfield T (eds) (2010) Disagreement. Oxford University Press, Oxford, pp 187–215) is elaborated diachronically by use of a simulation. The Hegselmann–Krause model is extended to multiple epistemic systems and used to investigate the (...) role of consensus and difference splitting in peer disagreement. I find that the very possibility of multiple epistemic systems downgrades the epistemic value of consensus and makes difference splitting a suboptimal strategy. (shrink)
With the ever expanding array of professional journals, pressures on the peer review process have increased considerably. Unless editors and publishers recognize the need for improving the efficiency of the process, the future of traditional peer review may be at risk. This is a review of the studies that have followed up the suggestions made by Ingelfinger in 1974 for improvement of manuscript peer review. Implementation of changes has been slow, despite the abundance of literature that suggests (...) the necessary improvements. Conscientious self-regulation is expected of editors who, in the current publication scenario, possess enormous power without liability. Suitability of peer review to outsourcing should be assessed and if it is absolutely essential to outsource peer review, care should be taken to ensure that it is implemented systematically and monitored regularly for quality. Finally, it is time for high earning publishers to consider compensation for the efforts of the reviewers. (shrink)
Two important aspects of the relationship between peer review and innovation includes the acceptance of articles for publication in journals and the assessment of applications for grants for the funding of research work. While there are well-known examples of the rejection by journals of first choice of many papers that have radically changed the way we think about the world outside ourselves, such papers do get published eventually, however tortuous the process required. With grant applications the situation differs in (...) that the refusal of a grant necessarily curtails the possible research that may be attempted. Here there are many reasons for conservatism and reservation as to the ability of a grant allocation process based on peer review to deliver truly innovative investigations. Other methods are needed; although such methods need not be applied across the board, they should constitute the methods whereby some 10–20% of the grant monies are assigned. The nomination of prizes for specific accomplishments is one way of achieving innovation although this presumes that investigators or institution already have available the money necessary to effect the innovations; otherwise it is a question of the selection and funding of particular individuals or institutions and requiring them to solve particular problems that are set in the broadest of terms. (shrink)
In some recent articles, Dr. Leigh Turner [Doffing the Mask: Why Manuscript Reviewers Ought to Be Identifiable,” Journal of Academic Ethics, 1 (2003), pp. 41–48; “Promoting F.A.I.T.H. in Peer Review: Five Core Attributes in Effective Peer Review,” Journal of Academic Ethics, 1 (2003), pp. 181–188.] makes some rather critical observations regarding the processes of peer-review in academic journals. I shall note them in turn, note wherein I concur and wherein I disagree, and discuss some of Turner's suggestions (...) to resolve such difficulties. It is hoped that my comments on Turner's much-appreciated points will engage readers of this august and well-edited journal to take more seriously Turner's arguments for the sake of the betterment of academic research. (shrink)
This paper reviews the published empirical evidence concerning journal peer review consisting of 68 papers, all but three published since 1975. Peer review improves quality, but its use to screen papers has met with limited success. Current procedures to assure quality and fairness seem to discourage scientific advancement, especially important innovations, because findings that conflict with current beliefs are often judged to have defects. Editors can use procedures to encourage the publication of papers with innovative findings such as (...) invited papers, early-acceptance procedures, author nominations of reviewers, structured rating sheets, open peer review, results-blind review, and, in particular, electronic publication. Some journals are currently using these procedures. The basic principle behind the proposals is to change the decision from whether to publish a paper to how to publish it. (shrink)
Peer review and publication is one of the factors proposed in Daubert v. Merrell Dow Pharmaceuticals, Inc. as indicia of the reliability of scientific testimony. This Article traces the origins of the peer-review system, the process by which it became standard at scientific and medical journals, and the many roles it now plays. Additionally, the Author articulates the epistemological rationale for pre-publication peer-review and the inherent limitations of the system as a scientific quality-control mechanism. The Article explores (...) recent changes in science, in scientific publishing, and in the academy that have put the system under strain. The Author argues that Justice Blackmun's advice to courts - that peer-reviewed publication is relevant, but is not dispositive - is of little practical help. Instead, the Author suggests questions that courts should ask in assessing the significance of the fact that testimony is, or is not, based on peer-reviewed publication and illustrates with reference to another Bendectin case, Blum v. Merrell Dow Pharmaceuticals, Inc., where some of these questions were asked. (shrink)
A peer instruction model was used whereby 78 residence dons (36 males, 42 females) provided instruction regarding academic integrity for 324 students (125 males, 196 females) under their supervision. Quantitative and qualitative analyses were conducted to assess survey responses from both the dons and students regarding presentation content, quality, and learning. Overall, dons consistently identified information-based slides about academic integrity as the most important material for the presentations, indicating that fundamental information was needed. Although student ratings of the usefulness (...) of the presentations were middling, students did indicate knowledge gains. Both interest and personal value for academic integrity were highly predictive of positive evaluations of the presentations. Dons and students provided suggestions for improvement and identified more global concerns. (shrink)
Peer review is an important component of scholarly research. Long a black box whose practical mechanisms were unknown to researchers and readers, peer review is increasingly facing demands for accountability and improvement. Numerous studies address empirical aspects of the peer review process. Much less consideration is typically given to normative dimensions of peer review. This paper considers what authors, editors, reviewers, and readers ought to expect from the peer review process. Integrity in the review process (...) is vital if various parties are to have trust, or faith, in the credibility of peer review mechanisms. Trust in the quality of peer review can increase or diminish in response to numerous factors. Five core elements of peer review are identified. Constitutive elements of scholarly peer review include: fairness in critical analysis of manuscripts; the selection of appropriate reviewers with relevant expertise; identifiable, publicly accountable reviewers; timely reviews, and helpful critical commentary. The F.A.I.T.H. model provides a basis for linking conceptual analysis of the core norms of peer review with empirical research into the adequacy and effectiveness of various processes of peer review. The model is intended to describe core elements of high-quality peer review and suggest what factors can foster or hinder trust in the integrity of peer review. (shrink)
To be effective, whistleblowing policies should be adapted to the organisational culture. They need to be custom-made and not follow a one-size-fits-all logic, specifically when they are installed to stimulate responsible peer reporting, a highly sensitive and value-laden type of whistleblowing. This paper attempts to illustrate that grid-group cultural theory could help to construct a whistleblowing policy by linking reporting styles to the organisational culture. First, we will identify four types of policy measures that are hypothesized to be effective (...) in four types of organisational culture. Second, we develop the hypothesis that certain organisational cultures can induce peer reporting that is harmful for the organisation. The whistleblowing policy can then be used as a catalyst for cultural change. (shrink)
Despite the presumed frequency of conflicts of interest in scientific peer review, there is a paucity of data in the literature reporting on the frequency and type of conflicts that occur, particularly with regard to the peer review of basic science applications. To address this gap, the American Institute of Biological Sciences conducted a retrospective analysis of conflict of interest data from the peer review of 282 biomedical research applications via several onsite review panels. The overall conflicted-ness (...) of these panels was significantly lower than that reported for regulatory review. In addition, the majority of identified conflicts were institutional or collaborative in nature. No direct financial conflicts were identified, although this is likely due to the relatively basic science nature of the research. It was also found that 65 % of identified conflicts were manually detected by AIBS staff searching reviewer CVs and application documents, with the remaining 35 % resulting from self-reporting. The lack of self-reporting may be in part attributed to a lack of perceived risk of the conflict. This result indicates that many potential conflicts go unreported in peer review, underscoring the importance of improving detection methods and standardizing the reporting of reviewer and applicant conflict of interest information. (shrink)
The principles and practices of research peer review are described. While the principles are fundamentally generic and apply to peer review across the full spectrum of performing institutions as well as to manuscript/proposal/program peer review, the focus of this paper is peer review of proposed and ongoing programs in federal agencies. The paper describes desireable characteristics and important intangible factors in successful peer review. Also presented is a heuristic protocol for the conduct of successful (...) class='Hi'>peer review research evaluations and impact assessments. Problems with peer review are then outlined, followed by examples of peer review of proposed and existing programs in selected federal agencies. Some peer review variants, such as the Science Court, are described, and then research requirements to improve peer review are discussed. (shrink)
This article explores the role of the Deaf child as peer educator. In schools where sign languages were banned, Deaf children became the educators of their Deaf peers in a number of contexts worldwide. This paper analyses how this peer education of sign language worked in context by drawing on two examples from boarding schools for the deaf in Nicaragua and Thailand. The argument is advanced that these practices constituted a child-led oppositional pedagogy. A connection is drawn to (...) Freire’s (1972) theory of critical pedagogy. Deaf children’s actions as peer educators are framed as an act of resistance towards the oppression of their language and culture. A contrast is drawn between oralist pedagogy that is historically associated with punitive practices and didactic methods and the experiential and dialogic interaction that characterised peer learning of sign languages. The argument is made that the peer teaching and learning processes enabled the self-actualisation of the Deaf children whereas the oralist methods were based on a deficit model that focused on modifying deaf children according to the norms of hearing society. The implications of this for current policy and practice are inferred to be about access to sign languages and the importance of Deaf communities in deaf children’s education. The argument is made that space needs to be created for deaf children to engage in peer learning. (shrink)
The major challenge facing today’s biomedical researchers is the increasing competition for available funds. The competitive review process, through which the National Institutes of Health (NIH) awards grants, is built upon review by a committee of expert scientists. The NIH is firmly committed to ensuring that its peer review system is fair and objective.
A relatively high incidence of unsatisfactory review decisions is widely recognised and acknowledged as ‘the peer review problem’. Factors contributing to this problem are identified and examined. Specific examples of unreasonable rejection are considered. It is concluded that weaknesses of the ‘peer review’ system are significant and that they are well known or readily recognisable but that necessary counter-measures are not always enforced. Careful management is necessary to discount hollow opinion or error in review comment. Review and referee (...) functions should be quite separate. (shrink)
Peer review of manuscripts for biomedical journals has become a subject of intense ethical debate. One of the most contentious issues is whether or not peer review should be anonymous. This study aimed to generate a rich, empirically-grounded understanding of the values held by journal editors and peer reviewers with a view to informing journal policy. Qualitative methods were used to carry out an inductive analysis of biomedical reviewers’ and editors’ values. Data was derived from in-depth, open-ended (...) interviews with journal editors and peer reviewers. Data was “read for” themes relevant to reviewer anonymisation and interactions among editors, reviewers, and authors. Editors and peer reviewers provided three arguments that would support a more open and interactive peer-review process. First, a number of participants emphasised the importance of not only ensuring the scientific quality of published research but also nurturing their colleagues and supporting their communities. Second, many spoke about the ongoing moral responsibilities that reviewers and editors felt toward authors. Finally, participants spoke at length about their enjoyment of social interactions and of the value of collective, rather than isolated, reasoning processes. Whether or not journal editors decide to allow anonymous review , the values of editors and reviewers need to be seriously addressed in codes of publication ethics, in the management of biomedical journals, and in the establishment of journal policies. (shrink)
Peer review is a widely accepted instrument for raising the quality of science. Peer review limits the enormous unstructured influx of information and the sheer amount of dubious data, which in its absence would plunge science into chaos. In particular, peer review offers the benefit of eliminating papers that suffer from poor craftsmanship or methodological shortcomings, especially in the experimental sciences. However, we believe that peer review is not always appropriate for the evaluation of controversial hypothetical (...) science. We argue that the process of peer review can be prone to bias towards ideas that affirm the prior convictions of reviewers and against innovation and radical new ideas. Innovative hypotheses are thus highly vulnerable to being “filtered out” or made to accord with conventional wisdom by the peer review process. Consequently, having introduced peer review, the Elsevier journal Medical Hypotheses may be unable to continue its tradition as a radical journal allowing discussion of improbable or unconventional ideas. Hence we conclude by asking the publisher to consider re-introducing the system of editorial review to Medical Hypotheses. (shrink)
The documented low levels of reliability of the peer review process present a serious challenge to editors who must often base their publication decisions on conflicting referee recommendations. The purpose of this article is to discuss this process and examine ways to produce a more reliable and useful peer review system.
Born in the 17th century, journal peer review is an extremely diverse technology, constantly torn between two often incompatible goals: the validation of manuscripts conceived as a collective industrial-like reproducible process performed to assert scientific statements, and the dissemination of articles considered as a means to spur scientific discussion, raising controversies, and civically challenging a state of knowledge. Such a situation is particularly conducive to clarifying the processes of valuation and evaluation in journal peer review. In this article, (...) such processes are considered as specific tests in order to emphasize the uncertain properties of pre-tests manuscripts. On the one hand, evaluation tests are examined at the core of the validation of manuscripts, such as defining the coordination of judging instances or controlling the modalities of inter-knowledge between reviewers and authors. They are also studied regarding the dissemination of articles, notably through the contemporary conception of a continuing evaluation test termed “post publication peer review”. On the other hand, valuation tests are both part of the validation of manuscripts, such as the weighting of different judgments of the same manuscript and the tensions that these hierarchies cause, and of the dissemination of articles, such as attention metrics recording the uses of articles. The conclusion sketches out how the articulation of these different tests has recently empowered readers as a new key judging instance for dissemination and for validation, potentially transforming the definition of peers, and thus the whole process of journal peer review. (shrink)
Scientific editors’ policies, including peer review, are based mainly on tradition and belief. Do they actually achieve their desired effects, the selection of the best manuscripts and improvement of those published? Editorial decisions have important consequences—to investigators, the scientific community, and all who might benefit from correct information or be harmed by misleading research results. These decisions should be judged not just by intentions of reviewers and editors but also by the actual consequences of their actions. A small but (...) growing number of studies has put editorial policies to a strong scientific test. In a randomized, controlled trial. blinding reviewers to author and institution was usually successful and improved the quality of reviews. Two studies have shown that, contrary to conventional wisdom, reviewers early in their careers give better reviews than senior reviewers. Many studies have shown low agreement between reviewers but there is disagreement about whether this represents a failing of peer review or the expected and valuable effect of choosing reviewers with complementary expertise. In a study of whether manuscripts are improved by peer review and editing, articles published in Annals of Internal Medicine were improved in 33 of 34 dimensions of reporting quality, but published articles still had room for improvement. Because of the central place of peer review in the scientific community and the resources it requires, more studies are needed to define what it does and does not accomplish. (shrink)
This study investigated students’ perceptions following a prepared, common presentation regarding academic integrity provided by their residence dons. This peer instruction study utilized both quantitative and qualitative analyses of survey data within a pre-test post-test design. Overall, students reported gains in knowledge, as well as confidence in their knowledge of academic integrity. Notably, students reported increases in their personal value for academic integrity after participating in the presentations. Overall, the quality and content of the presentations were judged positively, and (...) participants’ ratings of the presentation were predictive of increases in personal value of academic integrity, as well as self-reported knowledge and confidence gains. Qualitative analyses supported that the key ideas in the presentation served as the focal material for discussion, but also introduced specific topics that students wanted to explore in greater depth. (shrink)
The paper offers a critical review of Roberto Farnetiâs paper a minor philosophy. The state of the art of philosophical scholarship in Italy , recently published in Philosophia. It is argued that overall the status and interest of philosophy as practiced nowadays in Italy is less disappointing than Farneti makes out. It is also maintained that submitting papers to peer-refereed international journals can help cure the moral and sociological disease that besets the Italian academia, but that, as such, it (...) is less likely to improve the scientific quality of contributions in philosophy than Farneti claims. In passing, a few recommendations both to the philosophical community at large and to the Italian Government are put forward. (shrink)
In this paper, we will discuss Peter van Inwagen’s contribution to the epistemological debate about revealed peer disagreement. Roughly, this debate focuses on situations in which at least two participants disagree on a certain proposition based on the same evidence. This leads to the problem of how one should react rationally when peer disagreement is revealed. Van Inwagen, as we will show, discusses four possible reactions, all of which he rejects as unsatisfying. Our proposal will be to point (...) to hidden assumptions in van Inwagen’s reasoning and ask whether he is willing to reject at least one of these to get rid of the problem. In short, our thesis amounts to the following: Of the two epistemological claims, which we call “Weak” and “Full-blown Fallibilism”, van Inwagen cannot simultaneously accept the first and reject the latter, while this is what he seems to suggest. Revealing this potential dilemma for van Inwagen’s position will lead to a more detailed discussion of how “rationality”, “truth”, “evidence” and “justification” interrelate and how a closer look at their relation might help solving the puzzle of revealed peer disagreement. (shrink)
Peer review is a topic of considerable concern to many researchers, and there is a correspondingly large body of research on the topic. This issue of Science and Engineering Ethics presents recent work on peer review that is both grounded in empirical science and is applicable to policy decisions. This research raises two basic questions; (a) how does current peer review operate, and (b) how can it be improved? Topics addressed include descriptions of how peer review (...) is used in Federal agencies. whether peer review leads to better manuscripts, demographic characteristics of authors or reviewers (status or institutional affiliation), blinding of reviewers, authors, or results, reliability and consistency of reviews, accepting a paper before the study is done, simultaneous submission, and use of dispute resolution procedures such as scientific dialectical and pleading protocols. (shrink)
Justifying ethical practices is no easy task. This paper considers moral justifications for peer review so as to persuade even the sceptical individualist. Two avenues provide a foundation for that justification: self-interest and social contract theory. A wider notion of “interest” permits the self-interest approach to justify not only submitting one’s own work to peer review but also removing oneself momentarily from the production of primary knowledge to serve as a rigorous, independent, and honest referee. The contract approach (...) offers a non-selfish alternative and relies on four types of binding social contracts: those implicit in accepting funds, those implicit in asserted professional status, those to contribute what is of most value to society, and those to defend the ideals of the Academy. Efforts to restore respect for rigorous, independent, honest peer review should begin in earnest. (shrink)
This paper presents a framework that editors, peer reviewers, and authors can use to identify and resolve efficiently disputes that arise during peer review in scientific journals. The framework is called a scientific dialectical brief. In this framework, differences among authors and reviewers are formatted into specific assertions and the support each party provides for its position. A literature review suggests that scientists use five main types of support; empirical data, reasoning, speculation, feelings, and status. It is suggested (...) that the scientific dialectical brief format can streamline the review process by facilitating rapid differentiation between stronger and weaker support, so that valuable time can be focused on the better-substantiated claims. The paper concludes with some suggestions for implementation. (shrink)
The question of the morality of abortion has long been the subject of intense, sometimes acrimonious debate. Even people within the same religious or philosophical tradition often disagree on the issue. For example, there are Christians who are “pro- choice” and there are Christians who are “pro-life.” Both sides marshal biblical, theological, and philosophical arguments in support of their positions. The substance of the abortion debate seems to reduce to one tricky question: when does personhood begin? Christian experts in various (...) fields, such as theology, biblical studies, ethics, and philosophy, have protracted disagreements over this question. In this article we will apply insights from the current literature on epistemic peer disagreement to the abortion issue. We will assume that there is only one correct answer to the abortion question. However, after making a crucial distinction between rationality as understood by internalists versus externalists, we will argue that there is more than one rational answer to the abortion question, since there is more than one rational way to weight evidence. We will conclude that, in a case of disagreement between two Christians who are epistemic peers with regards to the morality of abortion, both parties can be rational in adhering to their respective positions, but that this does not entail or even support ethical relativism. (shrink)