Abstract
With the ever expanding array of professional journals, pressures on the peer review process have increased considerably. Unless editors and publishers recognize the need for improving the efficiency of the process, the future of traditional peer review may be at risk. This is a review of the studies that have followed up the suggestions made by Ingelfinger in 1974 for improvement of manuscript peer review. Implementation of changes has been slow, despite the abundance of literature that suggests the necessary improvements. Conscientious self-regulation is expected of editors who, in the current publication scenario, possess enormous power without liability. Suitability of peer review to outsourcing should be assessed and if it is absolutely essential to outsource peer review (due to financial constraints on the publisher), care should be taken to ensure that it is implemented systematically and monitored regularly for quality. Finally, it is time for high earning publishers to consider compensation (financial or otherwise) for the efforts of the reviewers.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Anderson, K. (2013). Validation vs. Filtration and Designation — Are we mismarketing the core strengths of peer review? http://scholarlykitchen.sspnet.org/2013/02/18/validation-vs-filtration-and-designation-are-we-mismarketing-the-core-strengths-of-peer-review/
Ashforth, B. E. (2008). Becoming vanilla pudding: how we undermine our passion for research. Journal of Management Inquiry, 14, 400–403.
Badeian, A. (2003). The manuscript review process: the proper roles of authors, referees and editors. Journal of Management Inquiry, 12, 331–338.
Björk, B. C., & Solomon, D. (2013). The publishing delay in scholarly peer-reviewed journals. Journal of Informetrics, 7(4), 914–923.
Black, N., von Rooyen, S., Godlee, F., Smith, R., & Evans, S. (1998). What makes a good reviewer and a good review for a general medical journal. JAMA, 280, 231–233.
Bohannon, J. (2013). Who’s Afraid of Peer Review? Science, 342(6154), 60–65.
Bornmann, L., & Daniel, H.-D. (2010). How long is the peer review process for journal manuscripts? CHIMIA, 64(1/2), 72–77.
Bornmann, L., & Daniel, H-D. (2010). Do author-suggested reviewers rate submissions more favorably than editor-suggested reviewers? A study on Atmospheric Chemistry and Physics, PLoS One, doi: 10.1371/journal.pone.0013345
Bornmann, L., & Daniel, H-D. (2010). Predictive validity of editorial decisions at an electronic open access journal. Publishing in the networked world: transforming the nature of communication 14th International Conference on Electronic Publishing (ICPE), Helsinki, Finland. http://www.elpub.net. Accessed on 4/21/2014
Bornmann, L., Mutz, R., & Daniel, H-D. (2010). A Reliability-Generalization Study of Journal Peer Reviews: A Multilevel Meta-Analysis of Inter-Rater Reliability and Its Determinants. doi: 10.1371/journal.pone.0014331
Callaham, M. L., & McCulloch, C. (2011). Longitudinal trends in the performance of scientific peer reviewers. Annals of Emergency Medicine, 57(2), 141–148.
Callaham, M. L., & Schriger, D. L. (2002). Effect of structured workshop training on subsequent performance of journal peer reviewers. Annals of Emergency Medicine, 40(3), 323–328.
Callaham, M. L., Baxt, W. G., Waeckerle, J. F., & Wears, R. L. (1998). Reliability of Editors’ Subjective Quality Ratings of Peer Reviews of Manuscript. JAMA, 280(3), 229–231.
Callaham, M. L., Knopp, R. K., & Gallagher, E. J. (2002). Effect of written feedback by editors on quality of reviews: two randomized trials. JAMA, 287(21), 2781–2783.
Carmi, R., & Koch, C. (2007). Improving peer review with CARMA. Learned Publishing, 20(3), 173–176.
Cawley, V. (2011). Is peer review unethical? An ethical analysis. 2011 International Conference on Social Science and Humanity IPEDR vol.5, 106–112. www.ijssh.org/papers/36-H058.pdf. Accessed on 4/21/2014
Chetty R, Saez E, Sándor L, (2014). How Can We Increase Prosocial Behavior? An Experiment with Referees at the Journal of Public Economics http://obs.rc.fas.harvard.edu/chetty/referee_experiment.pdf
Cicchetti, D. V. (1997). Referees, editors and publication practices: improving the reliability and usefulness of the peer review system. Science and Engineering Ethics, 3, 51–62.
Cooper, L. M. (2009). Problems, pitfalls and promise in the peer review process. Perspectives on Psychological Science, 4(1), 84–90.
COPE (Committee on Publication Ethics). (2011). – COPE revised best practice guidelines for journal editors. http://publicationethics.org/resources/guidelines
Corbyn Z. (2013). Price doesn't always buy prestige in open access. Nature doi:10.1038/nature.2013.12259
Corlett, J. A. (2005). Ethical Issues in Journal Peer-review. Journal Academy Ethics, 2(4), 355–366.
Coronel, R., & Opthof, T. (1999). The role of the reviewer in editorial decision-making. Cardiovascular Research, 43(2), 261–264.
Davis P. (2013). Society for Scholarly Publishing - Rewarding reviewers: money, prestige, or some of both? http://scholarlykitchen.sspnet.org/2013/02/22/rewarding-reviewers-money-prestige-or-some-of-both/ Accessed on 4/21/2014
Davis, P. (2013). Journal usage half-life. http://www.publishers.org/_attachments/docs/journalusagehalflife.pdf
Davis P. (2014). A metric for the quality of peer review.http://scholarlykitchen.sspnet.org/2014/02/04/a-metric-for-the-quality-of-peer-review-interview-with-adam-etkin-of-prescore/
Deci, E. L. (1971). Effects of Externally Mediated Rewards on Intrinsic Motivation. Journal of Personality and Social Psychology, 18(1), 105–115.
Deci, E. L., Koestner, R., & Ryan, R. M. (1999). A Meta-analytic Review of Experiments Examining the Effects of Extrinsic Rewards on Intrinsic Motivation. Psychological Bulletin, 125(6), 627–668.
Dzeng, E. (2014). How-academia-and-publishing-are-destroying-scientific-innovation-a-conversation-with-sydney-brenner. http://kingsreview.co.uk/magazine/blog/2014/02/24/
Eve, MP. (2013). What’s “open” got to do with it? https://www.martineve.com/2013/10/03/whats-open-got-to-do-with-it/ Accessed on 4/21/2014
Feurer, I. D., Becker, G. J., Picus, D., Ramirez, E., Darcy, M. D., & Hicks, M. E. (1994). Evaluating peer reviews: pilot testing of a grading instrument. JAMA, 272, 98–100.
Frey, B. S. (2003). Publishing as prostitution? – Choosing between one’s own ideas and academic success. Public Choice, 116, 205–223.
Frey, B. S., & Jegen, R. (2001). Motivation Crowding Theory: A Survey of Empirical Evidence. Journal of Economic Surveys, 15(5), 589–611.
Garfield, E. (1986). Refereeing and peer review, part 2: The research on refereeing and alternatives to the present system. Essays of an Information Scientist, 9, 239–248.
Gigerenzer, G., & Gaissmaier, W. (2011). Heuristic Decision Making. Annual Review of Psychology, 62, 451–482.
Goodman, D., Dowson, S., & Yaremchuk, J. (2007). Open access and accuracy: author-archived manuscripts vs. published articles. Learned Publishing, 20(3), 203–215.
Graur, D. (2014). Peer review: Payback time for referee refusal. Nature, 505, 483. doi:10.1038/505483a.
Green, S. M., & Callaham, M. L. (2011). Implementation of a journal peer reviewer stratification system based on quality and reliability. Annals of Emergency Medicine, 57(2), 149–152.
Greenberg, D., Rosen, A. B., Olchanski, N. V., Stone, P. W., Nadai, J., & Neumann, P. J. (2004). Delays in publication of cost utility analyses conducted alongside clinical trials: registry analysis. BMJ, 328, 1536–1537.
Gupta, P., Kaur, G., Sharma, B., Shah, D., & Choudhury, P. (2006). What is submitted and what gets accepted in Indian Pediatrics: analysis of submissions, review process, decision making, and criteria for rejection. Indian Pediatrics, 43(6), 479–489.
Hames, I. (2013). COPE’s new Ethical Guidelines for Peer Reviewers: background, issues, and evolution. http://c.ymcdn.com/sites/www.ismte.org/resource/resmgr/files/hames_article.pdf
Houry, D., Green, S., & Callaham, M. (2012). Does mentoring new peer reviewers improve review quality? A randomized trial. BMC Medical Education, 12, 83.
Howard, L., & Wilkinson, G. (1999). Peer review and editorial decision-making. Neuroendocrinology Letters, 20(5), 256–260.
Ingelfinger, J. (1974). Peer review in biomedical publication. American Journal of Medicine, 56(5), 686–692.
James, M. J., & Faisal, U. (2013). Empirical study on addressing high employee attrition in BPO industry focusing on employee salary and other factors in Karnataka and Kerala states of India. Research Journal Management Science, 2(9), 7–11.
Jennings, CG. (2006). ‘Quality and value: the true purpose of peer review. What you can’t measure, you can’t manage: the need for quantitative indicators in peer review’, http://www.nature.com/nature/peerreview/ debate/nature05032.html. Accessed on 4/21/2014
JISC Collections. (2011). Society Journal Publishing Transfer Guidelines to Help Achieve a Successful Transition. http://www.jisc-collections.ac.uk/Documents/Reports/jrnl%20transfer%20paper.pdf
Knowledge Exchange. (2010). Submission fees - A tool in the transition to open access? Mark Ware ConsultingLtd. http://www.knowledgeexchange.info/Files/Filer/downloads/Open%20Access/KE_Submission_fees_Short_Report_2010-11-25.pdf. Accessed on 4/21/2014
Kravitz, RL., Franks, P., Feldman, MD., Gerrity, M., Byrne, C., & Tierney, WM. (2010). Editorial Peer Reviewers' Recommendations at a General Medical Journal: Are They Reliable and Do Editors Care? PLoS One, DOI: 10.1371/journal.pone.0010072
Kumar, MN. (2009). A review of the review process: manuscript peer-review in biomedical research. Biology and Medicine, 1 (4): Rev3.
Kumar, M. N. (2010). Peer reviewer as collaborator model for publishing. Learned Publishing, 23(1), 33–38.
Lampert, RH., & Kaufman, CS. (2005). Outsourcing editorial and content production services.http://www.kwfco.com/sites/default/files/resources/Outsourcing%20and%20Offshoring%20vol%202.pdf. Accessed on 4/21/2014
Laufer, M. (2007). The culture of scientific publication. Interciencia, 32(10), 684.
Lomangino, K., Kaufman, CS., & Wills, A. (2004). Outsourcing and offshoring editorial services.http://www.kwfco.com/sites/default/files/resources/Outsourcing%20and%20Offshoring%20vol%201.pdf. accessed on 4/21/2014
Marsh, H. W., Bond, N. W., & Jayasinghe, U. W. (2007). Peer review process: Assessments by applicant-nominated referees are biased, inflated, unreliable and invalid. Australian Psychologist, 42(1), 33–38.
Moher, D., & Jadad, A. R. (2003). How to peer review a manuscript. In F. Godlee & T. Jefferson (Eds.), Peer review in the health sciences (pp. 183–190). London: BMJ Books.
Nature Editorial. (2008). It takes time. Nature Structural and Molecular Biology, 15, 889. doi:10.1038/nsmb0908-889.
Nature editorial. Peer-review policy. (2011). http://www.nature.com/authors/policies/peer_review.html. Accessed on 4/21/2014
Nickerson, R. S. (2005). What authors want from journal reviewers and editors. American Psychologist, 60, 661–662.
Paulus, W. (2008). Why are the Acta Neuropathologica so fast, with a mean time from submission to first decision of 16 days? Acta Neuropathologica, 115(3), 371–372.
PEER economics Report. (2011) www.peerproject.eu/fileadmin/…/reports/PEER_Economics_Report.pdf. Accessed on 4/21/2014
Pitkin, R. M., & Burmeister, L. F. (2002). Prodding tardy reviewers: a randomized comparison of telephone, fax, and e-mail. JAMA, 287(21), 2794–2795.
PRC (Publishing Research Consortium). (2008). ‘Peer review in scholarly journals: perspective of the scholarly community – an international study’, www.publishingresearch.org.uk/documents/PRCsummary4Warefinal.pdf. Accessed on 4/21/2014
Ray, J. G. (2002). Judging the judges: the role of journal editors (editorial). The Quarterly Journal of Medicine, 95, 769–774.
Rice, C. (2013). What Science - and the Gonzo Scientist got wrong: open access will make research better. http://curt-rice.com/2013/10/04/what-science-and-the-gonzo-scientist-got-wrong-open-access-will-make-research-better/ Accessed on 4/21/2014
Ross, D. (2014). http://connection.sagepub.com/blog/2014/04/10/open-access-in-the-humanities-and-social-sciences/
Rothwell, P. M., & Martyn, C. N. (2000). Reproducibility of peer review in clinical neuroscience – Is agreement between reviewers any greater than would be expected by chance alone? Brain, 123(9), 1964–1969.
Saper, CB., Maunsell, JH and Sagvolden., T. (2009). The neuroscience peer review consortium. Behavioral and Brain Functions, 5, 4
Schulman, K., Sulmasy, D. P., & Roney, D. (1994). Ethics, economics, and the publication policies of major medical journals. JAMA, 272, 154–156.
Sense about Science. (2004). Peer review and the acceptance of new scientific ideas. Discussion paper from a Working Party on equipping the public with an understanding of peer review. www.senseaboutscience.org/data/files/resources/17/peerReview.pdf. Accessed on 4/21/2014
Snell, L., & Spencer, J. (2005). Reviewers’ perceptions of the peer review process for a medical education journal. Medical Education, 39(1), 90–97.
Souder, L. (2011). The ethics of scholarly peer review: a review of the literature. Learned Publishing, 24(1), 55–72.
Southgate, D. A. T. (1992). The ethics of peer review (editorial). British Journal of Nutrition, 67, 303–304.
Squazzoni, F., Giangiacomo, B., & Károly, T. (2013). “Does incentive provision increase the quality of peer review? An experimental study.”. Research Policy, 42(1), 287–294.
Stamm, T. (2005). Head and Face medicine – a new journal for intra-inter disciplinary science, why? When? Where? Head and Face Medicine, 1, 1.
Stamm, T., Meyer, U., Wiesmann, H.-P., Kleinheinz, J., Cehreli, M., & Zafer, C. C. (2007). Head and Face Medicine, 3, 27.
Swan, A. (1999). What authors want: the ALPSP research study on the motivation and concerns of contributors to learned journals. Learned Publishing, 12(3), 170–172.
The Guardian. (2001). http://www.theguardian.com/uk/2001/may/26/highereducation.physicalsciences
Tite, L., & Schroter, S. (2007). Why peer reviewers decline to review? A survey. Journal of Epidemiology and Community Health, 61, 9–12.
Todd, M. (2014). http://www.socialsciencespace.com/2014/03/how-much-nsf-funding-goes-to-social-science/
Torgerson, D. J., Adamson, J., Cockayne, S., Dumville, J., & Petherick, B. E. (2005). Submission to multiple journals: a method of reducing time to publication? BMJ, 330, 305–307.
Tsang, E. W. K., & Frey, B. S. (2007). The as-is journal review process: Let authors own their ideas. Academy of Management Learning and Education, 6, 28–36.
Turner, L. (2003). Promoting F.A.I.T.H. in peer review: Five core attributes of effective peer review. Journal of Academic Ethics, 1(2), 181–188.
Value Notes. (2012). White paper - The Outsourcing Imperative for Publishers. http://www.valuenotes.biz/knowledge-center/white-papers/the-outsourcing-imperative-for-publishers/ Accessed on 4/21/2014
Van Noorden, R. (2013a). Open access: The true cost of science publishing. Nature, 495, 426–429.
Van Noorden, R. (2013b). Company offers portable peer review. Nature, 494(7436), 161. doi:10.1038/494161a.
Wates, E., & Campbell, R. (2007). Author’s version vs. publisher’s version: an analysis of the copy-editing function. Learned Publishing, 20(2), 121–129.
West, J. D., Bergstrom, T. C., & Bergstrom, C. T. (2010). Cost-effectiveness of open access publications. College and Research Libraries, 71, 236–244.
Wilkes, M. S., & Kravitz, R. L. (1995). Policies, practices, and attitudes of North American medical journal editors. Journal of General Internal Medicine, 10, 443–450.
Wong, V. S., & Callaham, M. L. (2012). Medical journal editors lacked familiarity with scientific publication issues despite training and regular exposure. Journal of Clinical Epidemiology, 65(3), 247–252.
Yankaeur, A. (1990). Who are the peer reviewers and how much do they review? JAMA, 263, 1338–1340.
Conflict of Interest
None.
Funding
The author has not received any funding for this study.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Kumar, M.N. Review of the Ethics and Etiquettes of Time Management of Manuscript Peer Review. J Acad Ethics 12, 333–346 (2014). https://doi.org/10.1007/s10805-014-9220-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10805-014-9220-4