General forum:

Back    All discussions

PhilPapers and blind review
Now that this great resource has launched, it seems possible for philosophers to make unpublished work accessible to a much wider audience then previously. So I wonder: What do others think of posting papers on PhilPapers that are currently under blind review or will soon be. On the one hand, posting them seems to be a great way to get feedback that will help improve the papers. On the other hand, it may undermine the blind review of the papers for publication. 

PhilPapers and blind review
Tricky issue. I guess the right answer to your question turns on largely normative considerations. Arguably, by making papers available online with full identification when they are under blind review, one undermines the blind review system whether one is well-known or not (and more if one is well-known). Take the limit case: suppose every paper under review was always available online and easy to find. This would be functionally almost as if papers under blind review always came with the author's name on a blank page at the end (you don't have to look at it, but it's really tempting). If the blind review system is a good one, one ought not to undermine it by putting one's work online when it is under review.

I'm a little concerned by what follows: if PhilPapers encourages people to put their works online while they are under review, it contributes to undermining the blind review system. However, I'm not aware that they have a particularly acute blind review problem in disciplines where online archives are a lot more used than PhilPapers is in philosophy, e.g. in math and physics. I don't know if it is because referees are good at restraining themselves or authors don't submit to arXiv before/during blind review, but the same solution should work in philosophy if we ever get to the level of digitization they have in those fields.

It's interesting though to try to think about a better solution. Maybe we could allow people to submit blind copies of their papers to PhilPapers. The author(s) would be known to the system but not revealed until the paper's blind review mode is turned off by the original submitter. Obviously papers with anonymous authors are a bit less interesting to read, but there would be the advantage of knowing that the paper is being worked on actively and that the author is likely to want to discuss it (also anonymously) in the attached forum. I'd be curious to know what people think about this (crazy?) scheme.

PhilPapers and blind review
It strikes me that for a great deal of papers, a quick google search on the title would already reveal the author. So this seems to me a standing issue, rather than a new one. (I'm not sure that philpapers makes it easier to find papers if you know the title---google is pretty good already! Rather it points you towards papers whose titles you *don't know*---and that's a huge thing.)

One suggestion for anyone worried about the issue: submit the paper to journals under a different name to the one you put on the online version.

PhilPapers and blind review
Following Robert's suggestion, I Googled a paper of mine that is on my website. But I didn't Google the title. I Googled a random sentence in it. My paper, in pdf form, was the second result. 
So he's right that PhilPapers doesn't really make that problem worse. It's already the case that you can find a paper, even with a different title. So if one is really concerned about making sure a blind referee can't find your paper, it seems to only solution is to take it offline while under review. (As I think about it more, I'm not sure even that will work, since Google caches pages. So maybe you just shouldn't put unpublished papers online.) I've seen a few personal websites that do this. Do folks think this is the right thing to do?

Even if that weren't a worry, here's a slightly different one: It seems possible that making one's papers widely available might make it less likely that there are very few reviewers a Journal could turn to who haven't already seen the paper. Hence blind review would not be practically possible even if the reviewers don't try to find your paper.

This is not purely academic for me; I have several papers that I'm thinking of posting, but are either under review or will soon be. (They're already on my website, but for whatever reason I hadn't realized just how easy it would be to find them.) I wonder if there is a widely held view about this in the profession?

I actually like the blind papers submission idea David proposes, though I'm not sure the problems it is meant to address are worth the effort. 

PhilPapers and blind review
Yep, it's really hard to keep things blind with a reviewer who's determined to unmask the author! Even the title of a paper in a conference list, or on a list of "works in progress" can be turned up by google pretty quickly. That may be addressed by changing the title for a submitted version. But  if the paper is online, googling random sentences is indeed another way in which it can be unmasked. (Lest it seem that I spend my time unmasking authors whose papers I'm refereeing---this is the same process one goes through to find the source of unreferenced quotations, etc).

Of course, blind review is still in place if referees *don't* google search for titles or random references (or do searches in philpapers). It seems to me that there has to be a certain level of trust in the good faith of referees *not* to unmask authors (at least before the report is submitted---afterwards is a bit of a grey area).

Personally, I have been sent papers to referee where I'd already read in draft (and so knew the author) a few times. Each time, I've contacted the editor to let them know that this is the case, and usually they've asked me to go ahead with the review anyway. So that wasn't a fully blind process, even at journals that go through the right process. I don't really know what to think about that (I've treated it as the editor's responsibility to make such decisions).

I do know people who, in the light of this sort of thing, don't put drafts online. But that isn't cost-free: the whole point of the enterprise, ultimately, is about sharing research, and I find it incredibly valuable when people put stuff online. And for myself, I've got valuable feedback from online drafts in the past. So for me, the benefits of making drafts available outweigh the costs. But I'd be interested to know what others think.

PhilPapers and blind review
Supposedly one can add some code to a page to prevent Google from indexing it, although I have never tried to do it.

So a partial solution might be to link to one's work in progress only on such a modified page and link to it from your homepage.  Then at least people who knew your name and were looking for your work could find it, but reviewers looking for your paper without knowing your name would not find it.

PhilPapers and blind review
Reply to David Bourget
"However, I'm not aware that they have a particularly acute blind review problem in disciplines where online archives are a lot more used than PhilPapers is in philosophy, e.g. in math and physics."

But in math, at least, the process is not blind: although authors don't know the identity of the referees, the referees do know the identity of the authors. Indeed, a mathematician friend of mine was genuinely puzzled by the practice of blind review, and I couldn't convince him that it has any advantages.

PhilPapers and blind review
I have thought much about this and related issues in the past few years -- while I have been publishing primarily in the domain of biomedicine (with some things ending up in the general direction of drug safety and epidemiology).  Blind review is not used there, but rather "half blind review" in which the author's name (or authors' names) are known to the reviewers, but the reviewers are not made known to the author.  I really do not like this approach since I think it is the worst of all possible ones and introduces pressures that protect self-interested reviews and those who deliver them.

Historically, I have favored the normal full blind review.  However, the digital age has (as noted above) introduced severe complications since so many now make their drafts available online.  (There was something of a problem like this previously, as some have noted, in that a paper was often sent around for comments or presented in various venues prior to journal submission.  But now the problem is virtually universal.  I do wonder, as an aside, what the view of editors and publishers generally is concerning the availability of online versions since they still strive for some degree of exclusivity.  But I digress.)

When I recently challenged Edward Shorliffe concerning the policy of "half blind" reviewing with respect to the Journal of Biomedical Informatics, his argument was that in a discipline where work is done at such detailed level, it is just trivially easy to tell on the basis of paper contents what "research group" or "hospital" has done a study and hence to infer who the authors are.  So there's no point in attempting to enforce blind review.  Personally, I think this answer is nonsense and is simply a knee-jerk defense of the way in which these things have always been done in medicine -- which appears to value authority above all else.  I have become really concerned about the quality of scholarship that I'm seeing in publications in a variety of fields (including medical/biomedical ones, informatics, and computer science), and particularly at conferences where a huge ratio of presentations are made by graduate students of academics who have significant reputations in their fields.  And the presentations are very often, and I use this description with some thought, crap.  I am hoping this is not the case in philosophy, but it's been a long, long time since I attended a philosophy conference.

More recently, I have noted that the journal Applied Ontology employs a process which I have come to view as perhaps the most reasonable in our age of information.  This is a policy of totally unblind review in which the reviewers know the identity of the author(s) and the author knows the identity of the reviewers.  They are also open to suggestions from the author (upon manuscript submission) concerning who among the reviewers are felt to be potentially good candidates or potentially bad ones -- though of course their editors make the final decision.  I'm sure that there are detriments to this approach as well, but currently it appears to me to be the most reasonable.

I note also that in at least some conference, editors are providing authors with the opportunity and a period of time for a "rebuttal" to reviewers' comments.  This, too, I think is a good practice since it introduces the possibility of clarifying disagreements and enhancing objectivity -- at least in theory.  The downside of such a process is that unless the timelines are kept short and rigidly enforced, it can really stretch out time to publication.