Publishing referee reports

So stem cell researchers are calling for referee reports of published papers to themselves be published.

Stem celli biology is highly topical and is attracting great interest not only within the research community but also from politicians, patient groups and the general public. However, the standard of publications in the field is very variable. Papers that are scientifically flawed or comprise only modest technical increments often attract undue profile. At the same time publication of truly original findings may be delayed or rejected. At the recent EuroSyStem/EMBO Conference on Advances in Stem Cell Research there was much discussion about the peer review process. Peer review is the guardian of scientific legitimacy and should be both rigorous and constructive. Indeed most scientists spend considerable time and thought reviewing manuscripts. As authors we have all benefited from insightful referee reports that have improved our papers. We have also on occasion experienced unreasonable or obstructive reviews.

We suggest a simple step that would greatly improve transparency, fairness and accountability; when a paper is published, the reviews, response to reviews and associated editorial correspondence could be provided as Supplementary Information, while preserving anonymity of the referees. We note that this procedure has recently been adopted by The EMBO Journal. We wish to encourage other journals to follow suit and would like to hear your considered opinions.

This strikes me as a broadly sensible idea as it will put pressure on editors but also on referees. Even under the veil of anonymity, having your written word published does impose some discipline.

At the moment, our scientific dialogue in economics is kind of silly. We have a non-transparent set of negotiations over publication of the ‘finished work’ when the conversation is potentially equally important. We then have appendices confined to separate web repositories or links to working papers when surely space should not be the issue. Finally, we end the whole process on publication. What about allowing non-anonymous post-publication review? Given the now central role of Google Scholar in our lives, surely adding appendices, addenda, previous reviews and new reviews would all be a good idea. Let’s let the publication look more like a blog post with comments than some sort of ‘on the shelf’ book.

Alex Tabarrok worries that while pressure on editors is a good thing, it is hard to get people to take these jobs and this might discourage them. That seems weak. If something is good for performance, it is good for performance and we should consider it. In any case, I agree with him, that an experiment or to in economics might be a good idea.

One thought on “Publishing referee reports”

    Not a bad idea.  I had one referees report where it seemed they had only skimmed the paper.  Basically asked for no changes.  We had found half a dozen typos and one chart that was wrong, so our response to referee comments was also to change a bunch of things we had picked up.
    I also had one referee who was a complete nightmare.  We got involved in a 6 month, 6+ round back and forth over statistical significance, one vs two sided distributions, modelling techniques, etc…  Our collaboration figured out who it was and realized it was mostly a personal grudge with another researcher our group was closely aligned with (but who had nothing to do with the current project).


Comments are closed.

%d bloggers like this: