Statistics from Altmetric.com
F Godlee, T Jefferson (Pp 271; £30.00). London: BMJ Books, 1999. ISBN 0 7279 1181 3
Discussing the merits or otherwise of peer review in the health sciences is very similar to discussing the merits or otherwise of the jury system in the legal process of the courts. Each is surprisingly lacking in evidence upon which to make clear decisions, but each is concerned with systems for judging or rating the evidence, each has a history—and almost certainly a future—of policy crafted by values and opinion. The strongest critics and the strongest defenders are often one and the same. Part of the problem may be that we expect too much of both systems, more than they can ever truly deliver. We therefore create an inevitable expectations/reality gap. One clear example is that of detecting scientific misconduct—something that peer review is so ill equipped to do that we should probably explicitly accept that it can't.
Furthermore, both processes—the jury system and peer review—use untrained participants. However, they do differ at least in the fact that those who undertake peer review in the health sciences have experienced review of their own work, while the experience of jurors is likely to be largely through media representation of the legal process (indeed, many who have direct experience are actively excluded!).
I would not begin to claim to know how the jury system is evaluated, but the peer review of publications and grant applications is now developing a more structured and formalised body of research activity to supplement the vast experience of journal editors and grant giving bodies.
This ambitious book has three sections. The first seeks to review the literature and the evidence base for peer review. This is both a strength and a weakness; a strength because it brings together what was known in 1999 in great detail and with intelligent and perceptive discussion, and a weakness because this is a rapidly changing field and hence this section of the book is already out of date. Nonetheless, as a repository of extensive review, this is unrivalled.
The second section covers “how to do it” and the sound advice and discussion here will be of value to all involved in the process (both practitioners and recipients). It used to be said that clinical skills were learned as “see one, do one, teach one”. For those beginning peer review their “see one” experience is predominantly based upon receipt of the peer review of their own papers or grant submissions—the outcome of the peer review process. As an introduction this is analogous to seeing a healthy patient with an appendicectomy scar and then performing an appendicectomy. And who is teaching our peer reviewers?
The final section includes the inevitable crystal ball gazing and addresses the potential of Internet based peer review, peer review anchored within systematic reviews of the literature and hence addressing the additional contribution that the paper makes to the knowledge base, and a concluding reflective piece from Richard Smith who has been such a prime mover in this field. Richard's conclusions are interesting and he notes that “It is the lack of change, rather than the rapidity of change (in peer review), that is remarkable”. This in itself may well reflect the absence of preferred alternatives. Whether the Internet and electronic publishing will change all this remains to be seen; as Richard Smith states: “The future is unknown and unknowable”. I suspect peer review will continue to buck the trend and that change will continue to be slow.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.