Peer review has been part of the groundwork to scientific study for over 300 years, but scientists are starting to call into question the reliability of this once untouchable institution.
Peer review exists as a way to assess scientific integrity of other people’s work. When a scientific study is to be published, the journal it is published in sends it to be looked over by fellow scientists. Their job, to scrutinise and pick through it, ensuring there are no inaccurate conclusions, unfounded claims or illogical leaps. This is a long process and can take up to a year sometimes. The purpose of it is to maintain the integrity of science, ensuring all additions stand up to scrutiny.
There are several problems with the current process, however, and no clear way to fix them.
Firstly, there is actually very little evidence peer review works to improve validity or scientific quality. A 2002 study concluded that its effects are ‘uncertain’ and ‘untested’.
Furthermore, it can be an extremely long and drawn out process, taking an average of 17 weeks. Its also quite costly, with estimates of up to $250 (roughly £200) per submitted manuscript.
Peer review can be open to all kinds of bias and discrimination too. A paper submitted in 1982 by Douglas Peters and Stephen Ceci found one of the clearest examples of this. They took 12 published studies that came from well-known and respected institutions, changed the author names and institutions to made-up ones, then sent them back to the journals. Nine out of twelve of them were rejected for ‘poor quality’. They concluded that this was bias against authors from less prestigious institutions.
Women, some argue, are most unfairly treated by the peer review process. Cognitive bias against women in science is well-documented; generally they must demonstrate a higher level of competence and skill to achieve the same level of recognition as their male counterparts. It’s also been demonstrably proven that female scientists have a lower success rate when applying for grants. Therefore, it’s not a leap to suggest the peer review system is open to the same kind of discrimination, especially since the most common age group of reviewers (40-65 year-olds), are the most likely to hold an in-built association between the male gender and science.
Any such prejudices can influence the peer review process, depending on who’s doing the reviewing. Academic status, ethnic background and institution prestige are all examples of things that could play into someone’s implicit biases. Additionally, because reviewers are usually left unnamed, there is little accountability if an unfair review is published.
This, compounded with the secrecy many journals surrounded the process with, has driven calls to alter the process to ensure it’s fairer.
Journals already have a few practices to try to reduce this kind of bias. In a double-blind review, the author of the paper is anonymous to the reviewer. The idea is that this limits the reviewer’s in-built biases from having too much of an effect. However, there is no guarantee of that. Things like writing style or specialist equipment can give away an author’s identity: complete anonymity is never guaranteed.
In response to this, the practice of open reviewing has been gaining traction. This is where both the reviewers and authors know each other’s identities. Sometimes, the full peer review report is published, alongside the editor’s responses. There is a strong belief that this encourages healthy scrutiny and increases accountability in reviews. It also shines a light onto the best and most in-depth reviews, giving recognition to those who have dedicated the time to helping improve other people’s work.
After their two-year pilot, Elsevier – a publishing company – found that 70% of editors said open peer review resulted in reports that are ‘more in depth and constructive for authors to improve the quality of their manuscript’.
Many prestigious journals are beginning to pilot and adopt a more transparent peer-review process. In 2019, Nature Human Behaviour announced that it’s giving all of its authors the chance to publish their peer-review comments, alongside their research articles.
Some journals have also taken to updating authors at every step of the peer-review process as another way to make peer review more transparent. Alerting authors as a paper passes certain ‘checkpoints’ goes a long way to improving trust in the process.
Changing the methods behind a staple of science like peer-review is a bold step, but many deem it necessary for modern science. The open access pilot schemes are beginning to be scaled up and implemented across entire journals.
By lifting the veil on the publication process, not only can the quality of scientific journals be improved, but public trust in science as well.