Scholarly peer review information

This article's lead section may be too long. Please read the length guidelines and help move details into the article's body. (May 2023)

Scholarly peer review or academic peer review (also known as refereeing) is the process of having a draft version of a researcher's methods and findings reviewed (usually anonymously) by experts (or "peers") in the same field. Peer review is widely used for helping the academic publisher (that is, the editor-in-chief, the editorial board or the program committee) decide whether the work should be accepted, considered acceptable with revisions, or rejected for official publication in an academic journal, a monograph or in the proceedings of an academic conference. If the identities of authors are not revealed to each other, the procedure is called dual-anonymous peer review.

Academic peer review requires a community of experts in a given (and often narrowly defined) academic field, who are qualified and able to perform reasonably impartial review. Impartial review, especially of work in less narrowly defined or inter-disciplinary fields, may be difficult to accomplish, and the significance (good or bad) of an idea may never be widely appreciated among its contemporaries. Peer review is generally considered necessary to academic quality and is used in most major scholarly journals. However, peer review does not prevent publication of invalid research,^[1] and as experimentally controlled studies of this process are difficult to arrange, direct evidence that peer review improves the quality of published papers is scarce.^[2]

Scholarly peer review has been subject to several criticisms, and various proposals for reforming the system have been suggested over the years. Many studies have emphasized the problems inherent to the process of peer review.^[3] Moreover, Ragone et al.,^[4] have shown that there is a low correlation between peer review outcomes and the future impact measured by citations. Brezis and Birukou also show that the peer review process is not working properly. They underline that the ratings are not robust, e.g., changing reviewers can have a dramatic impact on the review results. Two main elements affect the bias in the peer process:^[5]

• The first element is that referees display homophily in their taste and perception of innovative ideas. So reviewers who are developing conventional ideas will tend to give low grades to innovative projects, while reviewers who have developed innovative ideas tend, by homophily, to give higher grades to innovative projects.
• The second element leading to a high variance in the peer review process is that reviewers are not investing the same amount of time to analyze the projects (or equivalently are not with the same abilities). Brezis and Biruku^[5] show that this heterogeneity among referees will lead to seriously affect the whole peer review process, and will lead to main arbitrariness in the results of the process.^[5]

The peer process is also in use for projects acceptance. (For projects, the acceptance rates are small and are between 1% and 20%, with an average of 10%. In the European H2020 calls, the acceptance rate is 1.8%.) Peer review is more problematic when choosing the projects to be funded since innovative projects are not highly ranked in the existing peer-review process. The peer-review process leads to conformity, i.e., the selection of less controversial projects and papers. This may even influence the type of proposals scholars will propose, since scholars need to find financing for their research as discussed by Martin, 1997:^[6] "A common informal view is that it is easier to obtain funds for conventional projects. Those who are eager to get funding are not likely to propose radical or unorthodox projects. Since you don't know who the referees are going to be, it is best to assume that they are middle-of-the-road. Therefore, the middle-of-the-road application is safer".^[5]

Other attempts to reform the peer review process originate among others from the fields of metascience and journalology. Reformers seek to increase the reliability and efficiency of the peer review process and to provide it with a scientific foundation.^[7][8][9] Alternatives to common peer review practices have been put to the test,^[10][11] in particular open peer review, where the comments are visible to readers, generally with the identities of the peer reviewers disclosed as well, e.g., F1000, eLife, BMJ, and BioMed Central.^[12] In the case of eLife, peer review is used not for deciding whether to publish an article, but for assessing its importance and reliability.^[13] Likewise, the recognition and recruitment of peer reviewers continues to be a significant issue in the field of scholarly publishing.^[14]