Why do we do peer review?

Peer review is considered the backbone of scientific publishing. However, sometimes even the most seasoned reviewers forget what peer review is all about - ensuring the validity, quality, and integrity of scientific work. Interestingly enough, systematic peer review on every piece of scientific work hasn't been around for that long! It has been used routinely only since the 1970s as a result of increased public funding for scientific research and the accountability for this funding demanded by the public and legislators (Baldwin, 2018). For those curious to learn more, we strongly recommend reading this insightful interview with Melinda Baldwin on the history of peer review.

Who does peer review?

Not all researchers are involved in the peer review process equally. There is a strong imbalance in who does peer review, with research suggesting that 20% of researchers perform 69-94% of all peer reviews (Kovanis et al., 2016). Historically, senior scientists, male scientists (Fox et al., 2016), and those from developed regions (e.g. USA, UK, Japan; Ahmad, 2018) did the majority of peer reviews.

PhD candidates and early-career postdocs were not often approached by journals for peer review. However, the recent boom in manuscript submissions across all disciplines has led editors to consider junior scientists as peer reviewers too. Yet, they rarely get training on how to do a good peer review. Some organizations offer peer-review-mentorship which trains young researchers in how to do peer review, but unfortunately, this is the exception and not the norm. Paper-Wizard can help address this training gap by giving researchers structured, detailed feedback on their own manuscripts. Experiencing what thorough peer review looks like on your work teaches you what to look for when reviewing others papers.

What are some issues with peer review?

Beyond the shortage of peer reviewers that has recently faced the academic world, the peer review system experiences many other issues including those of quality, bias, and unpaid labour.

The quality of peer review varies widely across disciplines and within journals. A meta-analysis of inter-reviewer reliability found that agreement between reviewers on the same manuscript is often no better than chance (Bornmann et al., 2010), which raises questions about consistency. Bias is another issue that can occur throughout peer review. A recent study (Barnett, 2025) found that reviewers who asked authors to cite the reviewer's own articles were less likely to recommend acceptance. However, if the author followed up with this recommendation, the reviewer was more likely to recommend acceptance of the revised article.

Here, we are going to focus on how to carry out an effective peer review - one that improves an academic manuscript, benefits the editor and authors, and pushes human knowledge forward.

What makes a good peer review?

Studies examining thousands of reviews have found that the most valuable ones possess these 7 key characteristics: specific, constructive, courteous, consistent, objective, organized, and rigorous (Superchi et al., 2019; Sizo et al., 2025). Reviews that simply say "this needs more work" or "the methods are flawed" without explaining how or why rank amongst the least useful to both authors and editors. According to Benos et al. (2003), peer reviewers also need to check:

Quality: there are no procedural or logical mistakes,
Support for conclusions: the results support the conclusions made,
No citation errors are present,
Studies including humans and non-human animals have followed ethical standards,
Originality and significance of the work.

How to structure your peer review report and what to consider when preparing it.

Your report should be organized to benefit both the author and the editor. A commonly used framing is the Summary - Major Comments - Minor Comments structure (Sedaghat et al., 2024), which should include:

Summary of the work
- Objectives, study design, and key findings
- Interpretations/Conclusions
- Contributions/Significance of the work
Major Comments
- Methodological/statistical issues
- Scientific value and rigour
- Logical issues and inconsistencies
Minor comments
- Grammar, typos, syntax
- Stylistic issues with tables, figures, supplementary materials
- Other minor issues which, if resolved, can improve the paper

Other things to consider

There are some other things you should keep in mind.

Be kind and respectful.
Number your comments for ease.
Maintain confidentiality and don't share the manuscript with anyone else.
Disclose any conflict of interest.
Give concrete advice/criticism.
Inform the editor when you are unsure about specific aspects of the paper.
Meet deadlines.
Be objective, fair and unbiased.
Read the paper at least 2-3 times.
Follow the journal's instructions.

If you follow these steps, you would have helped the authors, the editors, and the progress of science. Most importantly, you would have put your critical thinking skills into practice and might have learned something new.

We've built our review process around these principles so Paper-Wizard can give you the most helpful feedback possible. The challenges we've outlined with reviewer shortages, inconsistent review quality, and the lack of training for early-career researchers, all point to a system under strain. While Paper-Wizard isn't a replacement for human peer review, it can serve as a valuable first step in the process. By providing detailed, structured feedback on your manuscript before you submit to a journal, Paper-Wizard helps you:

Catch issues early - Identify methodological gaps, logical inconsistencies, and clarity problems before they reach reviewers
Reduce reviewer burden - A more polished submission means reviewers can focus on substantive scientific questions rather than basic errors
Build your reviewing instincts - For early-career researchers who rarely receive formal peer review training, seeing structured feedback on your own work can help you understand what reviewers look for

Think of it as a rigorous self-check that applies the same principles of good peer review - specific, constructive, and organised - to help you submit your best work. If you want to see an example, try Paper-Wizard to get expert review of your manuscript.

References

Ahmad, Junaid. (2018). Publons-Global-State-Of-Peer-Review-2018.

Baldwin, M. (2018). Scientific Autonomy, Public Accountability, and the Rise of "Peer Review" in the Cold War United States. Isis, 109(3), 538–558. https://doi.org/10.1086/700070

Barnett, Adrian (2025) Are peer reviewers influenced by their work being cited? eLife 14:RP108748 https://doi.org/10.7554/eLife.108748.3

Benos, D. J., Kirk, K. L., & Hall, J. E. (2003). HOW TO REVIEW A PAPER. Advances in Physiology Education, 27(2), 47–52. https://doi.org/10.1152/advan.00057.2002

Bornmann, L., Mutz, R., & Daniel, H.-D. (2010). A Reliability-Generalization Study of Journal Peer Reviews: A Multilevel Meta-Analysis of Inter-Rater Reliability and Its Determinants. PLoS ONE, 5(12), e14331. https://doi.org/10.1371/journal.pone.0014331

Kovanis, M., Porcher, R., Ravaud, P., & Trinquart, L. (2016). The Global Burden of Journal Peer Review in the Biomedical Literature: Strong Imbalance in the Collective Enterprise. PLOS ONE, 11(11), e0166387. https://doi.org/10.1371/journal.pone.0166387

Sedaghat, A. R., Bernal-Sprekelsen, M., Fokkens, W. J., Smith, T. L., Stewart, M. G., & Johnson, R. F. (2024). How to be a good reviewer: A step-by-step guide for approaching peer review of a scientific manuscript. Laryngoscope investigative otolaryngology, 9(3), e1266. https://doi.org/10.1002/lio2.1266

Sizo, A., Lino, A., Rocha, Á., & Reis, L. P. (2025). Defining quality in peer review reports: a scoping review. Knowledge and Information Systems. https://doi.org/10.1007/s10115-025-02435-0

Superchi, C., González, J. A., Solà, I., Cobo, E., Hren, D., & Boutron, I. (2019). Tools used to assess the quality of peer review reports: a methodological systematic review. BMC Medical Research Methodology, 19(1). https://doi.org/10.1186/s12874-019-0688-x

How to do a good peer review?