what is peer review process

• Science Notes Posts
• Contact Science Notes
• Todd Helmenstine Biography
• Anne Helmenstine Biography
• Free Printable Periodic Tables (PDF and PNG)
• Periodic Table Wallpapers
• Interactive Periodic Table
• Periodic Table Posters
• How to Grow Crystals
• Chemistry Projects
• Fire and Flames Projects
• Holiday Science
• Chemistry Problems With Answers
• Physics Problems
• Unit Conversion Example Problems
• Chemistry Worksheets
• Biology Worksheets
• Periodic Table Worksheets
• Physical Science Worksheets
• Science Lab Worksheets
• My Amazon Books

Understanding Peer Review in Science

Peer review is an essential element of the scientific publishing process that helps ensure that research articles are evaluated, critiqued, and improved before release into the academic community. Take a look at the significance of peer review in scientific publications, the typical steps of the process, and and how to approach peer review if you are asked to assess a manuscript.

What Is Peer Review?

Peer review is the evaluation of work by peers, who are people with comparable experience and competency. Peers assess each others’ work in educational settings, in professional settings, and in the publishing world. The goal of peer review is improving quality, defining and maintaining standards, and helping people learn from one another.

In the context of scientific publication, peer review helps editors determine which submissions merit publication and improves the quality of manuscripts prior to their final release.

Types of Peer Review for Manuscripts

There are three main types of peer review:

• Single-blind review: The reviewers know the identities of the authors, but the authors do not know the identities of the reviewers.
• Double-blind review: Both the authors and reviewers remain anonymous to each other.
• Open peer review: The identities of both the authors and reviewers are disclosed, promoting transparency and collaboration.

There are advantages and disadvantages of each method. Anonymous reviews reduce bias but reduce collaboration, while open reviews are more transparent, but increase bias.

Key Elements of Peer Review

Proper selection of a peer group improves the outcome of the process:

• Expertise : Reviewers should possess adequate knowledge and experience in the relevant field to provide constructive feedback.
• Objectivity : Reviewers assess the manuscript impartially and without personal bias.
• Confidentiality : The peer review process maintains confidentiality to protect intellectual property and encourage honest feedback.
• Timeliness : Reviewers provide feedback within a reasonable timeframe to ensure timely publication.

Steps of the Peer Review Process

The typical peer review process for scientific publications involves the following steps:

• Submission : Authors submit their manuscript to a journal that aligns with their research topic.
• Editorial assessment : The journal editor examines the manuscript and determines whether or not it is suitable for publication. If it is not, the manuscript is rejected.
• Peer review : If it is suitable, the editor sends the article to peer reviewers who are experts in the relevant field.
• Reviewer feedback : Reviewers provide feedback, critique, and suggestions for improvement.
• Revision and resubmission : Authors address the feedback and make necessary revisions before resubmitting the manuscript.
• Final decision : The editor makes a final decision on whether to accept or reject the manuscript based on the revised version and reviewer comments.
• Publication : If accepted, the manuscript undergoes copyediting and formatting before being published in the journal.

Pros and Cons

While the goal of peer review is improving the quality of published research, the process isn’t without its drawbacks.

• Quality assurance : Peer review helps ensure the quality and reliability of published research.
• Error detection : The process identifies errors and flaws that the authors may have overlooked.
• Credibility : The scientific community generally considers peer-reviewed articles to be more credible.
• Professional development : Reviewers can learn from the work of others and enhance their own knowledge and understanding.
• Time-consuming : The peer review process can be lengthy, delaying the publication of potentially valuable research.
• Bias : Personal biases of reviews impact their evaluation of the manuscript.
• Inconsistency : Different reviewers may provide conflicting feedback, making it challenging for authors to address all concerns.
• Limited effectiveness : Peer review does not always detect significant errors or misconduct.
• Poaching : Some reviewers take an idea from a submission and gain publication before the authors of the original research.

Steps for Conducting Peer Review of an Article

Generally, an editor provides guidance when you are asked to provide peer review of a manuscript. Here are typical steps of the process.

• Accept the right assignment: Accept invitations to review articles that align with your area of expertise to ensure you can provide well-informed feedback.
• Manage your time: Allocate sufficient time to thoroughly read and evaluate the manuscript, while adhering to the journal’s deadline for providing feedback.
• Read the manuscript multiple times: First, read the manuscript for an overall understanding of the research. Then, read it more closely to assess the details, methodology, results, and conclusions.
• Evaluate the structure and organization: Check if the manuscript follows the journal’s guidelines and is structured logically, with clear headings, subheadings, and a coherent flow of information.
• Assess the quality of the research: Evaluate the research question, study design, methodology, data collection, analysis, and interpretation. Consider whether the methods are appropriate, the results are valid, and the conclusions are supported by the data.
• Examine the originality and relevance: Determine if the research offers new insights, builds on existing knowledge, and is relevant to the field.
• Check for clarity and consistency: Review the manuscript for clarity of writing, consistent terminology, and proper formatting of figures, tables, and references.
• Identify ethical issues: Look for potential ethical concerns, such as plagiarism, data fabrication, or conflicts of interest.
• Provide constructive feedback: Offer specific, actionable, and objective suggestions for improvement, highlighting both the strengths and weaknesses of the manuscript. Don’t be mean.
• Organize your review: Structure your review with an overview of your evaluation, followed by detailed comments and suggestions organized by section (e.g., introduction, methods, results, discussion, and conclusion).
• Be professional and respectful: Maintain a respectful tone in your feedback, avoiding personal criticism or derogatory language.
• Proofread your review: Before submitting your review, proofread it for typos, grammar, and clarity.
• Couzin-Frankel J (September 2013). “Biomedical publishing. Secretive and subjective, peer review proves resistant to study”. Science . 341 (6152): 1331. doi: 10.1126/science.341.6152.1331
• Lee, Carole J.; Sugimoto, Cassidy R.; Zhang, Guo; Cronin, Blaise (2013). “Bias in peer review”. Journal of the American Society for Information Science and Technology. 64 (1): 2–17. doi: 10.1002/asi.22784
• Slavov, Nikolai (2015). “Making the most of peer review”. eLife . 4: e12708. doi: 10.7554/eLife.12708
• Spier, Ray (2002). “The history of the peer-review process”. Trends in Biotechnology . 20 (8): 357–8. doi: 10.1016/S0167-7799(02)01985-6
• Squazzoni, Flaminio; Brezis, Elise; Marušić, Ana (2017). “Scientometrics of peer review”. Scientometrics . 113 (1): 501–502. doi: 10.1007/s11192-017-2518-4

Related Posts

• Technical Support
• Find My Rep

You are here

What is peer review.

Peer review is ‘a process where scientists (“peers”) evaluate the quality of other scientists’ work. By doing this, they aim to ensure the work is rigorous, coherent, uses past research and adds to what we already know.’ You can learn more in this explainer from the Social Science Space.  

Peer review brings academic research to publication in the following ways:

• Evaluation – Peer review is an effective form of research evaluation to help select the highest quality articles for publication.
• Integrity – Peer review ensures the integrity of the publishing process and the scholarly record. Reviewers are independent of journal publications and the research being conducted.
• Quality – The filtering process and revision advice improve the quality of the final research article as well as offering the author new insights into their research methods and the results that they have compiled. Peer review gives authors access to the opinions of experts in the field who can provide support and insight.

Types of peer review

• Single-anonymized  – the name of the reviewer is hidden from the author.
• Double-anonymized  – names are hidden from both reviewers and the authors.
• Triple-anonymized  – names are hidden from authors, reviewers, and the editor.
• Open peer review comes in many forms . At Sage we offer a form of open peer review on some journals via our Transparent Peer Review program , whereby the reviews are published alongside the article. The names of the reviewers may also be published, depending on the reviewers’ preference.
• Post publication peer review can offer useful interaction and a discussion forum for the research community. This form of peer review is not usual or appropriate in all fields.

To learn more about the different types of peer review, see page 14 of ‘ The Nuts and Bolts of Peer Review ’ from Sense about Science.

Please double check the manuscript submission guidelines of the journal you are reviewing in order to ensure that you understand the method of peer review being used.

• Journal Author Gateway
• Journal Editor Gateway
• Transparent Peer Review
• How to Review Articles
• Using Sage Track
• Peer Review Ethics
• Resources for Reviewers
• Reviewer Rewards
• Ethics & Responsibility
• Sage Editorial Policies
• Publication Ethics Policies
• Sage Chinese Author Gateway 中国作者资源
• Open Resources & Current Initiatives
• Discipline Hubs

Peer review process

Introduction to peer review, what is peer review.

Peer review is the system used to assess the quality of a manuscript before it is published. Independent researchers in the relevant research area assess submitted manuscripts for originality, validity and significance to help editors determine whether a manuscript should be published in their journal.

How does it work?

When a manuscript is submitted to a journal, it is assessed to see if it meets the criteria for submission. If it does, the editorial team will select potential peer reviewers within the field of research to peer-review the manuscript and make recommendations.

There are four main types of peer review used by BMC:

Single-blind: the reviewers know the names of the authors, but the authors do not know who reviewed their manuscript unless the reviewer chooses to sign their report.

Double-blind: the reviewers do not know the names of the authors, and the authors do not know who reviewed their manuscript.

Open peer: authors know who the reviewers are, and the reviewers know who the authors are. If the manuscript is accepted, the named reviewer reports are published alongside the article and the authors’ response to the reviewer.

Transparent peer: the reviewers know the names of the authors, but the authors do not know who reviewed their manuscript unless the reviewer chooses to sign their report. If the manuscript is accepted, the anonymous reviewer reports are published alongside the article and the authors’ response to the reviewer.

Different journals use different types of peer review. You can find out which peer-review system is used by a particular journal in the journal’s ‘About’ page.

Why do peer review?

Peer review is an integral part of scientific publishing that confirms the validity of the manuscript. Peer reviewers are experts who volunteer their time to help improve the manuscripts they review. By undergoing peer review, manuscripts should become:

More robust - peer reviewers may point out gaps in a paper that require more explanation or additional experiments.

Easier to read - if parts of your paper are difficult to understand, reviewers can suggest changes.

More useful - peer reviewers also consider the importance of your paper to others in your field.

For more information and advice on how to get published, please see our blog series here .

How peer review works

The peer review process can be single-blind, double-blind, open or transparent.

You can find out which peer review system is used by a particular journal in the journal's 'About' page.

N. B. This diagram is a representation of the peer review process, and should not be taken as the definitive approach used by every journal.

What is peer review?

From a publisher’s perspective, peer review functions as a filter for content, directing better quality articles to better quality journals and so creating journal brands.

Running articles through the process of peer review adds value to them. For this reason publishers need to make sure that peer review is robust.

Editor Feedback

"Pointing out the specifics about flaws in the paper’s structure is paramount. Are methods valid, is data clearly presented, and are conclusions supported by data?” (Editor feedback)

“If an editor can read your comments and understand clearly the basis for your recommendation, then you have written a helpful review.” (Editor feedback)

Peer Review at Its Best

What peer review does best is improve the quality of published papers by motivating authors to submit good quality work – and helping to improve that work through the peer review process. 

In fact, 90% of researchers feel that peer review improves the quality of their published paper (University of Tennessee and CIBER Research Ltd, 2013).

What the Critics Say

The peer review system is not without criticism. Studies show that even after peer review, some articles still contain inaccuracies and demonstrate that most rejected papers will go on to be published somewhere else.

However, these criticisms should be understood within the context of peer review as a human activity. The occasional errors of peer review are not reasons for abandoning the process altogether – the mistakes would be worse without it.

Improving Effectiveness

Some of the ways in which Wiley is seeking to improve the efficiency of the process, include:

• Reducing the amount of repeat reviewing by innovating around transferable peer review
• Providing training and best practice guidance to peer reviewers
• Improving recognition of the contribution made by reviewers

Visit our Peer Review Process and Types of Peer Review pages for additional detailed information on peer review.

Transparency in Peer Review

Wiley is committed to increasing transparency in peer review, increasing accountability for the peer review process and giving recognition to the work of peer reviewers and editors. We are also actively exploring other peer review models to give researchers the options that suit them and their communities.

Explainer: what is peer review?

Professor of Organisational Behaviour, Cass Business School, City, University of London

Novak Druce Research Fellow, University of Oxford

Disclosure statement

Thomas Roulet does not work for, consult to, own shares in or receive funding from any company or organisation that would benefit from this article, and has no relevant affiliations.

Andre Spicer does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.

City, University of London provides funding as a founding partner of The Conversation UK.

University of Oxford provides funding as a member of The Conversation UK.

View all partners

We’ve all heard the phrase “peer review” as giving credence to research and scholarly papers, but what does it actually mean? How does it work?

Peer review is one of the gold standards of science. It’s a process where scientists (“peers”) evaluate the quality of other scientists’ work. By doing this, they aim to ensure the work is rigorous, coherent, uses past research and adds to what we already knew.

Most scientific journals, conferences and grant applications have some sort of peer review system. In most cases it is “double blind” peer review. This means evaluators do not know the author(s), and the author(s) do not know the identity of the evaluators. The intention behind this system is to ensure evaluation is not biased.

The more prestigious the journal, conference, or grant, the more demanding will be the review process, and the more likely the rejection. This prestige is why these papers tend to be more read and more cited.

The process in details

The peer review process for journals involves at least three stages.

1. The desk evaluation stage

When a paper is submitted to a journal, it receives an initial evaluation by the chief editor, or an associate editor with relevant expertise.

At this stage, either can “desk reject” the paper: that is, reject the paper without sending it to blind referees. Generally, papers are desk rejected if the paper doesn’t fit the scope of the journal or there is a fundamental flaw which makes it unfit for publication.

In this case, the rejecting editors might write a letter summarising his or her concerns. Some journals, such as the British Medical Journal , desk reject up to two-thirds or more of the papers.

2. The blind review

If the editorial team judges there are no fundamental flaws, they send it for review to blind referees. The number of reviewers depends on the field: in finance there might be only one reviewer, while journals in other fields of social sciences might ask up to four reviewers. Those reviewers are selected by the editor on the basis of their expert knowledge and their absence of a link with the authors.

Reviewers will decide whether to reject the paper, to accept it as it is (which rarely happens) or to ask for the paper to be revised. This means the author needs to change the paper in line with the reviewers’ concerns.

Usually the reviews deal with the validity and rigour of the empirical method, and the importance and originality of the findings (what is called the “contribution” to the existing literature). The editor collects those comments, weights them, takes a decision, and writes a letter summarising the reviewers’ and his or her own concerns.

It can therefore happen that despite hostility on the part of the reviewers, the editor could offer the paper a subsequent round of revision. In the best journals in the social sciences, 10% to 20% of the papers are offered a “revise-and-resubmit” after the first round.

3. The revisions – if you are lucky enough

If the paper has not been rejected after this first round of review, it is sent back to the author(s) for a revision. The process is repeated as many times as necessary for the editor to reach a consensus point on whether to accept or reject the paper. In some cases this can last for several years.

Ultimately, less than 10% of the submitted papers are accepted in the best journals in the social sciences. The renowned journal Nature publishes around 7% of the submitted papers.

Strengths and weaknesses of the peer review process

The peer review process is seen as the gold standard in science because it ensures the rigour, novelty, and consistency of academic outputs. Typically, through rounds of review, flawed ideas are eliminated and good ideas are strengthened and improved. Peer reviewing also ensures that science is relatively independent.

Because scientific ideas are judged by other scientists, the crucial yardstick is scientific standards. If other people from outside of the field were involved in judging ideas, other criteria such as political or economic gain might be used to select ideas. Peer reviewing is also seen as a crucial way of removing personalities and bias from the process of judging knowledge.

Despite the undoubted strengths, the peer review process as we know it has been criticised . It involves a number of social interactions that might create biases – for example, authors might be identified by reviewers if they are in the same field, and desk rejections are not blind.

It might also favour incremental (adding to past research) rather than innovative (new) research. Finally, reviewers are human after all and can make mistakes, misunderstand elements, or miss errors.

Are there any alternatives?

Defenders of the peer review system say although there are flaws, we’re yet to find a better system to evaluate research. However, a number of innovations have been introduced in the academic review system to improve its objectivity and efficiency.

Some new open-access journals (such as PLOS ONE ) publish papers with very little evaluation (they check the work is not deeply flawed methodologically). The focus there is on the post-publication peer review system: all readers can comment and criticise the paper.

Some journals such as Nature, have made part of the review process public (“open” review), offering a hybrid system in which peer review plays a role of primary gate keepers, but the public community of scholars judge in parallel (or afterwards in some other journals) the value of the research.

Another idea is to have a set of reviewers rating the paper each time it is revised. In this case, authors will be able to choose whether they want to invest more time in a revision to obtain a better rating, and get their work publicly recognised.

• Peer review

Events and Communications Coordinator

Assistant Editor - 1 year cadetship

Executive Dean, Faculty of Health

Lecturer/Senior Lecturer, Earth System Science (School of Science)

Sydney Horizon Educators (Identified)

• SpringerLink shop

Peer-review process

Peer review exists to ensure that journals publish good science which is of benefit to entire scientific community.

Sometimes authors find the peer-review process intimidating because it can lead to the rejection of their manuscript. Keep in mind that revisions and improvement are part of the publication process and actually help raise the quality of your manuscript.

Peer review is a positive process

Peer review is an integral part of scientific publishing that confirms the validity of the science reported. Peer reviewers are experts who volunteer their time to help improve the journal manuscripts they review—they offer authors free advice .

Through the peer-review process, manuscripts should become:

• More robust : Peer reviewers may point out gaps in your paper that require more explanation or additional experiments.
• Easier to read : If parts of your paper are difficult to understand, reviewers can tell you so that you can fix them. After all, if an expert cannot understand what you have done, it is unlikely that a reader in a different field will understand.
• More useful : Peer reviewers also consider the importance of your paper to others in your field and can make suggestions to improve or better highlight this to readers.

Of course, in addition to offering authors advice, another important purpose of peer review is to make sure that the manuscripts published in the journal are of the correct quality for the journal’s aims.

Different types of peer review

There are different forms of peer review used by journals, although the basis is always the same, field experts providing comments on a paper to help improve it. The most common types are

Closed – where the reviewers are aware of the authors’ identities but the authors do not know who reviewed their manuscript.

Double blind – in this case neither authors nor reviewers know each other’s identities.

Open – where there reviewers are aware of the authors’ identity and the reviewers’ identity is revealed to the authors. In some cases journals also publish the reviewers’ reports alongside the final published manuscript.

Back │ Next

Peer review process

Introduction to peer review.

Peer review exists to ensure that journals publish good science. This benefits the entire scientific community.

Sometimes scientists find the peer review process intimidating because it can lead to the rejection of their manuscript. Keep in mind that revisions and improvement are part of the publication process and actually help raise the quality of your manuscript.

Peer review is a positive process

Peer review is an integral part of scientific publishing that confirms the validity of the science reported. Peer reviewers are experts who volunteer their time to help improve the journal manuscripts they review-they offer authors free advice.

Through the peer review process, manuscripts should become:

• More robust: Peer reviewers may point out gaps in your paper that require more explanation or additional experiments.
• Easier to read: If parts of your paper are difficult to understand, reviewers can tell you so that you can fix them.
• More useful: Peer reviewers also consider the importance of your paper to others in your field.

Of course, in addition to offering authors advice, another important purpose of peer review is to make sure that the manuscripts the journal eventually publishes are of high quality. If a journal publishes too many low-quality manuscripts, its reputation and number of readers will decline.

Editorial rejection

Your journal manuscript can be rejected if it:

• Lacks proper structureLacks the necessary detail for readers to fully understand the authors' analysis
• Has no new science
• Does not clearly explain which parts of the findings are new science, versus what was already known
• Lacks up-to-date references
• Contains theories, concepts, or conclusions that are not fully supported by its data, arguments, and information
• Does not provide enough details about materials and methods to allow other scientists to repeat the experiment

  - Hypotheses tested   - The experimental design   - Sample characteristics and descriptive statistics

• Has a weak study design or faulty statistical analysis

Has poor language quality

Publication is a difficult process, and you must be prepared to defend your submission against rejection from both editors and peer reviewers. However, do not be too persistent. Generally, only one letter defending your submission will be accepted for each of the review stages (editorial review and peer review). If you are unsuccessful after sending a response letter, then you should strongly consider selecting another journal .

When revising your manuscript and responding to peer review comments:

• Address all points raised by the editor and reviewers
• Describe the revisions to your manuscript in your response letter
• Perform any additional experiments or analyses the reviewers recommend (unless you feel that they would not make your paper better; if this is the case, explain why in your response letter)
• Provide a polite and scientific rebuttal to any points or comments you disagree with
• Differentiate between reviewer comments and your responses in your letter
• Clearly show the major revisions in the text, either with a different color text, by highlighting the changes, or with Microsoft Word's Track Changes feature
• Return the revised manuscript and response letter within the time period the editor tells you

Reviewer comment: "In your analysis of the data you have chosen to use a somewhat obscure fitting function (regression). In my opinion, a simple Gaussian function would have sufficed. Moreover, the results would be more instructive and easier to compare to previous results."

Response in agreement with the reviewer: "We agree with the reviewer's assessment of the analysis. Our tailored function does make it impossible to fully interpret the data in terms of the prevailing theories. In addition, in its current form, we agree it would be difficult to tell that this measurement constitutes a significant improvement over previously reported values. We have therefore re-analyzed the data using a Gaussian fitting function."

Response disagreeing with the reviewer: "We agree with the reviewer that a simple Gaussian fit would facilitate comparison with the results of other studies. However, our tailored function allows for the analysis of the data in terms of the Smith model [Smith et al, 1998]. We have added two sentences to the paper (page 3, paragraph 2) to explain the use of this function and Smith's model."

Note that in both comments (agreeing and disagreeing) the author is polite and shows respect for the reviewer's opinion. Also, in both circumstances the author makes a change to the manuscript that addresses the reviewer's question.

Remember, the reviewer is probably a highly knowledgeable person. If their suggestion is incorrect, it is likely because they misunderstood your manuscript, indicating that you should make your text clearer.

Use the Response letter template from Edanz to make writing your reply easier.

This site uses cookies to store information on your computer. Some are essential to make our site work; others help us improve the user experience. By using the site, you consent to the placement of these cookies. Read our  privacy policy  to learn more.

Peer Review Summary

Summary of the Nature, Objectives, Scope, Limitations of, and Procedures Performed in System and Engagement Reviews (as Referred to in a Peer Review Report)

      1. Firms (and individuals) enrolled in the AICPA Peer Review Program (the program) are required to have a peer review, of their accounting and auditing practice once every three years according to the AICPA Standards for Performing and Reporting on Peer Reviews, Effective for Peer Reviews Commencing on or After May 1, 2022 (the standards). An accounting and auditing practice is defined by the standards as a practice that performs engagements under Statements on Auditing Standards (SASs), Statements on Standards for Accounting and Review Services (SSARSs), Statements on Standards for Attestation Engagements (SSAEs),  Government Auditing Standards (the Yellow Book) issued by the U.S. Government Accountability Office, or Public Company Accounting Oversight Board (PCAOB) standards. Engagements covered in the scope of the program are those included in the firm’s accounting and auditing practice that are not subject to PCAOB permanent inspection. A firm is not required to enroll in the program if it only performs preparation engagements under the SSARSs, however, if it elects to enroll due to licensing or other requirements, it is required to have a peer review under the standards. The peer review is conducted by an independent evaluator, known as a peer reviewer (reviewer). The AICPA oversees the program and the peer review is administered by an entity approved by the AICPA to perform that role.

      2. The peer review helps to monitor a firm’s accounting and auditing practice (practice monitoring). The objective of practice monitoring, and the program itself, is to promote and enhance quality in the accounting and auditing services provided by firms (and individuals) subject to the standards, in service of the public interest.

      3. There are two types of peer reviews: system reviews and engagement reviews. System reviews focus on a firm’s system of quality control and engagement reviews focus on work performed on particular submitted engagements. A further description of system and engagement reviews, as well as a summary of the nature, objectives, scope, limitations of, and procedures performed on them, is provided in the following sections.

System Reviews

      4. A system review is a study and appraisal by a reviewer, of a firm’s system of quality control (system) to perform accounting and auditing work. The system represents the policies and procedures that the firm has designed and is expected to follow when performing its work. The reviewer’s objectives are to

a.  obtain reasonable assurance that the reviewed firm’s system of quality control for its accounting and auditing practice has been designed and complied with to provide the firm with reasonable assurance of performing or reporting in conformity with the requirements of applicable professional standards in all material respects, and

b.  report on the reviewed firm’s system of quality control as required by the standards.

      5. Professional standards are literature, issued by various organizations, that contain the framework and rules that a firm is expected to comply with when designing its system and when performing its work. Professional standards for design of and compliance with a system of quality control include but are not limited to quality control standards issued by the AICPA that pertain to leadership responsibilities for quality within the firm (the “tone at the top”), relevant ethical requirements (such as independence, integrity and objectivity), acceptance and continuance of client relationships and specific engagements, human resources, engagement performance, and monitoring.

      6. To plan a system review, a reviewer obtains an understanding of (1) the firm’s accounting and auditing practice, such as the industries of its clients, and (2) the design of the firm’s system of quality control, including its policies and procedures and how the firm monitors its compliance with those policies and procedures. The reviewer assesses the risk levels within different aspects of the firm’s practice and its system. The reviewer obtains this understanding through inquiry of firm personnel and review of documentation on the system, such as firm manuals.

      7. Based on the engagements firms perform, their practices may be reviewed or inspected on a periodic basis by regulatory or governmental entities, including but not limited to the Department of Health and Human Service, the Department of Labor, and the PCAOB. The reviewer obtains an understanding of those reviews or inspections, and considers the effect on the nature and extent of peer review procedures performed.

      8. Based on the reviewer’s planning procedures, a sample of the firm’s work is reviewed, individually called engagements. The reviewer selects engagements for the period covered by the peer review from a cross section of the firm’s accounting and auditing practice with emphasis on higher risk engagements. The reviewer’s selection of engagements is required to include those performed under  Government Auditing Standards , including compliance audits under the Single Audit Act, audits of employee benefit plans pursuant to the Employee Retirement Income Security Act of 1974, audits of federally insured depository institutions, and examinations of service organizations (SOC 1 ®  and SOC 2 ®  engagements) when applicable (these are known as  must select engagements ). The scope of a peer review only covers accounting and auditing engagements performed under the SASs, the SSARSs, SSAEs, Government Auditing Standards , and PCAOB standards but does not include the firm’s engagements subject to PCAOB permanent inspection, nor does it include tax or consulting services. The reviewer will also consider administrative elements of the firm’s practice to test the elements listed previously from quality control standards.

      9. The reviewer examines engagement working paper files and reports, interviews selected firm personnel, reviews representations from the firm, and examines selected administrative and personnel files. Obtaining an understanding of the system and testing it forms the basis for the reviewer’s conclusions in the peer review report.

      10. When a firm receives a report from the reviewer with a peer review rating of pass , it means that the firm’s system is appropriately designed and is being complied with to provide the firm with reasonable assurance of performing and reporting in conformity with applicable professional standards in all material respects. If a firm receives a report with a peer review rating of pass with deficiencies , this means that except for the deficiencies described, the system of quality control for the accounting and auditing practice of the reviewed firm in effect for the year ended has been suitably designed and complied with to provide the firm with reasonable assurance of performing and reporting in conformity with applicable professional standards in all material respect. When a firm receives a report with a peer review rating of fail , the reviewer has determined as a result of the significant deficiencies described, the system of quality control for the accounting and auditing practice of the reviewed firm in effect for the year ended was not suitably designed or complied with to provide the firm with reasonable assurance of performing and reporting in conformity with applicable professional standards in all material respects.

      11. If a deficiency or significant deficiency included in the peer review report is associated with an engagement that was not performed and reported on in conformity with applicable professional standards in all material respects (“nonconforming”) in a must select engagement or is industry specific, the report will identify the must-select engagement or industry. However, because the purpose of a system review is to report on the firm’s system of quality control, the peer review report may not describe every engagement that was deemed nonconforming.

      12. The firm is responsible for evaluating actions to promptly remediate nonconforming engagements when appropriate, and for remediating weaknesses in its system of quality control, if any. The firm’s response is evaluated to determine if it is appropriate, whether lack of response is indicative of other weaknesses in the firm’s system of quality control, or whether monitoring procedures are necessary to determine if the deficiencies and nonconforming engagements were remediated.

      13. There are inherent limitations in the effectiveness of any system and, therefore, noncompliance with the system may occur and not be detected. A peer review is based on selective tests, directed to assess whether the design of and compliance with the firm’s system provides the firm with reasonable, not absolute, assurance of performing and reporting in conformity with applicable professional standards in all material respects. Consequently, the peer review would not necessarily detect all weaknesses in the design of or compliance with the system. The peer review does not provide assurance with respect to any individual engagement conducted by the firm or that none of the financial statements audited by the firm should be restated. Projection of any evaluation of a system to future periods is subject to the risk that the system may become inadequate because of changes in conditions or because the degree of compliance with the policies or procedures may deteriorate.

Engagement Reviews

      14. An engagement review is a study and appraisal by a reviewer, of a sample of a firm’s accounting work. The reviewer’s objectives are to

a.  evaluate whether the engagements submitted for review are performed and reported on in conformity with applicable professional standards in all material respects and

b.  report on the evaluation of selected engagements.

      15. Engagement reviews are available to firms that do not perform audits or other similar work and perform work only under the SSARSs and the SSAEs, other than examinations as their highest levels of service.

      16. Professional standards are literature, issued by various organizations, that contain the framework and rules that a firm is expected to comply with when performing accounting work.

      17. The reviewer looks at a sample of the firm’s work, individually called engagements. The scope of an engagement review covers only accounting or attestation engagements; it does not include tax or consulting services. An engagement review consists of reading the financial statements or information submitted by the firm and the accountant’s report thereon, together with certain background information and representations from the firm and, except for certain compilation engagements, the documentation required by applicable professional standards.

      18. When the firm receives a report with a review rating of pass , the reviewer has concluded that nothing came to the reviewer’s attention that caused the reviewer to believe that the engagements submitted for review were not performed and reported on in conformity with the requirements of applicable professional standards in all material respects. A report with a peer review rating of pass with deficiencies is issued when the reviewer has concluded at least one but not all the engagements submitted for review were nonconforming. A report with a peer review rating of fail is issued when the reviewer has concluded all the engagements submitted for review were nonconforming.

      19. If a industry-specific noncompliance results in a nonconforming engagement, the deficiency in the report will identify the industry.

      20. The firm is responsible for evaluating actions to promptly remediate engagements deemed as nonconforming, when appropriate.

      21. Review of the firm’s documentation or procedures related to its system of quality control is outside the scope of an engagement review. An engagement review does not include tests of the firm’s administrative or personnel files, interviews of selected firm personnel, or other procedures performed in a system review. Therefore, an engagement review does not provide the reviewer with a basis for expressing any form of assurance on the firm’s system of quality control for its accounting practice.

We are the American Institute of CPAs, the world’s largest member association representing the accounting profession. Our history of serving the public interest stretches back to 1887. Today, you'll find our 431,000+ members in 130 countries and territories, representing many areas of practice, including business and industry, public practice, government, education and consulting.

About AICPA

• Mission and History
• Annual Reports
• AICPA Media Center
• AICPA Research
• Jobs at AICPA
• Order questions
• Forgot Password
• Store policies

Association of International Certified Professional Accountants. All rights reserved.

• Terms & Conditions

Transforming the understanding and treatment of mental illnesses.

Información en español

Celebrating 75 Years! Learn More >>

• Opportunities & Announcements
• Funding Strategy for Grants
• Grant Writing & Approval Process
• Managing Grants
• Clinical Research
• Small Business Research

Peer Review Process

Alert : For due dates on or after January 25, 2025 - changes coming to how most research grant applications will be reviewed. Learn about the new simplified review framework  .

The review process involves two levels of review.  The first level is led by the Scientific Review Officer (SRO) and is carried out by a Scientific Review Group (SRG) composed primarily of non-federal scientists who have expertise in relevant scientific disciplines and research areas. NIMH SRGs that review applications involving human subjects may also include public representatives (see What Do Public Reviewers Do? ). The second level of review is performed by an Institute’s or Center’s (IC’s) National Advisory Council or Board. Councils are composed of both scientific and public representatives chosen for their expertise, interest, or activity in matters related to health and disease. Only applications that are recommended for approval by both the SRG and the Advisory Council may be recommended for funding.  Final funding decisions are made by the IC Directors. For more information see NIH Peer Review  .

What is a Scientific Review Officer? Each Scientific Review Group (SRG) is led by a Scientific Review Officer (SRO). The SRO is an extramural staff scientist and the Designated Federal Official (DFO) responsible for ensuring that each application receives an objective and fair initial peer review, and that all applicable laws, regulations, and policies are followed. SROs have multiple responsibilities including: analyzing the scientific content of grant applications and checking for completeness, documenting and managing conflicts of interest, and identifying the scientists invited to serve on review committees based on scientific and technical qualifications and other considerations including: Authority in their scientific field Dedication to high quality, fair, and objective reviews Ability to work collegially in a group setting Experience in research grant review Balanced representation SROs are also responsible for assigning the applications to reviewers for critique preparation and assignment of scores, attending and overseeing administrative and regulatory aspects of peer review meetings, and preparing summary statements for all applications reviewed.

First Level of Review

Initial peer review of NIMH assigned applications is administered by either the Center for Scientific Review or the NIMH Review Branch. The focus of the review and review criteria are specified in the funding opportunity announcement (FOA). Peer review meetings are announced in the Federal Register  and are closed to the public.

Reviewers provide an overall impact/priority score (1-9) to reflect their assessment of the likelihood of the project to exert a sustained, powerful influence on the research field(s) involved. To this end, reviewers are asked to consider and score (1-9) the following review criteria: significance, investigator, innovation, approach, and environment. As applicable for the project proposed, reviewers evaluate the following additional items while determining scientific and technical merit and in providing an overall impact/priority score, but do not give separate scores: Protection of Human Subjects; Inclusion of Women, Minorities, and Children; Vertebrate Animals; Biohazards; Resubmission; Renewal; and Revision.

Other criteria may be considered for other grant mechanisms. Please see NIH’s webpage on the Peer Review Process  for an overview of the instructions given to reviewers for assessment of NIH grant applications.

Within a few days after the review meeting, your priority score and percentile ranking (if applicable) are available to you online via your eRA Commons account. Typically, within four to six weeks of the peer review committee meeting your summary statement will be available via your eRA account. It will include: the written critiques provided by the assigned reviewers, the SRO’s summary of the discussion, scores for each review criterion, and administrative notes of special consideration. If your application was not discussed (usually the lower half of the applications reviewed in a review meeting), you will receive the reviewer critiques and preliminary scores for each review criterion.

Applicants just receiving their scores or summary statements should consult our What’s Next  page for detailed guidance. Applicants seeking advice beyond that available online may want to contact the NIH Program Officer listed at the top of the summary statement.

What Do Public Reviewers Do? The role of the public reviewer is to bring the perspectives of individuals and family members who have been directly affected by mental illness, as well as from clinicians, caregivers, and/or policy makers to the first level of review   and to enhance the capability of the review committee to evaluate the “real world” relevance and practicality of each research application. Even though many of the scientific experts have knowledge in these areas, further emphasis on this perspective will help identify the most meritorious applications. Public reviewers are asked to evaluate issues such as the protection of human subjects, the feasibility of the proposed research and the importance or relevance of a particular application to answering questions about mental illness and understanding behavior, including: Is this area important? Do we need this information? Is this research going to improve patient care? For more information on public reviewers and the application process to become one, please visit our Public Reviewers pages .

Second Level of Review

As noted above, the Advisory Council/Board of the potential awarding IC (for NIMH, the National Advisory Mental Health Council [NAMHC] ) performs the second level of review.  Members of the NAMHC are chosen by the respective IC and are approved by Department of Health and Human Services.

• The NAMHC either concurs with or does not concur with the recommendations of the first level of review; and
• The NAMHC may make additional recommendations, particularly with regard to program priority based on the relevance of the application to the public health mission of NIMH.

Back to Step 3

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• NATURE INDEX
• 01 May 2024

Plagiarism in peer-review reports could be the ‘tip of the iceberg’

• Jackson Ryan 0

Jackson Ryan is a freelance science journalist in Sydney, Australia.

You can also search for this author in PubMed   Google Scholar

Time pressures and a lack of confidence could be prompting reviewers to plagiarize text in their reports. Credit: Thomas Reimer/Zoonar via Alamy

Mikołaj Piniewski is a researcher to whom PhD students and collaborators turn when they need to revise or refine a manuscript. The hydrologist, at the Warsaw University of Life Sciences, has a keen eye for problems in text — a skill that came in handy last year when he encountered some suspicious writing in peer-review reports of his own paper.

Last May, when Piniewski was reading the peer-review feedback that he and his co-authors had received for a manuscript they’d submitted to an environmental-science journal, alarm bells started ringing in his head. Comments by two of the three reviewers were vague and lacked substance, so Piniewski decided to run a Google search, looking at specific phrases and quotes the reviewers had used.

To his surprise, he found the comments were identical to those that were already available on the Internet, in multiple open-access review reports from publishers such as MDPI and PLOS. “I was speechless,” says Piniewski. The revelation caused him to go back to another manuscript that he had submitted a few months earlier, and dig out the peer-review reports he received for that. He found more plagiarized text. After e-mailing several collaborators, he assembled a team to dig deeper.

Meet this super-spotter of duplicated images in science papers

The team published the results of its investigation in Scientometrics in February 1 , examining dozens of cases of apparent plagiarism in peer-review reports, identifying the use of identical phrases across reports prepared for 19 journals. The team discovered exact quotes duplicated across 50 publications, saying that the findings are just “the tip of the iceberg” when it comes to misconduct in the peer-review system.

Dorothy Bishop, a former neuroscientist at the University of Oxford, UK, who has turned her attention to investigating research misconduct, was “favourably impressed” by the team’s analysis. “I felt the way they approached it was quite useful and might be a guide for other people trying to pin this stuff down,” she says.

Peer review under review

Piniewski and his colleagues conducted three analyses. First, they uploaded five peer-review reports from the two manuscripts that his laboratory had submitted to a rudimentary online plagiarism-detection tool . The reports had 44–100% similarity to previously published online content. Links were provided to the sources in which duplications were found.

The researchers drilled down further. They broke one of the suspicious peer-review reports down to fragments of one to three sentences each and searched for them on Google. In seconds, the search engine returned a number of hits: the exact phrases appeared in 22 open peer-review reports, published between 2021 and 2023.

The final analysis provided the most worrying results. They took a single quote — 43 words long and featuring multiple language errors, including incorrect capitalization — and pasted it into Google. The search revealed that the quote, or variants of it, had been used in 50 peer-review reports.

Predominantly, these reports were from journals published by MDPI, PLOS and Elsevier, and the team found that the amount of duplication increased year-on-year between 2021 and 2023. Whether this is because of an increase in the number of open-access peer-review reports during this time or an indication of a growing problem is unclear — but Piniewski thinks that it could be a little bit of both.

Why would a peer reviewer use plagiarized text in their report? The team says that some might be attempting to save time , whereas others could be motivated by a lack of confidence in their writing ability, for example, if they aren’t fluent in English.

The team notes that there are instances that might not represent misconduct. “A tolerable rephrasing of your own words from a different review? I think that’s fine,” says Piniewski. “But I imagine that most of these cases we found are actually something else.”

The source of the problem

Duplication and manipulation of peer-review reports is not a new phenomenon. “I think it’s now increasingly recognized that the manipulation of the peer-review process, which was recognized around 2010, was probably an indication of paper mills operating at that point,” says Jennifer Byrne, director of biobanking at New South Wales Health in Sydney, Australia, who also studies research integrity in scientific literature.

Paper mills — organizations that churn out fake research papers and sell authorships to turn a profit — have been known to tamper with reviews to push manuscripts through to publication, says Byrne.

The fight against fake-paper factories that churn out sham science

However, when Bishop looked at Piniewski’s case, she could not find any overt evidence of paper-mill activity. Rather, she suspects that journal editors might be involved in cases of peer-review-report duplication and suggests studying the track records of those who’ve allowed inadequate or plagiarized reports to proliferate.

Piniewski’s team is also concerned about the rise of duplications as generative artificial intelligence (AI) becomes easier to access . Although his team didn’t look for signs of AI use, its ability to quickly ingest and rephrase large swathes of text is seen as an emerging issue.

A preprint posted in March 2 showed evidence of researchers using AI chatbots to assist with peer review, identifying specific adjectives that could be hallmarks of AI-written text in peer-review reports .

Bishop isn’t as concerned as Piniewski about AI-generated reports, saying that it’s easy to distinguish between AI-generated text and legitimate reviewer commentary. “The beautiful thing about peer review,” she says, is that it is “one thing you couldn’t do a credible job with AI”.

Preventing plagiarism

Publishers seem to be taking action. Bethany Baker, a media-relations manager at PLOS, who is based in Cambridge, UK, told Nature Index that the PLOS Publication Ethics team “is investigating the concerns raised in the Scientometrics article about potential plagiarism in peer reviews”.

How big is science’s fake-paper problem?

An Elsevier representative told Nature Index that the publisher “can confirm that this matter has been brought to our attention and we are conducting an investigation”.

In a statement, the MDPI Research Integrity and Publication Ethics Team said that it has been made aware of potential misconduct by reviewers in its journals and is “actively addressing and investigating this issue”. It did not confirm whether this was related to the Scientometrics article.

One proposed solution to the problem is ensuring that all submitted reviews are checked using plagiarism-detection software. In 2022, exploratory work by Adam Day, a data scientist at Sage Publications, based in Thousand Oaks, California, identified duplicated text in peer-review reports that might be suggestive of paper-mill activity. Day offered a similar solution of using anti-plagiarism software , such as Turnitin.

Piniewski expects the problem to get worse in the coming years, but he hasn’t received any unusual peer-review reports since those that originally sparked his research. Still, he says that he’s now even more vigilant. “If something unusual occurs, I will spot it.”

doi: https://doi.org/10.1038/d41586-024-01312-0

Piniewski, M., Jarić, I., Koutsoyiannis, D. & Kundzewicz, Z. W. Scientometrics https://doi.org/10.1007/s11192-024-04960-1 (2024).

Article   Google Scholar  

Liang, W. et al. Preprint at arXiv https://doi.org/10.48550/arXiv.2403.07183 (2024).

Download references

Related Articles

• Peer review
• Research management

Illuminating ‘the ugly side of science’: fresh incentives for reporting negative results

Career Feature 08 MAY 24

Algorithm ranks peer reviewers by reputation — but critics warn of bias

Nature Index 25 APR 24

Researchers want a ‘nutrition label’ for academic-paper facts

Nature Index 17 APR 24

Structure peer review to make it more robust

World View 16 APR 24

Is ChatGPT corrupting peer review? Telltale words hint at AI use

News 10 APR 24

Mount Etna’s spectacular smoke rings and more — April’s best science images

News 03 MAY 24

How reliable is this research? Tool flags papers discussed on PubPeer

News 29 APR 24

Southeast University Future Technology Institute Recruitment Notice

Professor openings in mechanical engineering, control science and engineering, and integrating emerging interdisciplinary majors

Nanjing, Jiangsu (CN)

Southeast University

Staff Scientist

A Staff Scientist position is available in the laboratory of Drs. Elliot and Glassberg to study translational aspects of lung injury, repair and fibro

Maywood, Illinois

Loyola University Chicago - Department of Medicine

W3-Professorship (with tenure) in Inorganic Chemistry

The Institute of Inorganic Chemistry in the Faculty of Mathematics and Natural Sciences at the University of Bonn invites applications for a W3-Pro...

53113, Zentrum (DE)

Rheinische Friedrich-Wilhelms-Universität

Principal Investigator Positions at the Chinese Institutes for Medical Research, Beijing

Studies of mechanisms of human diseases, drug discovery, biomedical engineering, public health and relevant interdisciplinary fields.

Beijing, China

The Chinese Institutes for Medical Research (CIMR), Beijing

Research Associate - Neural Development Disorders

Houston, Texas (US)

Baylor College of Medicine (BCM)

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies