Peer review is a vital part of the process of bringing submissions to Academic Emergency Medicine to publication. Following are guidelines to help reviewers in approaching this important task.

Goals of peer review

1. To assist the editors with decision making
2. To help the authors improve the work
3. To ensure the quality of the journal

Traits of a good peer review

1. Thorough
2. Supportive
3. Timely

Use the questions and concepts below to guide your review. Not every question will be relevant for every study, and some study types may generate questions not included below. The review need not address each question – they are designed to assist in the review process, not serve as the structure of the review. While specific comments are helpful, general comments regarding methodology, relevance, etc. are more helpful to the Associate Editor in making a decision regarding acceptance. Particularly on Sheet A, be sure to emphasize overall concepts that you believe should affect a decision to accept or reject the paper.

FIRST READING

Just read it, don't mark it up yet – look at the forest.

Initial impressions upon completion of first reading:

1. Are all the necessary sections in place, and in the proper order?
2. Does the paper pass the "so what" test, especially within the context of our journal and its mission? Begin thinking about the following:
• What is the importance of this topic to academic EM and to the journal?
• What is the importance of this study compared to others in the same area?
• What is the relevance of this topic to the journal's audience?
• Is this manuscript useful in terms of the balance of topics the journal publishes?
3. Is this a real hypothesis-testing research study, or a descriptive study?
4. If the latter, have the authors tried to inappropriately make it look like the former?
5. Is the copy you received adequately blinded? (If not, please continue to review, but note the inadequate blinding in your review on Sheet A.)
6. Do you suspect you know the authors or the project? If so, do you perceive any conflicts of interest in completing this review? (If so, please contact the Associate Editor or the AEM office before proceeding.)

SECOND READING

Start marking up the manuscript – look at the trees.

Introduction:

• Does it explain why the study was done?
• Is it too long (really a literature review) or too short (inadequate background)?
• Is the research question and/or hypothesis stated explicitly and correctly?
• Can you tell why the authors chose this question? Is it the right question?

Methods

1. Would the study be completely reproducible using only this section?
2. Are the techniques and/or models used understandable?
3. Was the setting adequately explained?
4. Were inclusion and exclusion criteria explicitly stated?
5. Were controls adequately selected? Consider blinding, randomization, and bias issues.
6. Were all interventions, tests, measurements, and calculations explained adequately?
7. Were the correct outcomes examined?
8. Are there other outcomes that should have been examined but weren't?
9. Are all necessary and appropriate analytical methods, statistical tests, sample size calculations, and power calculations included and adequately explained?

Results

1. Do the data and results match the methods and design used?
2. Can the reader determine who was included and why?
3. Are all study subjects accounted for?
4. Are baseline characteristics of any subjects or groups adequately explained?
5. Are there any signs of data dredging, e.g.looking through a pre-existing database to see what types of questions the data might answer, digging deeper than was originally planned, etc.?
6. Are the statistics being applied to the data appropriately?

Discussion

1. Does the discussion place the results and the study in context?
2. Are the observations supported by the authors' data? Are they over- or under-stated?
3. Is it clear how important this study is relative to the topic, or to other studies done on the same topic?
4. Did the authors wander off the topic? Misrepresent the work of others? Miscast their work or its importance?
5. Is there information in this section that should have been included in the introduction, methods, or limitations sections?

Limitations

1. Are the limitations adequately explained?
2. Are suggestions included for addressing these in future work?
3. Are any significant limitations missing?

Conclusions

1. Are the conclusions supported by the data?
2. Are there any overstated or unsubstantiated conclusions?

Figures and Tables

1. Are they helpful in summarizing data or illustrating a point? Has this already been done in the text? If so, are the figures and/or tables needed?
2. Are they easy to read on their own? Must the reader flip back and forth to the text to understand the figures and/or tables?

References

1. Are they up-to-date (relative to the topic being studied)?
2. Have they been cited and used correctly?
3. Are there other studies that should have been discussed and referenced but were overlooked?

Abstract: Least important section – decisions will be based on the study, not the abstract.

1. Does the abstract adequately reflect and summarize the study?
2. Are there any conflicts between the study text and the abstract?

THIRD READING

Put it all together, start writing the review – look at the forest again.

1. Address the following on Sheet A, being blunt and honest – what do you really think?
• Originality
• Scientific Reliability
• Clinical Importance
• Suitability for AEM
• Clarity
• Grammar
2. Start addressing Major Strengths and Major Weaknesses (for Sheet A).
3. Is there a fatal flaw that the authors either overlooked or tried to bury?
4. Is the paper too long (common) or too short (less common)?
5. Start addressing General Comments (for Sheet B).
6. Save Specific Comments (for Sheet B) for last, to avoid losing the forest for the trees.
7. Make a disposition recommendation, explaining why you favor a certain disposition.