© 2018 Campbell, Heffernan, & Chen
Icons were made by Icon Pond from www.flaticon.com

Critical Appraisals

 

Critiquing overview 

1. First Read 

  • Skim the entire journal article to get an overall impression of the paper

  • Mark aspects of the paper that you need to review again for understanding purposes

2. Establish Research Context  

  • Who conducted the research?

  • What are the researchers interests? (ie. previous publications)

  • When and where was the research conducted?

3. Second Read: Analyze the text and determine the pros and cons of the paper

  • Analyze/critique the paper 

  • Make note of the positives and negatives for each part of the paper (ex. methods, results, etc.)

4. Second/Third Read: Establish the significance of the research   

  • Was the research successful?

  • Has it led to new research questions?

  • Have the authors found new ways to use existing knowledge?

  • Are other researchers citing this paper?

  • What impact will this paper have on the field?

Pros VS. Cons

CRITIQUING TIPS 

 

Structure

  • Caution: authors may not be in control of structure due to journal requirements

  • Check "for the author" section of the journal and determine the journal requirements prior to critiquing structure 

  • Obvious structural issues missed by reviewers should be critiqued

SUPPORT

  • Only make a critique if you have strong support for your claim.

  • After providing your support, you can provide suggestions  to fix the issue.

Superficial

  • Only critique aspects of the paper that impacted the reader and their interpretations of the findings

  • Ex. social implications of research, lacking the mention of significant results 

  • Avoid superficial claims (ex. figures should have been with their results; spelling mistakes)

POSITIVE

  • Not all of your critiques need to be negative

  • Some critiques can comment on the positive aspects of the paper. 

Questions to guide your critiquing

 
  • This determines the optimal study design

  • Good research questions include: the population of interest, studied parameter, and the outcome of interest

  • Did it introduce any biases?

  • Would another study design provide more comprehensive results?

  • Were there enough study participants?  

  • Check that all data relevant to the hypothesis have been reported and not omitted

  • Note: if the study investigates the statistical significance of associations that were not included in the original hypothesis then these analyses are prone to false-positive findings

  • Did the authors consider study participants lost to follow up? (ex. was an intention to treat analysis employed)

  • Was original data presented so readers can check the statistical accuracy of the paper?

  • Did authors use SEM or SD for their data? (based on generalizability of results)

  • Are the authors conclusions supported by the study’s data?   

  • Do the statistically significant results have any clinical relevance?

  • Were non-statistically significant differences mentioned (the omission of these could cause issues for some studies)?

Young, J. M., & Solomon, M. J. (2009). How to critically appraise an article. Nat Clin Pract Gastroenterol Hepatol, 6(2), 82-91. doi:10.1038/ncpgasthep1331

 

How do I Summarize a Scientific Article?

What Do I Need to Include?

A summary needs to touch on every section of the paper. You need to summarize the introduction, methods, results, and discussion sections of the paper. 

How is a Summary Different From an Abstract?

An abstract is a brief summary of the article. It still includes some of the main ideas/key findings from the paper but does not go into the same amount of detail that a summary should. Abstracts are mainly there to provide readers with a sense of what the article is about (overview).

 

Abstracts are a good place to start when you are writing a summary so you can get a general sense of what needs to be included in your summary, but your summary SHOULD NOT be the same as the abstract. If your summary only includes the information found in the abstract, then your summary is too condensed and doesn't include enough detail regarding the study!

Writing a Summary

Actively read (highlight, annotate, make notes) through each section of the article.

Actively read (highlight, annotate, make notes) through each section of the article.

Everyone approaches reading articles differently. Try some different styles and see what works for you!

 

  1. Read the abstract and then skim the introduction. Pay close attention to the figures to identify the ones that present the most important results. Move on to reading the discussion, paying special attention to the parts that interpret the results from the figure(s) you noted above. 

  2. Start by reading the abstract and discussion/conclusion. Conclusions will help you to understand if the goal mentioned in the abstract was reached, and they will highlight the interpretations made.

  3. Start by reading the introduction to understand the topic and research question. Move on to the results to determine if the research question has been answered.

  4. Read the methods section last, unless you can't make sense of the results without reading it first!

 

Some Other Tips...

  • Pay attention to the techniques used in methods—is it common or obscure?

  • Compare the results presented in the figures/tables to the interpretation drawn—are the authors over-exaggerating their results or interpreting something incorrectly?

  • Review other articles referenced in the paper—are the authors taking anything out of context just so they can use it as evidence in their paper?

  • Don't forget to look at the supplemental figures/tables (if any)!

  • Have the authors used a sufficient number of controls/samples in their study? Limited samples/controls will impact the significance/conclusions drawn.

  • Print out the article instead of reading it on your computer. Highlight using different colours based on your understanding. It is a lot easier to actively read this way!

Identify the research question/objective of the study.

Why is Knowing the Research Question Important?

The research question/objective will be important for when you are trying to identify what the main idea/key points are from each section, which will ultimately determine what ends up in your summary! 

Where Can I Find the Research Question/Objective?

The research question can be found (usually) near the end of the introduction, but can also be found as a summarized version in the abstract.

When writing the summary, remember to summarize all sections.

Try to think of the summary as a story. You want to start out by introducing the topic and why the study was conducted, explain how they planned to conduct the study, the results that they got from the experiments, and what those results mean.

 

Introduction

  • Identify the objective/research question and provide relevant background information

  • Discuss any gaps in previous literature if it helps explain why the current study was conducted

 

Methods

  • Identify what was measured (variables), and explain how this was done

  • Some things you can discuss include test subjects, treatments, controls, sample size, techniques, etc.

 

Results

  • Talk about the main findings from the study

  • Often, there will have been lots of tests that were performed yielding lots of different results. You want to only include the ones that are relevant to the objective/research question or any surprising results that get discussed later

Discussion

  • Include the interpretation of the results and how it relates back to the objective/research question

Critiquing Framework

Note: This framework can also be applied to other written assignments when you are stating a fact and supporting it. (This is a good framework to show critical thinking!)