Evidence-based medicine and critical appraisal — formative SAQs

SAQ 1 (10 marks)

You are asked to appraise a randomised controlled trial claiming a new therapy reduces the risk of a serious complication in children. The abstract reports a relative risk reduction of 50 percent. [1]

1. Outline the steps you would take to appraise this trial, in the correct order, before applying its result. (4) [7] [3]
2. Explain why quoting a 50 percent relative risk reduction alone is misleading, and describe how you would present the benefit accurately. (3) [1] [6]
3. Describe how you would decide whether the result applies to your patient and how to move from the evidence to a recommendation. (3) [7] [5]

Model answer

Frame the clinical question in PICO terms, then search the filtered layer first. Judge the internal validity before the results: identify the design, then score the risk of bias with RoB 2, paying attention to randomisation, allocation concealment, blinding, attrition and selective reporting. Only then extract the magnitude and precision of the effect, and check for publication bias if the result comes from a synthesis. [2] [7] [3]

A relative risk reduction quoted alone inflates perceived benefit because it hides the baseline. Present the absolute baseline risk and the absolute risk reduction, give the number needed to treat, and read the confidence interval so the family understands the range compatible with the data. [1] [6]

Assess applicability by comparing the trial population, intervention, comparator, outcomes and setting against the child, and rate the certainty of the body of evidence with GRADE, downgrading for indirectness where the match is poor. Combine that certainty with the balance of benefits and harms and the family's values to set a strong or weak recommendation, and document the reasoning. [7] [4] [5]

SAQ 2 (10 marks)

A colleague hands you a meta-analysis reporting a large benefit for a treatment, but the review pooled only published trials and its funnel plot is asymmetric. [7]

1. Explain what funnel-plot asymmetry suggests and how it affects the pooled estimate. (4) [7]
2. Describe how you would rate the certainty of this evidence using GRADE, and why. (3) [4]
3. Outline how you would decide whether to apply this evidence to a child in your clinic. (3) [7] [5]

Model answer

Funnel-plot asymmetry suggests publication bias, meaning small negative studies failed to reach publication so the pooled estimate overstates the true effect. The clinician should search trial registers for unpublished data and treat the headline effect as inflated until corrected. [7]

Under GRADE, begin with the design-based starting certainty and move it down for publication bias, and for any risk of bias, inconsistency, indirectness or imprecision in the included trials. Publication bias alone is sufficient grounds to lower the certainty, because the pooled estimate is no longer a trustworthy reflection of all the evidence. [4]

Before applying the result, check applicability to the child by comparing the population, intervention, comparator, outcomes and setting, and weigh the certainty-rated benefit against the harms and the family's values through shared decision-making. Where directness is poor or certainty is low, present the uncertainty honestly and prefer the reversible or better-evidenced option. [7] [5]