Interpreting systematic reviews and clinical guidelines — formative SAQs

SAQ 1 (10 marks)

You are asked to interpret a meta-analysis claiming a new therapy halves the risk of a serious complication in children. The review was industry-funded, its funnel plot is asymmetric, and its I-squared is 65 percent. [10]

1. Outline the steps you would take to appraise this review before applying its pooled result. (4) [1] [2]
2. Explain what the asymmetric funnel plot and the I-squared of 65 percent each suggest for the pooled estimate. (3) [2] [4]
3. Describe how you would convey the size of the benefit to the family. (3) [10] [8]

Model answer

First confirm the question was focused and preregistered, then check that the search was reproducible and exhaustive with a readable PRISMA 2020 flow diagram that reaches trial registers and unpublished data. Score the risk of bias of the review and its included studies with AMSTAR-2 or ROBIS, paying attention to protocol registration, duplicate screening, and the handling of the included studies' own bias. Only then read the pooled effect with its confidence interval, quantifying heterogeneity and checking for publication bias, before weighing applicability to the child and rating the certainty with GRADE. [1] [2] [10]

The asymmetric funnel plot suggests publication bias — small negative studies failed to reach publication, so the pooled estimate overstates the true effect. The I-squared of 65 percent indicates substantial heterogeneity, meaning the included studies disagree beyond chance, so the pooled estimate is fragile and should be treated cautiously until a clinically sensible subgroup explains the disagreement. The industry funding adds a further structural reason to read the certainty as low. [2] [4]

To convey the benefit, give the absolute baseline risk and the absolute risk reduction, the number needed to treat, and the confidence interval, so the family understands both the magnitude they can feel and the range compatible with the data. Where the certainty is low and the choice is preference-sensitive, present the uncertainty honestly and let the family share the decision. [10] [8]

SAQ 2 (10 marks)

A national guideline strongly recommends a therapy for a chronic condition, but the evidence base is entirely in adults and your patient is a 6-month-old infant excluded from every trial. [8]

1. Describe how you would appraise the trustworthiness of the guideline using AGREE II. (4) [7]
2. Explain how you would weigh applicability of the recommendation to the infant and the action you would take. (3) [8] [10]
3. Outline how the GRADE Evidence-to-Decision framework moves from evidence to a recommendation. (3) [7] [8]

Model answer

Appraise the guideline against the six AGREE II domains: scope and purpose, stakeholder involvement, rigour of development, clarity of presentation, applicability, and editorial independence, using the 23-item checklist. Check that it rests on a systematic review of the evidence, that it rated its own certainty with GRADE, and that it is current. Scrutinise the panel for conflicts of interest under editorial independence, because undisclosed conflicts tilt recommendations even when the method is otherwise sound. [7]

The infant sits outside the adult evidence base, so under GRADE rate the evidence down for indirectness, weigh age-specific pharmacology and safety, and treat the strong adult recommendation as not automatically transferable. The correct action is to adapt or depart from the recommendation with explicit reasons, shared with the family through shared decision-making, documented, and supported by specialist input given the high stakes and the sparse direct evidence. [8] [10]

The GRADE Evidence-to-Decision framework makes the move from evidence to recommendation transparent by weighing the certainty of the evidence alongside the balance of desirable and undesirable effects, the values and preferences of those affected, the resources required, equity, acceptability, and feasibility. The panel's strength — strong or weak — reflects its confidence that following the recommendation does more good than harm across most patients, not the size of the effect, and the two should always be read separately. [7] [8]