Paediatric study design and bias — formative SAQs

SAQ 1 (10 marks)

You are asked to appraise a randomised controlled trial of a new paediatric therapy. The trial reports a large and statistically significant benefit, but the methods section is vague about how participants were allocated to the treatment and placebo groups, and 25 percent of participants were lost to follow-up. [1]

1. Name the design manoeuvre that is most critical to the validity of a randomised trial, and explain why inadequate conduct of it empirically distorts the result. (3) [6]
2. Using the RoB 2 tool, outline the domains you would assess and the specific concern raised by the 25 percent loss to follow-up. (4) [8]
3. Explain how you would judge whether this trial's result applies to a seven-year-old child in your clinic, and what you would do if the applicability were uncertain. (3) [1] [8]

Model answer

The design manoeuvre most critical to a trial's validity is allocation concealment, which hides each assignment until it is irreversible and prevents staff from steering certain patients toward the arm they prefer. When allocation concealment is inadequate, the comparison between groups is corrupted at the moment of enrolment, because the groups may then differ in prognostic factors rather than only in the treatment received. The empirical evidence from Schulz and colleagues is that trials without adequate allocation concealment exaggerate the treatment effect by about 30 to 40 percent compared with adequately concealed trials. A large and significant benefit from a trial whose allocation is vaguely reported is therefore the signature of bias rather than of a breakthrough, and the result must be read down toward the null. [6] [8]

The RoB 2 tool assesses five domains: the randomisation process, which addresses both sequence generation and allocation concealment; deviations from intended interventions, which covers blinding; missing outcome data, which is the concern raised by the 25 percent loss; measurement of the outcome, which checks whether assessment was blinded; and selection of the reported result, which checks for selective reporting. The 25 percent loss to follow-up exceeds the 20 percent threshold that threatens validity, because those lost may differ systematically from those who remained in the trial. I would judge whether the loss was differential across arms, whether the missing data could have changed the conclusion, and whether a sensitivity analysis was performed. If the missing data could plausibly have erased the benefit, I would rate the missing-outcome domain as high risk of bias and downgrade the certainty accordingly. [8]

To judge applicability, I would compare the trial's population, intervention, comparator, outcome, and setting with the child in front of me using PICO concordance. I would check whether seven-year-olds were included, whether the dose and the outcome definition match my patient's needs, and whether the setting resembles my clinic. If the applicability were uncertain, for example because the trial excluded the child's age band or complexity, I would downgrade the certainty for indirectness, choose the reversible option where possible, involve the family in shared decision-making, and arrange close follow-up while better evidence is sought. [1] [8]

SAQ 2 (10 marks)

A colleague shows you a case-control study reporting that children who developed a rare adverse event were four times more likely to have been exposed to a particular medication in infancy. Your colleague tells a parent the medication "caused" the adverse event. [4]

1. Explain why a case-control design is appropriate for studying a rare adverse event, and name its signature bias. (3) [4] [5]
2. Distinguish confounding from effect modification, and state the correct response to each. (4) [5]
3. Explain why you would not tell the parent that the medication "caused" the adverse event, and how you would describe the finding honestly. (3) [5] [1]

Model answer

A case-control design is appropriate for a rare adverse event because it starts with the outcome and works backwards, assembling all available cases and a set of controls, which makes it far more efficient than assembling a cohort large enough to capture a rare outcome prospectively. Its signature bias is recall bias, because parents of affected children, searching for a cause, may remember past exposures more thoroughly than parents of unaffected controls, and that differential recall can manufacture an apparent association. The defence is to collect exposure data identically in cases and controls, ideally from records made before the outcome was known, and to blind interviewers to case-control status. [4] [5]

Confounding and effect modification are easily confused but demand opposite responses. Confounding is a distortion of the comparison by a third variable that is associated with the exposure, independently associated with the outcome, and not on the causal pathway, and the correct response is to remove it by randomisation, matching, restriction, or statistical adjustment. Effect modification, by contrast, is a true biological difference in the effect across subgroups, and the correct response is to report it rather than remove it, because adjusting away effect modification hides a real finding. The two are distinguished by whether adjustment changes the estimate toward the null, which suggests confounding, or reveals different estimates in different strata, which suggests effect modification. [5]

I would not tell the parent that the medication caused the adverse event, because a single case-control study establishes an association but cannot prove causation, and the design is open to recall bias and to confounding by factors that were not measured or adjusted for. I would describe the finding honestly as an association that raises a concern and warrants further study, acknowledge the limits of the design, and place the result in the hierarchy of evidence. I would then weigh the magnitude of the association against the benefit of the medication for the child, involve the family in shared decision-making, and arrange monitoring or an alternative where appropriate. [5] [1]