Appraising design and bias at the bedside — OSCE

Station brief (8–10 minutes)

Help the parents decide whether to start the new therapy by appraising the trial's design honestly and judging its applicability to their seven-year-old. Address the vague allocation and the 25 percent loss to follow-up as sources of bias, weigh the fact that the trial was conducted in adults, and integrate the evidence with the family's values. Do not invent jurisdiction-specific thresholds or specific outcome statistics. [1]

Tasks for the candidate

1. Identify the study design and name the design manoeuvre most critical to its validity, explaining the empirical consequence of inadequate allocation concealment. [3]
2. Outline the RoB 2 domains and explain the specific concern raised by the 25 percent loss to follow-up. [5]
3. Judge the external validity of an adult trial for a seven-year-old child, naming the threats of indirectness and developmental heterogeneity. [4]
4. Integrate the appraisal with the family's values through shared decision-making, and arrange follow-up. [1] [5]

Expected performance

Must hit. Design identified as a randomised controlled trial and allocation concealment named as the critical manoeuvre, with the empirical fact that inadequate concealment exaggerates the effect by about 30 to 40 percent; the RoB 2 domains listed with the missing-outcome domain flagged for the 25 percent loss; the adult trial judged as indirect for a seven-year-old with certainty downgraded; the decision integrated with the family's values; a written summary and follow-up arranged. [3] [5]

Merit. Acknowledges the online claim without dismissing the parent; distinguishes concealment from blinding and explains why both matter; quantifies the loss as above the 20 percent threshold that threatens validity; checks whether the loss was differential and whether a sensitivity analysis was done; states the PICO mismatch explicitly; uses teach-back and a decision aid for the preference-sensitive choice; names the next step for seeking paediatric evidence. [2] [4]

Fail. Quotes the dramatic benefit and calls the therapy transformative; ignores the vague allocation and the loss to follow-up; applies the adult trial to the child without judging applicability; gives no risk-of-bias assessment, no certainty rating, no shared decision, and no follow-up. [3] [1]

Sample candidate structure

"Thank you both for coming in. The study you read online is a randomised trial, which is the strongest design for testing a therapy, but its value depends on how well it was run, and there are two things in the methods that worry me. First, the paper is vague about how children were allocated to the treatment or the placebo group. Allocation concealment is the safeguard that prevents staff from steering certain patients toward one group, and when it is not done well the reported benefit is exaggerated by about 30 to 40 percent, so the dramatic result may be overstated. Second, a quarter of the participants dropped out, which is above the threshold that threatens the result, because those who left may have differed from those who stayed. I would also be cautious about applying an adult trial to your seven-year-old, because children can metabolise and respond to medicines differently. So my honest summary is that the therapy shows promise, but the evidence is not yet strong enough or specific enough to children for me to recommend it confidently. Let us weigh that together, look for paediatric evidence, and decide on a plan with close follow-up." [3] [5] [4] [1]