Psych MEQs / SAQs · Professional practice — critical appraisal and EBM
Critical appraisal of an antidepressant RCT abstract (MEQ)
FRANZCP/MRCPsych-style MEQ on RCT validity, bias taxonomy, ARR/NNT calculation, and applicability in psychiatry.
On this page & tools
Target exams
Model answer
Reveal model answer
(i) PICO. P: adults with moderate MDD (HAM-D 18–24). I: Drug A. C: placebo. O: depressive symptom change at 8 weeks (primary continuous HAM-D); response rates should also be sought as patient-interpretable outcomes.[1]
(ii) Validity threats. Open randomisation list → failed allocation concealment → selection/allocation bias; empirically linked to exaggerated effects.[3] Completer analysis with differential dropout (28% vs 18%) → attrition bias / missing outcome data domain; ITT not used.[4][5] Continuous scale primary outcome is acceptable if blinded, but abstract does not confirm rater blinding — if open, detection bias. Headline relative claim without absolute response rates is reporting/spin concern (CONSORT expects complete outcome reporting).[6] Map to RoB 2: randomisation process (concealment), missing outcome data, measurement of the outcome, selection of the reported result.[5]
(iii) Numbers for ARR/NNT. Need event rates (or counts) for a dichotomous patient-important outcome such as response or remission in each arm, analysed preferably by ITT. ARR = CER − EER; NNT = 1/ARR. RRR alone can make small absolute benefits look large and does not communicate effort required to help one patient.[2]
(iv) Worked numbers. CER 35%, EER 50% response → if response is the good event, absolute risk increase in response = 15% = 0.15; NNT = 1/0.15 ≈ 6.7 → 7. Relative increase in response = 0.15/0.35 ≈ 43% (similar rhetoric to the abstract's "40%" marketing line). Always state duration (8 weeks) and outcome definition.[2]
(v) Applicability. External validity is limited: a comorbid substance-using outpatient was excluded, so baseline risk, adherence, and drug–drug/interaction context differ; even if internal validity were repaired, one cannot assume the same ARR — shared decision-making with explicit uncertainty and monitoring is appropriate rather than automatic adoption.[1]
Common errors
Ignoring concealment; treating p=0.04 continuous difference as NNT; accepting RRR headline; equating completer analysis with ITT; declaring the drug "proven" for excluded populations; inventing CONSORT item numbers rather than principles.[3][4][6]
References
- [1]Richardson WS, Wilson MC, Nishikawa J, Hayward RS The well-built clinical question: a key to evidence-based decisions ACP J Club, 1995.PMID 7582737
- [2]Laupacis A, Sackett DL, Roberts RS An assessment of clinically useful measures of the consequences of treatment N Engl J Med, 1988.PMID 3374545
- [3]Schulz KF, Chalmers I, Hayes RJ, Altman DG Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials JAMA, 1995.PMID 7823387
- [4]Hollis S, Campbell F What is meant by intention to treat analysis? Survey of published randomised controlled trials BMJ, 1999.PMID 10480822
- [5]Sterne JAC, Savović J, Page MJ, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials BMJ, 2019.PMID 31462531
- [6]Schulz KF, Altman DG, Moher D; CONSORT Group CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials BMJ, 2010.PMID 20332509