Diagnostic accuracy and screening statistics — formative SAQs

SAQ 1 (10 marks)

A newborn bloodspot screen for a rare metabolic condition comes back positive in an otherwise well baby. The test has a sensitivity of 99 percent and a specificity of 99 percent, and the condition affects 1 in 10,000 babies. [4]

1. Build the 2×2 table for a population of 10,000 babies and calculate the sensitivity, specificity, and positive predictive value. (5) [4] [1]
2. Explain why the positive predictive value is so low despite the excellent specificity, and name the principle that governs it. (3) [4]
3. Describe how you would counsel the family and what the next step is. (2) [4] [13]

Model answer

Of 10,000 babies, one is truly affected (prevalence 1 in 10,000) and 9,999 are well. With a sensitivity of 99 percent, the test catches the single true case as one true positive. With a specificity of 99 percent, one percent of the 9,999 well babies test falsely positive, giving about 100 false positives. Sensitivity is one true positive divided by one affected, or 99 percent; specificity is about 9,899 true negatives divided by 9,999 well, or 99 percent; the positive predictive value is one true positive divided by about 101 positives, or roughly 1 percent. [4] [1]

The positive predictive value is low because it is a function of prevalence, not of the test's intrinsic quality. In a low-prevalence population the many false positives thrown up by the large healthy majority overwhelm the single true case, so even a 99 percent specific test yields mostly false positives. Sensitivity and specificity are fixed properties of the test at its threshold, but the predictive value inherits the ratio of diseased to well, which is what prevalence sets. [4]

I would counsel the family that most babies with a positive screen are well, that the screen is a reason to confirm rather than a diagnosis, and that the next step is the reference-standard confirmatory test. I would avoid conveying a diagnosis before confirmation and would acknowledge the anxiety the screen has caused. [4] [13]

SAQ 2 (10 marks)

A colleague hands you a diagnostic accuracy study claiming a bedside test is 95 percent sensitive and 93 percent specific for a serious paediatric condition, and proposes adopting it across a community population. [9]

1. Describe how you would appraise the study's validity using QUADAS-2, naming the four domains and one bias each domain may reveal. (5) [9]
2. Explain two reasons the test's positive predictive value may be much lower in the community than in the study, and the consequence for adoption. (3) [4] [10]
3. Outline how you would move from the test's likelihood ratio to a shared decision with a family. (2) [5] [7]

Model answer

I would apply QUADAS-2 across its four domains. Patient selection asks whether the study recruited consecutive children from the intended-use population or a mixture of severe cases and healthy controls, the latter producing spectrum bias. The index test domain asks how the test was conducted and whether the interpreter was blinded to the reference standard, revealing review bias. The reference standard domain asks whether the standard correctly classifies the target condition and is independent of the index test, revealing incorporation bias. Flow and timing asks whether every participant received the same reference standard at the appropriate interval, revealing partial or differential verification bias. Any high-risk domain inflates the headline accuracy. [9]

The positive predictive value will fall in the community for two reasons. First, the prevalence of the condition will be lower than in the study population, and because predictive value is governed by prevalence, fewer of the positives will be true. Second, if the study population was a tertiary spectrum, spectrum bias means the quoted sensitivity and specificity will not hold in the community's milder, earlier disease. The consequence is a flood of false positives, unnecessary follow-up, family anxiety, and possible overdiagnosis, so the test may do more harm than good at the community prevalence. [4] [10]

I would estimate the child's pre-test probability, select the test because its likelihood ratio crosses my decision threshold, combine the result with the pre-test probability using the likelihood ratio or Fagan nomogram to obtain the post-test probability, and then share that probability with the family in plain terms to make a shared decision about treatment, confirmation, or reassurance. [5] [7]