Artificial intelligence and clinical decision support in paediatrics — formative SAQs

SAQ 1 (10)

A children's hospital is evaluating a machine-learning deterioration score developed and cross-validated on its own electronic health record data. The executive team proposes switching off the existing bedside paediatric early warning score and letting the model drive all escalations. [2]

1. Define artificial intelligence, machine learning and clinical decision support, and distinguish a trained prediction model from a rule-based alert. (3) [10]
2. State the validation ladder and explain why external validation is the gate that must be cleared before the model drives a clinical escalation. (4) [2]
3. Outline why the model should not replace the bedside score, and the four safeguards that must accompany deployment. (3) [7]

Model answer

Definitions and the model-versus-rule distinction. Artificial intelligence is a computer system performing a task that would otherwise require human intelligence. Machine learning is the branch in which the system learns the relationship between inputs and an outcome from historical data, rather than being programmed with rules. Clinical decision support is the delivery of knowledge or a patient-specific output to the clinician at the point of care. The distinction the examiner tests is between a rule-based alert, built from if-then logic written by a human, and a trained prediction model, which discovers the feature-to-outcome relationship from data. The two are validated, governed and trusted differently, and conflating them loses the marks. [10]

The validation ladder and the external-validation gate. The validation ladder is internal cross-validation, external validation in a different population, and a prospective silent trial. Internal cross-validation tests the model on portions of its own training data and is the weakest evidence, because it cannot detect overfitting or population mismatch. External validation tests the model on a new population — a different hospital, a different age band — and it is the gate, because a model that has only been tested on its own data has not been shown to generalise. The Mayampurath study of a PICU-transfer prediction model is the canonical example: it was developed in one children's hospital and externally validated in another, the step that established transportability. A prospective silent trial then runs the model in the background against real outcomes before it is allowed to drive an action. Until external validation and ideally a silent trial are done, the model must not drive escalations. [2]

Why the model augments rather than replaces, and the four safeguards. The model should not replace the bedside score because the bedside assessment of the child always overrides any AI output, and automation bias has caused missed deterioration when a reassuring number silenced a worried clinician. Sensitivity and specificity are not fixed but shift with prevalence and threshold, so a tool tuned in one ward may mis-score children in another. The four safeguards are clinician over-read of every output, a written escalation plan with explicit thresholds and a twenty-four-hour contact, alert stewardship that measures the override rate and retunes the thresholds, and ongoing monitoring of performance drift, equity and alarm burden. Deployed together they hold the clinician accountable for every output. [7]

SAQ 2 (10)

A paediatric ward team overrides more than nine in ten of the drug-interaction alerts they receive. Separately, a deep-learning tool flags a possible skull fracture on a radiograph that the registrar cannot see, and the child is clinically well. [7]

1. Explain alert fatigue and the stewardship response, citing the relevant studies. (4) [7] [8] [9]
2. Outline the differential of a discrepant AI image interpretation, and the first action for the skull-fracture scenario. (3) [4]
3. State the clinician over-read and accountability principle, and why the responsibility does not transfer to the tool. (3) [10]

Model answer

Alert fatigue and stewardship. Alert fatigue is the exhaustion that follows an unmanageable burden of interruptive alerts, and it drives an override culture in which the team dismisses most alerts reflexively and the one actionable alert is lost in the noise. An alert overridden more than nine times in ten has lost its value. The stewardship response is to measure the override rate, retire or convert the lowest-value interruptive alerts to a passive display — the Fallon study replaced a burdensome interruptive alert with passive clinical decision support — and tune the thresholds with the prescribers, as the Simpao paediatric drug-interaction study did with a visual analytics dashboard. The Chaparro framework sets the best-practice cycle of monitoring and improving interruptive alerts, and a rising override rate is treated as a patient-safety signal rather than an inconvenience. [7] [8] [9]

Differential of a discrepant image interpretation and the first action. For a discrepant AI image interpretation, run the differential through a true finding the human reader missed, a false positive from artefact or unusual anatomy, a model overcall on an under-represented subtype, and a genuine disagreement in which the human reader is correct. For this child, who is clinically well and whose radiograph the registrar cannot reconcile with the flag, the first action is the clinician over-read against the device-independent standard: examine the image, compare it with any prior films, and seek the senior opinion or the formal radiology review before accepting or rejecting the diagnosis. The tool is an assist to the reader, not a replacement for the reader. [4]

Over-read and accountability. The clinician over-read and accountability principle holds that the treating clinician examines every output, questions it, and remains responsible for the decision that follows. The accountability does not transfer to the tool on deployment, because the clinician who acts on the output owns the consequence, and a fluent or confident output that is wrong remains the clinician's responsibility to catch. The verification obligation — checking the output against the source before it is acted on or entered into the record — is what keeps the human in the loop. This is the augmented-intelligence framing the examiner rewards: the tool assists, the clinician decides, and the decision is auditable. [10]