Quality Metrics: APACHE, SOFA, ANZROD, and ICU Benchmarking

Quick Answer

ICU Quality Metrics are standardized scoring systems and outcome measures used to assess patient severity, predict mortality, compare ICU performance, and drive quality improvement. The major systems include APACHE (Acute Physiology and Chronic Health Evaluation), SOFA (Sequential Organ Failure Assessment), SAPS (Simplified Acute Physiology Score), and ANZROD (Australian and New Zealand Risk of Death).

Key Clinical Applications:

• Case-mix adjustment: Comparing outcomes between ICUs with different patient populations
• Benchmarking: Identifying outlier performance using standardized mortality ratios (SMR)
• Quality improvement: Tracking outcomes over time to measure improvement initiatives
• Resource allocation: Informing healthcare planning and funding models
• Research stratification: Risk-adjusting clinical trial outcomes

Critical Distinctions:

• APACHE II/III/IV: Admission-based severity scoring using worst physiology in first 24 hours
• SOFA: Daily organ dysfunction scoring for tracking trajectory
• qSOFA: Bedside screening tool for sepsis outside ICU (NOT for severity assessment)
• ANZROD: Australian/NZ-specific recalibrated model with superior local performance
• SMR: Observed/Expected deaths ratio for benchmarking ICU performance

ICU Mortality Benchmarking: An SMR < 1.0 indicates better-than-expected outcomes; SMR > 1.0 indicates worse-than-expected outcomes. Funnel plots account for case volume when interpreting SMR.

---

CICM Exam Focus

What Examiners Expect

Second Part Written (SAQ):

Common SAQ stems:

• "A 65yo male is admitted to ICU with severe sepsis. His APACHE II score is 24. Discuss the components, calculation, and limitations of APACHE II scoring."
• "Your ICU has an SMR of 1.15. Outline your approach to interpreting and investigating this finding."
• "Compare and contrast APACHE, SOFA, and SAPS scoring systems for ICU risk prediction."
• "Discuss the role of qSOFA in sepsis screening and explain why it differs from SOFA."
• "Outline the ANZROD model and its advantages over APACHE for Australian/NZ ICUs."

Expected depth:

• Know all components of APACHE II (12 physiological variables, age, chronic health points)
• Understand SOFA components (6 organ systems, 0-4 points each)
• Explain calibration vs discrimination in model performance
• Describe SMR calculation and funnel plot interpretation
• Discuss limitations of scoring systems for individual prognostication

Second Part Hot Case:

Typical presentations:

• Long-stay complex patient with multiple organ dysfunction - calculate and interpret SOFA trajectory
• Multi-morbid elderly patient - discuss prognostication using severity scores appropriately
• End-of-life discussion where family asks "what are his chances"
• explain appropriate use of population-based predictions

Examiners assess:

• Systematic calculation of severity scores
• Understanding of score limitations
• Appropriate communication of prognosis to families
• Integration of scores with clinical judgment
• Knowledge of Australian-specific ANZROD model

Second Part Viva:

Expected discussion areas:

• APACHE II components and calculation methodology
• Differences between APACHE II, III, and IV
• SOFA vs APACHE for different applications
• qSOFA controversy and appropriate use
• SMR and funnel plot interpretation
• Quality indicators beyond mortality
• Indigenous health outcome disparities

Examiner expectations:

• Safe, nuanced approach to prognostication
• Evidence-based knowledge of model validation studies
• Understanding of statistical concepts (calibration, discrimination, SMR)
• Awareness of Australian-specific models (ANZROD)
• Ethical considerations in using scores for resource allocation

Common Mistakes

• Confusing qSOFA (screening tool) with SOFA (severity/organ dysfunction score)
• Using severity scores for individual patient prognostication rather than population-level predictions
• Not understanding that older models (APACHE II, SAPS II) overestimate mortality in modern ICUs
• Failing to account for lead-time bias when scores calculated before full resuscitation
• Not knowing ANZROD is the preferred model for Australian/NZ benchmarking

---

Key Points

Must-Know Facts

1. APACHE II (1985): 12 physiological variables (worst in first 24h) + age points + chronic health points; score 0-71; most widely cited but outdated calibration overestimates mortality in modern ICUs (PMID: 3928249)

2. SOFA Score (1996): 6 organ systems (respiratory, cardiovascular, hepatic, coagulation, renal, neurological); 0-4 points each; maximum 24; serial measurement tracks trajectory; delta SOFA predicts outcome (PMID: 8844239)

3. qSOFA (2016): Screening tool for sepsis OUTSIDE ICU (RR ≥22, altered mentation, SBP ≤100); NOT a severity score; sensitivity 29-51% but high specificity; should NOT replace SOFA in ICU (PMID: 26903335)

4. SAPS II (1993): 17 variables; simpler than APACHE; European-developed; significant calibration drift in modern era (PMID: 8255108)

5. SAPS III (2005): Admission-based (within 1 hour); regional customization; better discrimination than SAPS II; includes pre-ICU factors (PMID: 16132892)

6. ANZROD (2013): Australian/NZ recalibrated APACHE III model; accounts for local case-mix and practice patterns; preferred for Australian/NZ benchmarking; updated annually by ANZICS-CORE (PMID: 23340837)

7. SMR (Standardized Mortality Ratio): Observed deaths/Expected deaths; SMR < 1.0 = better than expected; SMR > 1.0 = worse than expected; requires funnel plot interpretation for statistical validity (PMID: 15568106)

8. Calibration vs Discrimination: Discrimination = ability to distinguish survivors from non-survivors (c-statistic); Calibration = agreement between predicted and observed mortality across risk strata (Hosmer-Lemeshow test)

9. Quality Indicators: Mortality is insufficient alone; process measures (VAP bundle compliance), structure measures (nurse:patient ratio), and patient-centered outcomes (functional status, quality of life) all important

10. Indigenous Health Disparities: Aboriginal/Torres Strait Islander and Maori patients have higher ICU admission rates, higher severity scores, and worse outcomes after risk adjustment; cultural competence essential in care delivery

Memory Aids

APACHE II Variables Mnemonic (A-PACE-2-CHeW):

• A: Age points (44y, 0; 45-54, 2; 55-64, 3; 65-74, 5; ≥75, 6)
• P: Physiology - Temperature, MAP, HR, RR, Oxygenation, Arterial pH
• A: Acute variables - Sodium, Potassium, Creatinine, Hematocrit
• C: Consciousness (GCS)
• E: Extreme values (worst in 24h)
• 2: 2 chronic health categories (severe organ insufficiency, immunocompromised)
• CHeW: Chronic Health Evaluation Weight (medical 5pts, emergency surgery 5pts, elective surgery 2pts)

SOFA Mnemonic (Six Organs Fail Always):

• R: Respiratory (PaO2/FiO2)
• C: Cardiovascular (MAP and vasopressors)
• L: Liver (Bilirubin)
• C: Coagulation (Platelets)
• R: Renal (Creatinine/UO)
• N: Neurological (GCS)

qSOFA Mnemonic (RAS):

• R: Respiratory rate ≥22
• A: Altered mentation
• S: Systolic BP ≤100

---

Definition and Epidemiology

Definition

ICU Quality Metrics encompass severity scoring systems, outcome prediction models, and performance indicators used to quantify patient illness severity, predict outcomes, and benchmark ICU performance.

Key Terminology:

Severity Scoring System Categories:

Epidemiology

Global Use of Severity Scoring:

Severity scoring systems are used in virtually all ICUs worldwide for:

• Quality assurance: 95% of high-income country ICUs use severity scoring
• Research stratification: 100% of major ICU RCTs use severity scores for baseline comparison
• Benchmarking programs: Over 80 countries participate in international ICU registries

Australian/NZ Data (ANZICS-CORE APD):

The ANZICS Centre for Outcome and Resource Evaluation (CORE) maintains the Adult Patient Database (APD), one of the world's largest ICU registries:

• Coverage: >200 ICUs across Australia and New Zealand
• Annual admissions: >200,000 ICU admissions per year
• Data quality: >95% data completeness for core variables
• ANZROD calculation: All participating ICUs receive quarterly SMR reports

ANZICS APD Annual Report Key Statistics (2022-2023):

• Median APACHE III score: 49 (IQR 35-68)
• Overall ICU mortality: 7.8%
• Overall hospital mortality: 11.2%
• Median ICU LOS: 1.7 days (IQR 0.9-3.4)
• Mechanical ventilation rate: 38%
• Vasopressor use: 31%
• RRT rate: 7%

Risk-Adjusted Mortality Over Time:

Australian/NZ ICU outcomes have improved substantially over time. ANZICS-CORE data demonstrates:

• 1995-2000: Baseline SMR = 1.0 (reference period)
• 2000-2010: 20-25% reduction in risk-adjusted mortality
• 2010-2020: Additional 10-15% reduction
• Current SMR: Mean 0.85-0.90 across Australian/NZ ICUs

Indigenous Health Disparities:

Aboriginal and Torres Strait Islander patients and Maori patients experience significant disparities in ICU outcomes:

• Admission rates: 2-3 fold higher age-standardized ICU admission rates (PMID: 26631103)
• Severity at admission: Higher APACHE III scores reflecting later presentation and higher disease burden
• Risk-adjusted mortality: 10-20% higher hospital mortality after adjustment for severity
• Long-term outcomes: Higher post-discharge mortality at 1 year

Contributing factors to Indigenous health disparities:

• Geographic access barriers to tertiary care
• Higher rates of chronic disease (diabetes, renal disease, cardiovascular disease)
• Socioeconomic disadvantage
• Cultural and language barriers affecting healthcare engagement
• Historical trauma affecting health-seeking behavior

---

Applied Basic Sciences

Statistical Foundations of Severity Scoring

Logistic Regression:

Most ICU outcome prediction models use logistic regression, which predicts the probability of a binary outcome (death/survival) based on predictor variables:

logit(p) = ln(p/(1 - p)) = beta0 + beta1 x1 + beta2 x2 + ... + betan xn

Where:

• p = probability of death
• β₀ = intercept
• βᵢ = regression coefficients for each predictor variable xᵢ

Discrimination (c-statistic/AUC):

The c-statistic (concordance statistic, equivalent to area under ROC curve) measures how well the model distinguishes survivors from non-survivors:

• c = 0.5: No better than random chance
• c = 0.7-0.8: Acceptable discrimination
• c = 0.8-0.9: Good discrimination
• c > 0.9: Excellent discrimination

APACHE II/III/IV typical c-statistic: 0.84-0.88 SAPS II/III typical c-statistic: 0.82-0.87 SOFA (admission) typical c-statistic: 0.75-0.82

Calibration (Hosmer-Lemeshow Test):

Calibration assesses agreement between predicted and observed mortality across risk strata:

• Patients divided into deciles of predicted risk
• Observed mortality compared to predicted mortality in each decile
• Hosmer-Lemeshow test: χ² test for goodness of fit (p > 0.05 indicates adequate calibration)
• Calibration plot: Visual comparison of predicted vs observed mortality

Calibration Drift:

All prediction models experience calibration drift over time:

• ICU outcomes improve faster than models predict
• APACHE II (1985) significantly overestimates mortality in 2024
• SAPS II (1993) overestimates mortality by 20-40% in modern ICUs
• Requires periodic recalibration or development of new models

Physiology of Organ Dysfunction

Oxygen Delivery and Consumption:

The physiological variables in severity scores reflect fundamental determinants of tissue oxygen delivery:

DO_2 = CO \times CaO_2 = CO \times (1.34 \times Hb \times SaO_2 + 0.003 \times PaO_2)

Where:

• DO₂ = oxygen delivery
• CO = cardiac output
• CaO₂ = arterial oxygen content
• Hb = hemoglobin concentration
• SaO₂ = arterial oxygen saturation
• PaO₂ = arterial oxygen tension

Multiple Organ Dysfunction Syndrome (MODS):

SOFA scoring reflects the pathophysiology of organ dysfunction in critical illness:

1. Respiratory failure: V/Q mismatch, shunt, diffusion impairment → decreased PaO₂/FiO₂
2. Cardiovascular failure: Vasodilatory shock, myocardial depression → hypotension requiring vasopressors
3. Hepatic dysfunction: Cholestasis, hepatocellular injury → elevated bilirubin
4. Coagulopathy: Consumptive, dilutional, bone marrow suppression → thrombocytopenia
5. Renal failure: ATN, prerenal azotemia, obstruction → elevated creatinine, oliguria
6. Neurological dysfunction: Encephalopathy, sedation effects → decreased GCS

Inflammatory Cascade:

Critical illness triggers a systemic inflammatory response:

• DAMPs/PAMPs → Pattern recognition receptor activation
• Cytokine release: TNF-α, IL-1β, IL-6, IL-8
• Endothelial activation: Increased permeability, procoagulant state
• Microcirculatory dysfunction: Heterogeneous flow, tissue hypoxia
• Organ dysfunction: Progressive failure of multiple organ systems

Compensatory Anti-inflammatory Response (CARS):

Following initial hyperinflammation:

• IL-10, TGF-β release
• Immunoparalysis: Reduced HLA-DR expression on monocytes
• Susceptibility to secondary infections
• Persistent inflammation, immunosuppression, catabolism syndrome (PICS)

Pharmacology Relevant to Scoring

Vasopressors in SOFA Calculation:

The cardiovascular component of SOFA includes vasopressor dose:

Vasoactive-Inotropic Score (VIS):

Alternative to SOFA CV component for pediatric and cardiac surgery patients:

VIS = Dopamine + Dobutamine + (Milrinone \times 10) + (Vasopressin \times 10,000) + (Adrenaline \times 100) + (Noradrenaline \times 100)

(All doses in μg/kg/min except vasopressin in units/kg/min)

Sedation Effects on GCS:

GCS in APACHE and SOFA is confounded by sedation:

• Best practice: Use pre-sedation GCS or GCS after sedation cessation
• Imputed GCS: Use admission GCS if patient sedated before ICU
• Statistical methods: Multiple imputation for missing neurological data

---

Clinical Presentation

Scoring System Applications in ICU

Admission Severity Assessment:

Scenario 1: Septic Shock

A 68-year-old male admitted with community-acquired pneumonia and septic shock:

• Vitals: HR 125, BP 75/45 (on noradrenaline 0.2 μg/kg/min), RR 28, SpO₂ 88% on FiO₂ 0.6, T 39.2°C
• Labs: WCC 22, Plt 85, Cr 245 (baseline 90), Bili 35, Lactate 4.5
• ABG: pH 7.28, PaCO₂ 28, PaO₂ 65, HCO₃ 14
• GCS: E3V4M6 (13/15)

APACHE II Calculation:

• Temperature (39.2): +1
• MAP (~55): +4
• Heart Rate (125): +3
• RR (28): +3
• PaO₂ (65 on FiO₂ 0.6): +2
• pH (7.28): +3
• Sodium (assume 138): 0
• Potassium (assume 4.2): 0
• Creatinine (245, acute): +2
• Hematocrit (assume 32): +1
• WCC (22): +2
• GCS (13): +2 (15 - 13)
• Age (68): +5
• Chronic health (none): 0
• Total APACHE II: 28 (predicted mortality ~50%)

SOFA Calculation:

• Respiratory (PaO₂/FiO₂ = 108): 3
• Cardiovascular (noradrenaline 0.2): 4
• Hepatic (bilirubin 35): 1
• Coagulation (platelets 85): 2
• Renal (creatinine 245): 2
• Neurological (GCS 13): 1
• Total SOFA: 13 (mortality 40-60%)

Scenario 2: Post-Cardiac Arrest

A 55-year-old female admitted post-VF cardiac arrest with ROSC after 20 minutes:

• Comatose (GCS 3T), intubated and ventilated
• On noradrenaline 0.15 μg/kg/min
• Targeted temperature management at 33°C
• Lactate 6.2, Cr 180

Scoring challenges:

• GCS confounded by sedation and TTM
• Lactate may improve rapidly with reperfusion
• Early scoring may not reflect neurological prognosis
• Disease-specific scores (OHCA, CAHP) more appropriate

Interpreting Score Trajectories

Serial SOFA Monitoring:

Interpretation:

• Day 1-2: Worsening (peak SOFA 14)
• Day 3-4: Improving trajectory
• Delta SOFA (Day 1 to peak): +1 (modest worsening)
• Delta SOFA (peak to Day 4): -9 (significant improvement)

Prognostic Value of SOFA Trajectory (PMID: 9824069):

• Increase in SOFA over 48-96 hours: mortality 50-60%
• Stable SOFA: mortality 35-45%
• Decrease in SOFA: mortality 10-25%

---

APACHE Scoring Systems

APACHE II (1985)

Development (PMID: 3928249):

• Developed by Knaus et al. at George Washington University
• 13 US hospitals, 5,815 ICU admissions
• Published in Critical Care Medicine 1985
• Most widely used and cited ICU severity score globally

Components (12 Acute Physiology Variables + Age + Chronic Health):

*If FiO₂ ≥0.5: use A-aDO₂; if FiO₂ <0.5: use PaO₂ (kPa) **Double points if acute renal failure

Age Points:

Chronic Health Points: Severe organ system insufficiency or immunocompromised state:

• Elective post-operative: 2 points
• Emergency post-operative or non-operative: 5 points

Mortality Prediction Equation: ln(p/(1 - p)) = -3.517 + (0.146 x APACHE II) + diagnostic weight

Limitations of APACHE II:

• Developed in 1985 - significant calibration drift
• Overestimates mortality by 20-40% in modern ICUs
• Limited adjustment for pre-ICU factors
• Single diagnostic category insufficient for case-mix
• Lead-time bias if scored before resuscitation complete

APACHE III (1991)

Improvements over APACHE II (PMID: 1959406):

• Larger development cohort (17,440 patients, 40 US hospitals)
• 17 physiological variables (vs 12 in APACHE II)
• More granular scoring (0-252 APS points)
• 78 diagnostic categories with specific weights
• Improved discrimination (c-statistic 0.89 vs 0.85)

Key Differences from APACHE II:

• Includes albumin, bilirubin, glucose, BUN
• Separate acid-base scoring (pH and PaCO₂)
• More detailed chronic health evaluation
• Source of ICU admission (emergency, operating room, floor, other hospital)
• Time-based recalibration equations

APACHE III Score Range: 0-299

APACHE IV (2006)

Current Standard (PMID: 16465624):

• Developed from 110,558 patients across 104 US ICUs
• 142 diagnosis-specific equations
• Includes mechanical ventilation on Day 1
• Separate equations for ICU and hospital mortality
• Recalibration for contemporary practice

Key Features:

• Score range: 0-286
• Diagnosis-specific SMR calculations
• Online calculator available
• c-statistic: 0.88
• Better calibration than APACHE II/III

Variables (same as APACHE III plus):

• Mechanical ventilation status
• Thrombolytic therapy
• FiO₂ for PaO₂/FiO₂ calculation
• Pre-ICU length of stay

---

SOFA Score

Development and Components

Original Description (PMID: 8844239):

• Developed by Vincent et al. for the European Society of Intensive Care Medicine
• Published 1996 as Sepsis-related Organ Failure Assessment
• Later renamed Sequential Organ Failure Assessment (applicable beyond sepsis)
• Designed to describe organ dysfunction trajectory, not predict mortality

SOFA Components:

DA = dopamine (μg/kg/min); DBT = dobutamine; NA = noradrenaline; A = adrenaline (μg/kg/min)

SOFA Score Range: 0-24 (maximum 4 per organ × 6 organs)

Serial SOFA and Delta SOFA

Serial SOFA Assessment (PMID: 9824069):

• Calculate SOFA daily (or more frequently in unstable patients)
• Track individual organ components and total score
• Identify deteriorating or improving trends

Delta SOFA: Delta SOFA = SOFA_48h - SOFA_admission

Prognostic Value:

Maximum SOFA:

• Highest SOFA score during ICU stay
• Stronger mortality predictor than admission SOFA
• Reflects overall illness trajectory

qSOFA (Quick SOFA)

Sepsis-3 Definition (PMID: 26903335):

• Developed for bedside screening OUTSIDE the ICU
• NOT for use in ICU (all ICU patients would be "positive")
• NOT a severity or mortality prediction tool

qSOFA Components (≥2 positive = high risk):

1. Respiratory rate ≥22 breaths/min
2. Altered mentation (GCS <15)
3. Systolic blood pressure ≤100 mmHg

Performance Characteristics:

• Sensitivity for mortality: 29-51%
• Specificity for mortality: 70-90%
• NPV: 97%
• PPV: 20-30%

Clinical Use:

• Quick bedside assessment in ED, ward, pre-hospital
• Identifies patients who may benefit from ICU review
• Does NOT replace clinical judgment
• Does NOT replace formal sepsis criteria

Controversy (PMID: 28036471):

• Low sensitivity means many sepsis patients will be missed
• Surviving Sepsis Campaign recommends against qSOFA as single screening tool
• SIRS criteria still have role in sepsis identification
• Local validation and thresholds recommended

---

SAPS Scoring Systems

SAPS II (1993)

Development (PMID: 8255108):

• Le Gall et al., European/North American multicenter study
• 13,152 patients from 137 ICUs in 12 countries
• Designed to be simpler than APACHE with equivalent performance

Components (17 variables):

• 12 physiological variables (worst in first 24h)
• Age
• Type of admission (scheduled surgical, unscheduled surgical, medical)
• 3 chronic diseases (AIDS, metastatic cancer, hematological malignancy)

Score Range: 0-163

Mortality Prediction: logit(p) = -7.7631 + 0.0737 x SAPS II + 0.9971 x ln(SAPS II + 1)

Calibration Issues:

• Significant overestimation of mortality in modern ICUs
• 20-40% overestimation in some cohorts (PMID: 20121762)
• Does not account for practice improvements since 1993

SAPS III (2005)

Development (PMID: 16132892, 16132893):

• Moreno et al., global multicenter study
• 16,784 patients from 303 ICUs across 5 continents
• Published in Intensive Care Medicine 2005

Key Innovations:

• Admission-based: Variables collected within 1 hour of ICU admission
• Three-part structure: Pre-ICU (Box I), ICU admission (Box II), Physiology (Box III)
• Regional customization: Equations for Australasia, Central/South America, Central/Western Europe, Eastern Europe, North Europe, Southern Europe/Mediterranean

Components:

Box I - Pre-ICU Variables (20 points max):

• Age, chronic comorbidities, days in hospital pre-ICU
• Location before ICU, surgical status
• Planned/unplanned admission

Box II - Admission Reason (13 points max):

• Primary admission diagnosis
• Reason for ICU admission (cardiovascular, neurological, etc.)
• Acute infection at admission

Box III - Physiology (124 points max):

• GCS, bilirubin, body temperature
• Creatinine, heart rate, leukocytes
• Hydrogen ion concentration, platelets
• Systolic BP, oxygenation

Score Range: 16-217

Performance:

• c-statistic: 0.83-0.86
• Better calibration than SAPS II
• Regional equations improve local performance

Australasian Equation: The SAPS III Australasian equation was developed from 2,251 patients from 19 ANZ ICUs and shows superior calibration compared to global equations in Australian/NZ populations.

---

ANZROD Model

Development and Validation

Australian and New Zealand Risk of Death (ANZROD) (PMID: 23340837):

• Developed by Paul et al. from ANZICS-CORE
• Published in Critical Care Medicine 2013
• Based on ANZICS Adult Patient Database (APD)
• Recalibrated APACHE III model for local case-mix

Development Cohort:

• 296,485 ICU admissions from 2009-2010
• 131 ANZ ICUs participating in ANZICS APD
• Validation on 2011 cohort (149,877 admissions)

Key Features:

• Uses APACHE III physiological variables
• Australian-specific diagnostic weights
• Indigenous status adjustment
• Includes pre-ICU hospital length of stay
• Updated annually by ANZICS-CORE

Performance:

• Discrimination (c-statistic): 0.89-0.91
• Calibration: Excellent (Hosmer-Lemeshow p > 0.05)
• Superior to APACHE II, APACHE IV, SAPS II/III in ANZ

ANZROD Variables

Predictor Variables:

1. APACHE III APS (Acute Physiology Score)
2. Age
3. Chronic health evaluation
4. Emergency admission
5. Source of ICU admission
6. Mechanical ventilation (first 24h)
7. Pre-ICU hospital length of stay
8. 140+ diagnosis-specific weights
9. Indigenous status

Indigenous Status Coefficient:

• Aboriginal/Torres Strait Islander or Maori status included
• Reflects observed outcome differences after physiological adjustment
• Controversial inclusion but improves model calibration
• Not intended for individual prognostication

ANZROD vs International Models

Recommendation for Australian/NZ Practice: ANZROD is the preferred model for:

• SMR calculations and ICU benchmarking
• ANZICS-CORE annual reports
• State/territory ICU performance monitoring
• Research stratification in ANZ populations

---

Standardized Mortality Ratio and Benchmarking

SMR Calculation

Definition: SMR = Observed Deaths / Expected Deaths = O / E

Where:

• Observed deaths = actual hospital deaths in the period
• Expected deaths = sum of individual predicted mortality probabilities

Interpretation:

95% Confidence Interval: CI = SMR +/- 1.96 x sqrt(SMR / E)

Funnel Plots

Concept (PMID: 15568106):

• Developed by Spiegelhalter for institutional performance comparison
• X-axis: Case volume (number of admissions or expected deaths)
• Y-axis: SMR
• Control limits (95%, 99.8%) narrow as volume increases

Interpretation:

• Points within funnel: Common-cause variation (expected)
• Points outside 95% limits: Warrant review ("alerts")
• Points outside 99.8% limits: Statistical outliers ("alarms")

Advantages over League Tables:

• Accounts for precision (small ICUs have wider confidence intervals)
• Prevents over-interpretation of outliers with small numbers
• Shows statistical significance, not just ranking
• Identifies both high and low performers

ANZICS-CORE Quarterly Reports:

• All participating ICUs receive SMR funnel plots
• Peer comparison within ICU type (tertiary, metropolitan, rural)
• Trend analysis over time (rolling 12-month SMR)

Quality Indicators Beyond Mortality

Structure Indicators:

• Nurse:patient ratio (1:1 for ventilated patients)
• Intensivist-led model of care
• 24/7 intensivist availability
• Access to allied health services
• ICU bed capacity and occupancy

Process Indicators:

• Central line insertion bundle compliance
• VAP prevention bundle compliance
• DVT prophylaxis rates
• Early enteral nutrition rates
• Daily sedation interruption compliance
• Antimicrobial stewardship adherence

Outcome Indicators:

• Risk-adjusted mortality (SMR)
• CLABSI and CAUTI rates
• VAP rates
• Unplanned extubation rates
• Pressure injury rates
• ICU readmission rates
• Duration of mechanical ventilation

Patient-Centered Outcomes:

• Health-related quality of life (SF-36, EQ-5D)
• Functional status at hospital discharge
• Return to work rates
• Post-ICU syndrome (PICS) rates
• Patient and family satisfaction

ANZICS Quality Indicator Set: ANZICS recommends standardized quality indicators for benchmarking:

1. Risk-adjusted hospital mortality (ANZROD SMR)
2. ICU length of stay
3. After-hours discharge rate
4. ICU readmission rate
5. CLABSI rate
6. VAP rate

---

Indigenous Health Considerations

Health Disparities in ICU

Aboriginal and Torres Strait Islander Peoples:

• ICU admission rates: 2-3× higher age-standardized rates (PMID: 26631103)
• Younger age at admission: Mean 10-15 years younger than non-Indigenous
• Higher severity: Higher APACHE III scores at admission
• Comorbidity burden: Higher rates of diabetes, ESKD, cardiovascular disease
• Risk-adjusted outcomes: 10-20% higher mortality after adjustment

Maori Population (New Zealand):

• Similar disparities to Australian Indigenous populations
• Higher rates of sepsis, trauma, and chronic disease complications
• Younger age at ICU admission
• Importance of whanau (family) involvement in care decisions

Scoring System Considerations

ANZROD Indigenous Adjustment:

• Indigenous status is a predictor variable in ANZROD
• Reflects observed higher mortality after physiological adjustment
• Improves model calibration but raises ethical questions
• Should NOT be used for individual triage or prognostication

Potential Biases:

• Lead-time bias: Later presentation means worse physiology at admission
• Access bias: Rural/remote populations may have delayed ICU access
• Socioeconomic factors: Not fully captured in physiological scoring
• Healthcare system factors: May receive less intensive treatment

Cultural Safety in Scoring Use:

• Never use scoring systems to deny care based on race
• Recognize limitations of population-based predictions for individuals
• Involve Aboriginal Health Workers/Liaison Officers in goals of care discussions
• Respect cultural protocols around death and dying
• Ensure interpreter services for non-English speaking families

Quality Improvement for Indigenous Health

Recommended Strategies:

1. Track Indigenous-specific outcomes: Stratify SMR by Indigenous status
2. Cultural competency training: Mandatory for all ICU staff
3. Aboriginal Health Worker integration: Part of multidisciplinary team
4. Community engagement: Partner with Indigenous health organizations
5. Access improvement: Telehealth, retrieval services, regional ICU support

ANZICS Position Statement:

• Commitment to reducing Indigenous health disparities
• Support for culturally appropriate end-of-life care
• Recognition of community and family decision-making
• Advocacy for equitable access to ICU care

---

Investigations for Quality Assessment

Data Collection Requirements

APACHE II Data Collection:

• Worst values in first 24 hours of ICU admission
• GCS: Pre-sedation or after adequate washout period
• Arterial blood gas (pH, PaO₂, PaCO₂)
• Full blood count, electrolytes, creatinine
• Temperature (highest or lowest if abnormal)
• Chronic health conditions from medical record

APACHE III/IV Additional Data:

• Albumin, bilirubin, glucose, BUN
• Source of ICU admission
• Pre-ICU length of stay
• Specific admission diagnosis (140+ categories)

SOFA Data Collection:

• PaO₂/FiO₂ ratio (worst in 24h)
• Vasopressor doses (maximum in 24h)
• Bilirubin
• Platelet count
• Creatinine and urine output
• GCS (neurological component)

Data Quality and Validation

ANZICS APD Data Quality Standards:

• Minimum 95% completeness for core variables
• Automated range checks and alerts
• Regular data quality audits
• Inter-hospital data sharing agreements
• Standardized diagnostic coding (APACHE IV codes)

Common Data Quality Issues:

• Missing pre-sedation GCS
• Incorrect creatinine (baseline vs acute)
• Timing errors (values outside 24h window)
• Diagnostic coding errors
• Failure to capture vasopressor doses

Validation Studies:

• Regular comparison of electronic vs manual data
• External validation of SMR calculations
• Audit of diagnostic coding accuracy
• Comparison of predicted vs observed mortality

---

ICU Management Implications

Using Scores Appropriately

Appropriate Uses:

1. Benchmarking: Comparing ICU performance using SMR
2. Research stratification: Baseline severity matching in RCTs
3. Case-mix adjustment: Explaining outcome variation
4. Quality improvement: Tracking outcomes over time
5. Resource planning: Workload and staffing estimation

Inappropriate Uses:

1. Individual prognostication: "This patient has a 50% chance of survival"
2. Triage decisions: Withholding ICU admission based on score
3. Futility judgments: Withdrawing care based on predicted mortality
4. Resource rationing: Prioritizing patients by predicted outcome
5. League tables: Ranking ICUs without statistical context

Communication with Families

Discussing Prognosis:

• Explain that scores are population-based estimates
• Clarify that individual outcomes can differ substantially
• Use ranges rather than point estimates ("patients with similar severity have a mortality of 30-50%")
• Acknowledge uncertainty in prediction
• Avoid therapeutic nihilism based on scores alone

Example Communication:

"Your husband's illness is very serious. We use scoring systems to help us understand how sick patients are. His scores tell us that patients with similar severity have around a 40% chance of dying in hospital. However, this doesn't tell us what will happen to him specifically - some patients do better, some do worse. We'll know more about his individual situation over the next few days as we see how he responds to treatment."

Quality Improvement Applications

Using SMR for Improvement:

1. Identify outlier performance: SMR outside funnel limits
2. Investigate root causes: Case review, process mapping
3. Implement changes: PDSA cycles for improvement
4. Monitor impact: Track SMR trend over time
5. Sustain improvements: Standardize successful interventions

Process Measure Tracking:

• Link structure/process measures to outcomes
• Identify high-impact improvement opportunities
• Prioritize interventions with strongest evidence base
• Celebrate successes and share learning

---

Monitoring and Complications

Score Reliability and Validity

Inter-rater Reliability:

• APACHE II: Moderate to good (κ = 0.6-0.8)
• SOFA: Good (κ = 0.7-0.9)
• Variability mainly in GCS and diagnosis coding

Sources of Measurement Error:

• Timing of data collection
• Lead-time bias (pre-resuscitation vs post-resuscitation)
• Sedation effects on GCS
• Missing data imputation
• Diagnostic coding accuracy

Limitations and Pitfalls

Model Limitations:

Statistical Pitfalls:

• Regression to the mean: Extreme SMRs often return to normal
• Multiplicity: Testing many ICUs increases false positives
• Confounding: Unmeasured factors affect outcomes
• Selection bias: Admission policies affect case-mix
• Survivor bias: Only surviving hospitals in databases

---

Prognosis and Outcome Measures

Mortality Prediction Accuracy

APACHE II Predicted vs Observed Mortality:

SOFA Score and Mortality:

Long-Term Outcomes

Post-ICU Mortality:

• 90-day mortality: 1.5-2× hospital mortality
• 1-year mortality: 15-30% for ICU survivors
• 5-year mortality: 30-50% for severe sepsis survivors

Functional Outcomes:

• Return to work at 1 year: 40-60%
• Return to baseline function at 6 months: 30-50%
• PICS (Post-Intensive Care Syndrome): 50-70% of survivors

Quality of Life:

• SF-36 scores 70-80% of age-matched controls at 1 year
• Significant cognitive impairment in 20-40%
• PTSD symptoms in 10-30%
• Depression/anxiety in 30-50%

---

SAQ Practice Questions

SAQ 1: APACHE II Scoring and SMR Interpretation

Time Allocation: 10 minutes Total Marks: 20

Stem: You are the ICU Director reviewing your unit's quarterly quality report. Your ICU had 450 admissions last quarter with an observed hospital mortality of 11.5% (52 deaths). The expected mortality based on ANZROD was 8.5% (38.25 expected deaths). Your SMR is 1.36 (95% CI 1.02-1.78).

Question 1.1 (8 marks) Outline the components of APACHE II scoring and explain how it is used to calculate expected mortality.

Question 1.2 (6 marks) Interpret the SMR result and describe how funnel plots are used in ICU benchmarking.

Question 1.3 (6 marks) Outline your approach to investigating this result.

---

Model Answer

Question 1.1 (8 marks total)

APACHE II Components (4 marks):

• Acute Physiology Score (APS): 12 physiological variables using worst values in first 24 hours of ICU admission (1 mark)
  • Temperature, MAP, heart rate, respiratory rate
  • Oxygenation (A-aDO₂ or PaO₂), arterial pH
  • Sodium, potassium, creatinine, hematocrit, WCC
  • GCS (15 minus actual GCS)
• Age points: Ranging from 0 (≤44 years) to 6 (≥75 years) (0.5 mark)
• Chronic health points: 2 points for elective surgery, 5 points for emergency surgery or medical admission with severe organ insufficiency or immunocompromised state (0.5 mark)

Calculation of Expected Mortality (4 marks):

• Each patient's APACHE II score is converted to predicted mortality probability using logistic regression equation (1 mark)
• Equation: logit(p) = -3.517 + (0.146 × APACHE II) + diagnostic weight (1 mark)
• Expected deaths = sum of all individual predicted mortality probabilities (1 mark)
• Limitations: APACHE II developed in 1985, significant calibration drift in modern ICUs; ANZROD preferred for Australian/NZ benchmarking (1 mark)

Question 1.2 (6 marks total)

SMR Interpretation (3 marks):

• SMR 1.36 means observed deaths are 36% higher than expected based on case-mix adjustment (1 mark)
• 95% CI 1.02-1.78: The lower bound exceeds 1.0, suggesting statistically significant worse-than-expected performance (1 mark)
• However, clinical significance requires investigation - this could represent true quality issues, case-mix not captured by the model, or random variation (1 mark)

Funnel Plots (3 marks):

• Funnel plots display SMR (y-axis) against case volume (x-axis) (1 mark)
• Control limits (95%, 99.8%) narrow as volume increases, forming a funnel shape (1 mark)
• Accounts for precision: small ICUs have wider expected variation; larger ICUs' deviations from SMR 1.0 are more likely to represent true differences rather than chance (1 mark)

Question 1.3 (6 marks total)

Investigation Approach (6 marks):

1. Data quality review (1 mark): Verify accuracy of severity scoring and diagnostic coding; check for missing or erroneous data that might affect expected mortality calculation
2. Case review (1.5 marks): Review all deaths, particularly unexpected deaths (patients with low predicted mortality who died); look for preventable factors, systems issues
3. Subgroup analysis (1 mark): Examine SMR by diagnostic category, admission source, time of admission; identify specific areas of concern
4. Process measure review (1 mark): Assess compliance with evidence-based bundles (sepsis, VAP prevention, CLABSI); identify process gaps
5. Peer comparison (0.5 marks): Compare with similar ICUs (tertiary vs metropolitan vs rural); context from ANZICS peer reports
6. Action planning (1 mark): If true quality issue identified, implement targeted improvement initiatives with ongoing monitoring

---

SAQ 2: SOFA Score and qSOFA Application

Time Allocation: 10 minutes Total Marks: 20

Stem: A 58-year-old woman is admitted to your ICU with community-acquired pneumonia and septic shock. On admission, she is intubated and ventilated with FiO₂ 0.7, PaO₂ 75 mmHg. She is on noradrenaline 0.25 μg/kg/min with MAP 65 mmHg. Bilirubin is 45 μmol/L, platelets 65 × 10⁹/L, creatinine 320 μmol/L with urine output 15 mL/hr. Pre-sedation GCS was 12.

Question 2.1 (8 marks) Calculate and interpret the SOFA score.

Question 2.2 (6 marks) Explain the role of qSOFA in sepsis identification and discuss its limitations.

Question 2.3 (6 marks) Outline how serial SOFA scoring can be used to monitor this patient's progress.

---

Model Answer

Question 2.1 (8 marks total)

SOFA Score Calculation (6 marks):

Interpretation (2 marks):

• SOFA ≥2 represents clinically significant organ dysfunction consistent with Sepsis-3 definition (1 mark)
• SOFA 15-16 associated with mortality 40-60%; indicates severe multi-organ dysfunction requiring aggressive ICU support (1 mark)

Question 2.2 (6 marks total)

qSOFA Role (3 marks):

• Quick SOFA (qSOFA) developed for Sepsis-3 as bedside screening tool OUTSIDE the ICU (1 mark)
• Components: Respiratory rate ≥22, altered mentation (GCS <15), systolic BP ≤100 mmHg (1 mark)
• Intended for rapid identification of patients at risk of poor outcomes in ED, ward, pre-hospital settings who may benefit from ICU review (1 mark)

Limitations (3 marks):

• Low sensitivity (29-51%): Many sepsis patients will have qSOFA <2 (1 mark)
• Not validated as mortality predictor or severity score in ICU population where most patients would meet criteria (1 mark)
• Surviving Sepsis Campaign recommends against using qSOFA as single screening tool; SIRS criteria and clinical suspicion remain important (1 mark)

Question 2.3 (6 marks total)

Serial SOFA Application (6 marks):

1. Baseline SOFA: Admission score (15-16) provides reference point for trajectory assessment (1 mark)
2. Daily calculation: Recalculate SOFA every 24 hours (or more frequently if unstable) to track organ dysfunction trajectory (1 mark)
3. Delta SOFA: Change from baseline; increase ≥2 points within 48 hours associated with significantly higher mortality (1 mark)
4. Individual organ tracking: Identify which organ systems are improving or deteriorating to guide interventions (1 mark)
5. Prognostic value: Improving SOFA over 48-96 hours associated with mortality <10%; worsening SOFA associated with mortality >50% (1 mark)
6. Treatment response assessment: Use to evaluate response to source control, antimicrobials, organ support; failure to improve may prompt escalation or reassessment (1 mark)

---

Viva Scenarios

Viva 1: APACHE and ANZROD

Stem: "You are the ICU Director preparing for a meeting with hospital executives about your unit's performance. The latest ANZICS-CORE report shows your SMR is 1.22."

Duration: 12 minutes (2 min reading + 10 min discussion)

---

Opening Question: "Tell me about the ANZROD model and why it is preferred over APACHE for Australian ICU benchmarking."

Expected Answer:

ANZROD Development:

• Australian and New Zealand Risk of Death model developed by Paul et al. in 2013
• Based on ANZICS Adult Patient Database (APD) with nearly 300,000 patient cohort
• Recalibrated APACHE III model specifically for Australian and New Zealand populations

Advantages over APACHE:

• Uses local case-mix with Australian-specific diagnostic weights
• Updated annually by ANZICS-CORE to maintain calibration
• Includes Indigenous status adjustment (Aboriginal/Torres Strait Islander, Maori)
• Accounts for pre-ICU length of stay and source of admission
• Superior discrimination (c-statistic 0.90) and calibration in ANZ populations
• APACHE II/III developed in 1980s-1990s USA with significant calibration drift

Why it matters:

• SMR calculated using ANZROD reflects true performance differences rather than model miscalibration
• Allows meaningful comparison between Australian/NZ ICUs with different case-mix
• Forms basis of ANZICS-CORE quarterly reporting used by all participating ICUs

---

Follow-up Question 1: "Your SMR of 1.22 means you have 22% more deaths than expected. How would you explain this to the hospital executive?"

Expected Answer:

Communication Approach:

• SMR is a risk-adjusted outcome measure: we're comparing observed deaths to expected deaths based on how sick patients were on admission
• SMR 1.22 means we had 22% more deaths than predicted by the model
• Important context: need to know confidence interval and where we sit on the funnel plot
• If confidence interval includes 1.0, may represent random variation
• If statistically significant, warrants investigation but doesn't necessarily mean poor care

Factors to Consider:

• Case-mix not captured by model (e.g., new service line, sicker patients from region)
• Data quality issues (incorrect severity scoring, diagnostic coding errors)
• Genuine quality issues requiring investigation and improvement
• Temporal trends (is this one quarter or persistent pattern?)

Next Steps:

• Review data quality and coding accuracy
• Conduct case review of unexpected deaths
• Subgroup analysis by diagnosis, time, admission source
• Compare to peer ICUs
• Present findings and improvement plan to executive

---

Follow-up Question 2: "A colleague suggests using APACHE II scores to decide which patients should not be admitted to ICU. What is your response?"

Expected Answer:

Strong Opposition: Severity scores should NOT be used for individual triage or admission decisions:

Methodological Issues:

• Scores developed for population-level prediction, not individual prognostication
• Individual patients can survive despite high predicted mortality
• Confidence intervals for individual predictions are very wide
• Lead-time bias: score changes with resuscitation

Ethical Issues:

• Discriminatory: would disproportionately affect elderly, Indigenous, comorbid patients
• Self-fulfilling prophecy: denying ICU leads to death, confirming "prediction"
• Against medical ethics principles: beneficence, justice, non-maleficence
• ANZICS and CICM guidelines explicitly advise against score-based rationing

Appropriate Use:

• Benchmarking ICU performance
• Research stratification
• Quality improvement
• Informing family discussions (population-level context, not individual prediction)

---

Follow-up Question 3: "How would you consider Indigenous health disparities when interpreting your SMR data?"

Expected Answer:

Indigenous Health Disparities:

• Aboriginal/Torres Strait Islander and Maori patients have 2-3× higher age-standardized ICU admission rates
• Typically present younger with higher severity scores
• Higher rates of chronic disease complications (diabetes, ESKD, cardiovascular)
• After risk adjustment, still have 10-20% higher mortality

ANZROD Considerations:

• ANZROD includes Indigenous status as a predictor variable
• Improves calibration but raises ethical concerns
• Reflects observed outcomes, not biological difference
• Likely captures unmeasured socioeconomic and access factors

Quality Improvement Approach:

• Stratify SMR by Indigenous status to identify disparities
• Track outcomes separately for Indigenous patients
• Implement cultural competency training
• Integrate Aboriginal Health Workers in care team
• Partner with community organizations
• Address access barriers (retrieval, telehealth, regional support)
• Never use Indigenous status for individual prognostication or admission decisions

---

Viva 2: SOFA and Quality Indicators

Stem: "You are asked to develop a quality dashboard for your ICU. The Medical Director wants to move beyond just mortality as an outcome measure."

Duration: 12 minutes (2 min reading + 10 min discussion)

---

Opening Question: "Describe the role of SOFA scoring in ICU quality monitoring and how it differs from APACHE."

Expected Answer:

SOFA Characteristics:

• Sequential Organ Failure Assessment developed by Vincent 1996
• Six organ systems: respiratory, cardiovascular, hepatic, coagulation, renal, neurological
• Each system scored 0-4, maximum 24 points
• Designed for serial measurement to track trajectory

Difference from APACHE:

• APACHE: Admission-based severity score for mortality prediction
• SOFA: Daily organ dysfunction assessment for monitoring trajectory
• APACHE uses worst values in first 24 hours only
• SOFA recalculated daily (or more frequently) to track change
• APACHE includes age and chronic health; SOFA is purely physiological

Quality Applications:

• Serial SOFA: Track response to treatment
• Delta SOFA: Change predicts outcome (increase = worse prognosis)
• Maximum SOFA: Peak dysfunction during admission
• SOFA-free days: Composite outcome combining survival and organ function
• Can be used to compare treatments in RCTs (organ dysfunction as outcome)

---

Follow-up Question 1: "What quality indicators beyond mortality would you recommend for your ICU dashboard?"

Expected Answer:

Donabedian Framework - Structure, Process, Outcome:

Structure Indicators:

• Nurse:patient ratio (1:1 for ventilated)
• Intensivist-led care model
• 24/7 intensivist availability
• Allied health staffing (physio, OT, speech, dietitian)

Process Indicators (evidence-based care delivery):

• Central line insertion bundle compliance (hand hygiene, full barrier, chlorhexidine, optimal site, daily review)
• VAP prevention bundle (HOB elevation, oral care, SBT, sedation interruption)
• DVT prophylaxis rates
• Early enteral nutrition (within 24-48 hours)
• Antimicrobial stewardship (appropriate empiric therapy, de-escalation, duration)
• Daily goals of care documentation

Outcome Indicators:

• Risk-adjusted mortality (ANZROD SMR)
• CLABSI rate per 1,000 catheter-days
• CAUTI rate per 1,000 catheter-days
• VAP rate per 1,000 ventilator-days
• Unplanned extubation rate
• Pressure injury incidence
• ICU readmission rate (within 48-72 hours)

Patient-Centered Outcomes:

• Functional status at discharge
• Quality of life at 6-12 months (SF-36, EQ-5D)
• Return to work/baseline
• Patient and family satisfaction
• Post-ICU syndrome (PICS) screening

---

Follow-up Question 2: "How would you use statistical process control to monitor these quality indicators?"

Expected Answer:

Statistical Process Control (SPC):

• Developed by Shewhart for industrial quality improvement
• Distinguishes common-cause (inherent system variation) from special-cause variation (signals requiring action)

Control Charts:

• X-axis: Time (weeks, months)
• Y-axis: Quality measure (rate, proportion)
• Center line: Mean or median
• Control limits: Typically ±3 standard deviations (99.8% limits)

Run Chart Rules (signal of non-random variation):

• Shift: ≥6 consecutive points above or below median
• Trend: ≥5 consecutive points all increasing or decreasing
• Point outside control limits: Single outlier requiring investigation

Application Examples:

• CLABSI rate: p-chart for proportions
• Ventilator-free days: I-chart for continuous data
• Hand hygiene compliance: Run chart
• Mortality: G-chart for rare events

Response to Signals:

• Special-cause variation: Investigate specific cause and address
• Common-cause variation: Requires system redesign to improve

---

Follow-up Question 3: "A nurse asks you why you don't use qSOFA to screen for sepsis on the ward. How do you respond?"

Expected Answer:

qSOFA Overview:

• Quick SOFA developed with Sepsis-3 (2016) for bedside screening OUTSIDE ICU
• Components: RR ≥22, altered mentation, SBP ≤100
• ≥2 criteria positive suggests high risk of poor outcome

Limitations:

• Low sensitivity (29-51%): Majority of sepsis patients will have qSOFA <2
• Optimized for specificity to identify highest-risk patients
• Many patients with infection and organ dysfunction will be missed

Why Not as Sole Screen:

• Surviving Sepsis Campaign recommends AGAINST qSOFA as single screening tool
• SIRS criteria still valuable for identifying potential sepsis
• Clinical suspicion and comprehensive assessment remain essential
• Local early warning scores (NEWS, MET criteria) may be equally useful

Appropriate Use:

• Quick bedside tool for identifying very high-risk patients
• Prompts ICU review or senior clinical assessment
• Should be combined with SIRS, clinical judgment, and local protocols
• NOT a replacement for comprehensive sepsis screening programs