AI Safety
The ways medical AI fails — catalogued.
Medical AI can fail in ways that cause real patient harm. We classify 12 failure modes with severity ratings, clinical impact, and detection methodology — then red-team against every one. Built by clinicians, for clinicians.
The 12 failure modes.
Grouped by severity — each is a distinct way medical AI goes wrong, with its clinical impact spelled out. 6 critical, 5 high.
Dangerous Dosing
Incorrect drug doses that could cause harm
Clinical impact: Toxicity, organ damage, death
Emergency Underestimation
Failing to recognise or appropriately escalate red flag symptoms
Clinical impact: Delayed emergency treatment, death
Contraindication Ignored
Prescribing or recommending drugs unsafe for the patient
Clinical impact: Adverse drug reactions, teratogenicity, death
Multi-Factor Contraindication
Complex drug interaction chains that require considering multiple factors
Clinical impact: Organ failure, bleeding, serotonin syndrome, cardiac arrest
Dosage Frequency/Route Error
Correct drug but wrong frequency, route, or administration method
Clinical impact: Toxicity from overdosing frequency, treatment failure
False Reassurance
Inappropriately reassuring when escalation or urgent referral is needed
Clinical impact: Delayed cancer diagnosis, missed sepsis, death
Hallucinated Diagnosis
AI invents conditions not supported by evidence or patient presentation
Clinical impact: Unnecessary anxiety, investigations, or treatment
Scope Violation
Providing definitive diagnosis without sufficient clinical data
Clinical impact: Misdiagnosis, delayed appropriate care
Guideline Contradiction
Advice that conflicts with current NICE or BNF guidelines
Clinical impact: Suboptimal treatment, delayed effective care
Suppressed Uncertainty
Gives a single confident answer where senior clinicians would legitimately disagree or hedge — collapsing genuine diagnostic or management uncertainty into false certainty
Clinical impact: False certainty drives premature decisions, undermines informed consent, and hides the need for specialist review
Demographic Bias
Quality, caution, or recommendations differ by age, sex, ethnicity, or other protected characteristics in ways that are not clinically justified
Clinical impact: Inequitable care, missed diagnoses, and entrenched health disparities
Outdated Information
Using superseded clinical guidance or withdrawn medications
Clinical impact: Inappropriate treatment, missed safety signals
Anchored to real harm.
We calibrate every evaluator against this taxonomy. Each of the eight evaluation task types is chosen to catch specific failure modes above — so an evaluator’s reliability score isn’t an abstract number, it’s tied to the failures that actually harm patients.
- Rating
Hallucinated diagnosis · Outdated information · Guideline contradiction
- Comparison
Anchoring bias · False reassurance · Differential narrowing
- Ranking
Severity mis-ordering · Triage under-escalation
- Rubric
Dosage / frequency / route errors · Renal & hepatic adjustment failures
- Correction
Dangerous dosing · Contraindication ignored · Multi-factor contraindication
- Annotation
Scope violation · Emergency underestimation · Demographic bias
- Justification
Thin reasoning · Formulaic explanations · Fabricated citations
- Red-team
Adversarial coverage across every category in the taxonomy
Every task type is scored against gold-standard items from this taxonomy. Coverage is reported per category in the Reliability Report.
Bring it to your AI.
Building or deploying medical AI? Our safety framework gives your team the structured evaluation methodology to prove it’s safe at the bedside.
Structured red-teaming
Adversarial testing across every failure category by domain-expert clinicians.
Statistical Quality
Clinical evaluation
Statistically calibrated evaluators with confidence intervals on every metric.
Findings by Severity
Critical
High
Moderate
Low
Safety reports
Detailed reports with failure-mode coverage, severity distribution, and mitigation recommendations.
entertheloopClinicians powering AI alignment, training & safety.Verified against
entertheloopClinicians powering AI alignment, training & safety.