Hold a report? It lives in your dashboard.Sign in
The Reliability Report.
A Reliability Report records how a clinician performed across the eight task types, with uncertainty on every figure rather than a bare number. The calibration that issued them closed in September 2026. Reports already issued still stand, and buyers can still verify them.
At a glance
- Status
- No longer issued
- Reports issued
- Still valid
- Format
- PDF and JSON
- Calibration now
- Daily loop
The report is the record.
A Reliability Report is a procurement-grade record of how one clinician performed: a reliability score with confidence intervals on every metric, in a form a buyer can check rather than take on trust. The reports issued under the full calibration are unchanged by its closure.
All eight task types.
Evaluation work spans more than rating answers. The calibration exercised the full range, so a report records performance across the work buyers actually need. The daily loop mixes the same eight kinds of task.
Rating
Score a single AI response against a quality scale.
Comparison
Choose the better of two responses, and say why.
Ranking
Order several responses from best to worst.
Rubric
Assess against structured, dimensional criteria.
Correction
Fix an unsafe or incorrect response.
Annotation
Label spans and structure in clinical text.
Justification
Write the clinical reasoning behind a judgement.
Red-team
Probe for failure modes across the safety taxonomy.
Measured, not asserted.
Every metric in a report carries its uncertainty rather than standing as a bare number, which is why a report keeps meaning what it says long after it was issued.
Per-metric confidence intervals
Beta-Binomial intervals for proportions, bootstrap intervals for continuous metrics, so uncertainty is quantified on every score.
Lower confidence bounds
The report read the lower bound rather than the point estimate, so a score reflects worst-case plausible performance.
Per-category coverage
Performance broken down across task types and the safety taxonomy, recorded in the report.
A report already issued still stands.
A Reliability Report issued under the full calibration is yours, as a PDF for people and JSON for systems. It stays on your profile, it is not withdrawn and not reissued, and a buyer you share it with can still verify it.
A short daily loop.
Calibration runs as a set of five cases, most days, on cases written for your own profession. A set takes about four minutes. There is no screening step in front of it and nothing to unlock.
The two-phase calibration, Quick Screen then Full Calibration, was retired in September 2026. Calibration now runs as a short daily loop. A signed Reliability Report issued under the old pathway still stands, and buyers can still verify it.
Get calibrated.
Registration is free. Calibration runs as a short daily loop: five cases, most days, on cases written for your profession.
entertheloopClinicians powering AI alignment, training & safety.Verified against
entertheloopClinicians powering AI alignment, training & safety.