RC Rick Collettesecurity · AI · music · people
Concept

Confidence Calibration

concept-confidence-calibration · canon/concepts/confidence-calibration.md

Confidence Calibration. The ongoing comparison between claimed confidence and observed correctness, used to decide whether future confidence claims remain trustworthy.

Calibration keeps confidence from becoming decoration. A system that labels conclusions high, medium, or low confidence should later compare those claims with observed outcomes and review evidence. When high-confidence answers stop being reliably right, the system should retrain, investigate, lower trust, or tighten its confidence gate.

Calibration evidence can also feed an eval gate: a model that no longer calibrates honestly should not be promoted unchanged.

Incoming References

Case Study 1
Law 1
Reference Architecture 1