RS Glossary
Calibration
Calibration describes how well stated confidence matches observed correctness across comparable cases. A well-calibrated system that assigns 80 percent confidence should be correct about 80 percent of the time on cases receiving that confidence.
- Review state
- Reviewed in POC
- Freshness
- Current
- Version
- 2
- calibration
Remember it this wayConfidence should match the record.
In plain language
Calibration asks whether confidence has earned its meaning. It compares what a person or system says about uncertainty with what happens over many relevant examples.
Why it matters
People often treat confident language as evidence of reliability. Calibration provides a record-based check. It can reveal overconfidence, underconfidence, and conditions where a confidence signal should not guide a decision.
Example
A classifier labels 100 comparable cases with 90 percent confidence, but only 65 labels are correct. The classifier may still be useful, but its confidence is poorly calibrated for those cases.
Common misconception
Calibration is not the same as accuracy, and it is not the same as the simpler question of whether one answer is correct. Confidence vs. Correctness separates two properties of a single answer. Calibration measures whether confidence levels match correctness rates across a defined set of comparable cases. A system can therefore be accurate overall and still be poorly calibrated.
Related terms
Sources
- Chuan Guo and colleagues, “On Calibration of Modern Neural Networks”calibration as agreement between predicted confidence and empirical correctness, and its distinction from accuracy.
Learn more
Public history
27 recorded events · 27 public summaries
Candidate vocabulary was imported from the Learning Lab discovery queue.
The captured candidate received a complete initial draft and entered section-level human review.
Radar reviewed the initial definition draft for human consideration.
Radar reviewed the initial plain-language-explanation draft for human consideration.
Radar reviewed the initial why-it-matters draft for human consideration.
Radar reviewed the initial example draft for human consideration.
Radar reviewed the initial common-misconception draft for human consideration.
Radar reviewed the initial remember-it-this-way draft for human consideration.
Radar reviewed the initial related-terms draft for human consideration.
Radar reviewed the initial sources draft for human consideration.
Radar reviewed the initial learn-more draft for human consideration.
A proposal was opened for section-level human review while preserving the current version.
A human reviewer opened a proposed revision while preserving the current approved or captured version.
plain-language-explanation approved by a human reviewer.
why-it-matters approved by a human reviewer.
example approved by a human reviewer.
common-misconception returned for revision.
remember-it-this-way approved by a human reviewer.
related-terms approved by a human reviewer.
sources approved by a human reviewer.
learn-more approved by a human reviewer.
Requested wording was prepared without changing approved or existing sections.
Radar reviewed the pending common-misconception section for human consideration.
definition approved by a human reviewer.
common-misconception approved by a human reviewer.
common-misconception approved wording became the current section text.
All section gates are approved; content version 2 is now a fully approved term.
Source: vault/Glossary/terms/calibration/term.md · sha256:ca0deb6473c2e12dedb3f185f3a36c898a6e5a5d033ed2c0d0a2d3ff7d872b00 · Release: rs-glossary-glossary-team-poc-0.5.5-2026-09-02