Calibration
Agreement between predicted confidence and observed correctness over comparable cases.
What Calibration does not show
A calibrated 80 percent confidence should be correct about 80 percent of the time, not every time.
Where Calibration comes up on this site
- Our crawl-waste check named a parameter called amp;plan. No site has one.What the calibration found, once the names were right The section shipped on 9 September.
Definitions that name Calibration
Related terms in Measurement and evaluation
How a measurement earns trust: evaluation sets, error rates, calibration, reproducibility, and the biases that make a number look better than it is.
AI answer reading · name confusion · hedged answer · Benchmark · Evaluation set · Golden dataset · Ground truth · Human evaluation · LLM-as-judge · Pairwise evaluation · Pointwise evaluation · Rubric · Inter-rater agreement · Accuracy · F1 score · False positive rate · False negative · True positive · True negative · Confidence score · Coverage · Abstention · Reproducibility · Repeatability · Variance · Confidence interval · Sample size · Sampling bias · Selection bias · Survivorship bias · Data leakage · Test contamination
Questions about Calibration
What is Calibration?
Agreement between predicted confidence and observed correctness over comparable cases.
What does Calibration not show or guarantee?
A calibrated 80 percent confidence should be correct about 80 percent of the time, not every time.
Which area of the glossary does Calibration belong to?
Measurement and evaluation: How a measurement earns trust: evaluation sets, error rates, calibration, reproducibility, and the biases that make a number look better than it is.
← True negative · Confidence score →
See it in the full glossary · 731 terms across 19 areas.