Glossary · Measurement and evaluation
Human evaluation
People judging outputs against stated criteria. It captures qualities automated metrics miss while introducing rater variation, cost, and fatigue.
Terms this definition uses
Measurement and evaluation
How a measurement earns trust: evaluation sets, error rates, calibration, reproducibility, and the biases that make a number look better than it is.
Benchmark · Evaluation set · Golden dataset · Ground truth · LLM-as-judge · Pairwise evaluation · Pointwise evaluation · Rubric · Inter-rater agreement · Accuracy · F1 score · False positive rate · False negative · True positive · True negative · Calibration · Confidence score · Coverage · Abstention · Reproducibility · Repeatability · Variance · Confidence interval · Sample size · Sampling bias · Selection bias · Survivorship bias · Data leakage · Test contamination
← Ground truth · LLM-as-judge →
See it in the full glossary · 579 terms across 19 areas. Scan a site to see which of these apply to it.