Glossary · Measurement and evaluation

Selection bias

Distortion caused by which sites, prompts, pages, or users enter analysis. Opt-in datasets especially should not be described as the whole web.

Measurement and evaluation

How a measurement earns trust: evaluation sets, error rates, calibration, reproducibility, and the biases that make a number look better than it is.

Benchmark · Evaluation set · Golden dataset · Ground truth · Human evaluation · LLM-as-judge · Pairwise evaluation · Pointwise evaluation · Rubric · Inter-rater agreement · Accuracy · F1 score · False positive rate · False negative · True positive · True negative · Calibration · Confidence score · Coverage · Abstention · Reproducibility · Repeatability · Variance · Confidence interval · Sample size · Sampling bias · Survivorship bias · Data leakage · Test contamination

← Sampling bias  ·  Survivorship bias →

See it in the full glossary · 579 terms across 19 areas. Scan a site to see which of these apply to it.