Corpus
The collection of documents available to a search or retrieval system.
What Corpus does not show
It defines what can be found; no ranking method can retrieve a document absent from the corpus.
Where Corpus comes up on this site
- How to check whether AI crawlers and agents can read your siteIn this corpus, of scans find robots.txt not answering 200 at all, find a challenge page served at 200, and find HTML sitting where the file should be.
- How to rank in ChatGPT: the part that is actually in your controlThe heaviest page in this corpus delivered 476,540 bytes carrying 2.3% visible text — the remainder markup, inline styles and script.
- Does llms.txt actually do anything?The median site in the vendor corpus delivers under one visible word per twenty bytes of payload.
- Do AI crawlers and agents render JavaScript?The heaviest page in this corpus delivered 476,540 bytes with 2.3% visible text .
- SEO audit tools compared: what each one actually fetchesThe most common defect in the corpus is medium severity and the critical ones are rare.
Terms this definition uses
Definitions that name Corpus
score version · percentile rank · BM25 · CCBot
Related terms in Retrieval engineering
How pages become chunks, vectors and candidates inside an answer engine, and why being retrieved is not being cited.
Document ingestion · Data connector · Parsing · OCR · Layout-aware parsing · Semantic chunking · Fixed-size chunking · Chunk overlap · Parent-child retrieval · Metadata filtering · Vector database · Vector index · Sparse vector · Dense vector · Cosine similarity · Dot-product similarity · Approximate nearest neighbor · HNSW · Top-k · Retrieval recall · Retrieval precision · Mean reciprocal rank · Normalized discounted cumulative gain · Relevance score · Candidate generation · Query rewriting · Query expansion · HyDE · Multi-query retrieval · Reciprocal rank fusion · Late interaction · Lexical retrieval · Semantic search · Exact-match retrieval · Freshness boosting · Authority weighting · Source diversity · Deduplication · Near-duplicate detection · Semantic cache · Retrieval latency · Retrieval cutoff · Citation mapping · Answer synthesis
Questions about Corpus
What is Corpus?
The collection of documents available to a search or retrieval system.
What does Corpus not show or guarantee?
It defines what can be found; no ranking method can retrieve a document absent from the corpus.
Which area of the glossary does Corpus belong to?
Retrieval engineering: How pages become chunks, vectors and candidates inside an answer engine, and why being retrieved is not being cited.
See it in the full glossary · 731 terms across 19 areas.