Near-duplicate detection
Identifying substantially similar content despite small wording or formatting changes.
What Near-duplicate detection means
Similarity thresholds determine whether syndication, updates, and legitimate variants are incorrectly merged.
Terms this definition uses
Related terms in Retrieval engineering
How pages become chunks, vectors and candidates inside an answer engine, and why being retrieved is not being cited.
Corpus · Document ingestion · Data connector · Parsing · OCR · Layout-aware parsing · Semantic chunking · Fixed-size chunking · Chunk overlap · Parent-child retrieval · Metadata filtering · Vector database · Vector index · Sparse vector · Dense vector · Cosine similarity · Dot-product similarity · Approximate nearest neighbor · HNSW · Top-k · Retrieval recall · Retrieval precision · Mean reciprocal rank · Normalized discounted cumulative gain · Relevance score · Candidate generation · Query rewriting · Query expansion · HyDE · Multi-query retrieval · Reciprocal rank fusion · Late interaction · Lexical retrieval · Semantic search · Exact-match retrieval · Freshness boosting · Authority weighting · Source diversity · Deduplication · Semantic cache · Retrieval latency · Retrieval cutoff · Citation mapping · Answer synthesis
Questions about Near-duplicate detection
What is Near-duplicate detection?
Identifying substantially similar content despite small wording or formatting changes.
Which area of the glossary does Near-duplicate detection belong to?
Retrieval engineering: How pages become chunks, vectors and candidates inside an answer engine, and why being retrieved is not being cited.
← Deduplication · Semantic cache →
See it in the full glossary · 731 terms across 19 areas.