Reference · glossary

# The words of the agentic web, *defined with their limits.*

Every definition says what the term can support **and what it cannot**, because most of the confusion in this field is a word doing more work than the evidence behind it.

In one sentence

The CrawlCheck glossary defines 792 AI-visibility, GEO, AEO and AI-crawler terms across 19 areas — from AI visibility score, llms.txt and entity graph to verified crawler, RAG and grounding — and every definition states what the term can support and what it cannot, because most confusion in this field is a word doing more work than the evidence behind it.

Read it as an agent

[Every term as JSON-LDGET /glossary.json](/glossary.json)[Any page as MarkdownAccept: text/markdown](/glossary/llms-txt)[The site map for modelsGET /llms.txt](/llms.txt)

Written against what CrawlCheck measures. Where a term is commonly used to mean more than it can carry — a citation, a forged crawler, a visibility score, a passing file — the definition says so rather than repeating the looser sense. Terms that name a finding code show the live share of scans carrying it.

Browse by area A to Z

**792**terms

**19**areas

**64**terms with a live finding rate

**67**guides the terms link into

Four stages

## Where a term belongs

A term in a later stage is worth nothing if an earlier one never happened: a page that was never fetched cannot be misread, and a fact that was never parsed cannot be quoted.

- 1

Reach

can a client get the bytes

[Crawlers and access](/glossary/area/crawlers-and-access) · [Delivery and rendering](/glossary/area/delivery-and-rendering) · [HTTP, edge and rendering](/glossary/area/http-edge-and-rendering) · [Crawling and indexing](/glossary/area/crawling-and-indexing)

- 2

Read

can it parse what it got

[Machine files](/glossary/area/machine-files) · [Entities and schema](/glossary/area/entities-and-schema) · [Linked data and entities](/glossary/area/linked-data-and-entities) · [Indexing and discovery](/glossary/area/indexing-and-discovery) · [Content architecture and SEO](/glossary/area/content-architecture-and-seo)

- 3

Quote

can an engine safely repeat it

[Answer engines and retrieval](/glossary/area/answer-engines-and-retrieval) · [Retrieval engineering](/glossary/area/retrieval-engineering) · [Models and generation](/glossary/area/models-and-generation) · [Measurement and evidence](/glossary/area/measurement-and-evidence) · [Measurement and evaluation](/glossary/area/measurement-and-evaluation) · [Proof and provenance](/glossary/area/proof-and-provenance)

- 4

Operate

can an agent act on it, safely

[Agents and protocols](/glossary/area/agents-and-protocols) · [Security and privacy](/glossary/area/security-and-privacy) · [Governance and rights](/glossary/area/governance-and-rights) · [Accessibility and provenance](/glossary/area/accessibility-and-provenance)

Added the week of November 16, 2026

## 20 new terms

The glossary grows every week, each term written against something CrawlCheck serves or measures. [Every week’s additions as JSON](/api/glossary/cadence).

[Cookie prefix](/glossary/cookie-prefix)[Partitioned cookie](/glossary/partitioned-cookie)[SameSite](/glossary/samesite)[Public suffix](/glossary/public-suffix)[Schemeful same-site](/glossary/schemeful-same-site)[Subresource Integrity](/glossary/subresource-integrity)[Integrity metadata](/glossary/integrity-metadata)[CSP report-only](/glossary/csp-report-only)[CSP hash](/glossary/csp-hash)[strict-dynamic](/glossary/strict-dynamic)[form-action](/glossary/form-action)[upgrade-insecure-requests](/glossary/upgrade-insecure-requests)[base-uri](/glossary/base-uri)[object-src](/glossary/object-src)[Referrer leakage](/glossary/referrer-leakage)[Dangling CNAME](/glossary/dangling-cname)[Subdomain takeover](/glossary/subdomain-takeover)[SPF lookup limit](/glossary/spf-lookup-limit)[TLS downgrade](/glossary/tls-downgrade)[Certificate Transparency](/glossary/certificate-transparency)

19 areas

## Browse by area

01 30 terms

### [Answer engines and retrieval](/glossary/area/answer-engines-and-retrieval)

How a generated answer gets made, and what your page is competing for inside it.

[AEO](/glossary/aeo)[GEO](/glossary/geo)[AI visibility score](/glossary/ai-visibility-score)[share of voice](/glossary/share-of-voice)[prompt set](/glossary/prompt-set)[brand mention](/glossary/brand-mention)[RAG](/glossary/rag)[+23 more](/glossary/area/answer-engines-and-retrieval)

02 46 terms

### [Crawlers and access](/glossary/area/crawlers-and-access)

Who is fetching, whether they are who they claim, and what your rules actually permit.

[visitor resolve](/glossary/visitor-resolve)[published crawler ranges](/glossary/published-crawler-ranges)[operator origin](/glossary/operator-origin)[signed agent](/glossary/signed-agent)[agent arrival](/glossary/agent-arrival)[agent conduct record](/glossary/agent-conduct-record)[agent traffic index](/glossary/agent-traffic-index)[+39 more](/glossary/area/crawlers-and-access)

03 24 terms

### [Machine files](/glossary/area/machine-files)

The files addressed to machines rather than readers, and what publishing one does and does not buy.

[llms.txt](/glossary/llms-txt)[llms-full.txt](/glossary/llms-full-txt)[agents.md](/glossary/agents-md)[entity map](/glossary/entity-map)[entity map parity](/glossary/entity-map-parity)[machine layer](/glossary/machine-layer)[well-known URI](/glossary/well-known-uri)[+17 more](/glossary/area/machine-files)

04 45 terms

### [Entities and schema](/glossary/area/entities-and-schema)

Declaring who you are in a form a resolver can use without guessing.

[invented value](/glossary/invented-value)[signed business record](/glossary/signed-business-record)[listing audit](/glossary/listing-audit)[listing supplier](/glossary/listing-supplier)[fix receipt](/glossary/fix-receipt)[structured data](/glossary/structured-data)[JSON-LD](/glossary/json-ld)[+38 more](/glossary/area/entities-and-schema)

05 36 terms

### [Indexing and discovery](/glossary/area/indexing-and-discovery)

Whether a page can be found and kept, separately from whether it can be fetched.

[nosnippet](/glossary/nosnippet)[max-snippet](/glossary/max-snippet)[max-image-preview](/glossary/max-image-preview)[unavailable_after](/glossary/unavailable-after)[noimageindex](/glossary/noimageindex)[index coverage](/glossary/index-coverage)[canonical](/glossary/canonical)[+29 more](/glossary/area/indexing-and-discovery)

06 17 terms

### [Delivery and rendering](/glossary/area/delivery-and-rendering)

What arrives on the wire, and what only exists after something else runs.

[broken embed](/glossary/broken-embed)[payload ratio](/glossary/payload-ratio)[render dependence](/glossary/render-dependence)[render dependence gap](/glossary/render-dependence-gap)[edge cache pinning](/glossary/edge-cache-pinning)[TTFB](/glossary/ttfb)[Core Web Vitals](/glossary/core-web-vitals)[+10 more](/glossary/area/delivery-and-rendering)

07 52 terms

### [Measurement and evidence](/glossary/area/measurement-and-evidence)

The vocabulary for saying how strongly something is known.

[technographic data](/glossary/technographic-data)[read equivalence](/glossary/read-equivalence)[content-change clock](/glossary/content-change-clock)[self-claim ledger](/glossary/self-claim-ledger)[agent task canary](/glossary/agent-task-canary)[revocation delta](/glossary/revocation-delta)[signed verdict](/glossary/signed-verdict)[+45 more](/glossary/area/measurement-and-evidence)

08 26 terms

### [Proof and provenance](/glossary/area/proof-and-provenance)

Showing a record existed at a date without asking anyone to trust us.

[BibTeX](/glossary/bibtex)[RIS citation format](/glossary/ris-format)[transparency log](/glossary/transparency-log)[signed tree head](/glossary/signed-tree-head)[inclusion proof](/glossary/inclusion-proof)[log witness](/glossary/log-witness)[split-view attack](/glossary/split-view-attack)[+19 more](/glossary/area/proof-and-provenance)

09 35 terms

### [Models and generation](/glossary/area/models-and-generation)

How a model produces text, what its training and decoding controls change, and what none of them can prove about a source.

[Autoregressive model](/glossary/autoregressive-model)[Transformer](/glossary/transformer)[Attention](/glossary/attention)[Self-attention](/glossary/self-attention)[Parameter](/glossary/parameter)[Inference](/glossary/inference)[Pretraining](/glossary/pretraining)[+28 more](/glossary/area/models-and-generation)

10 45 terms

### [Retrieval engineering](/glossary/area/retrieval-engineering)

How pages become chunks, vectors and candidates inside an answer engine, and why being retrieved is not being cited.

[Corpus](/glossary/corpus)[Document ingestion](/glossary/document-ingestion)[Data connector](/glossary/data-connector)[Parsing](/glossary/parsing)[OCR](/glossary/ocr)[Layout-aware parsing](/glossary/layout-aware-parsing)[Semantic chunking](/glossary/semantic-chunking)[+38 more](/glossary/area/retrieval-engineering)

11 46 terms

### [Crawling and indexing](/glossary/area/crawling-and-indexing)

How a crawler discovers, schedules, fetches and selects pages, and why a fetched page is not an indexed one.

[Sitemap directive](/glossary/sitemap-directive)[Crawler identity](/glossary/crawler-identity)[Product token](/glossary/product-token)[Identification string](/glossary/identification-string)[Crawl frontier](/glossary/crawl-frontier)[Crawl queue](/glossary/crawl-queue)[Crawl scheduler](/glossary/crawl-scheduler)[+39 more](/glossary/area/crawling-and-indexing)

12 72 terms

### [HTTP, edge and rendering](/glossary/area/http-edge-and-rendering)

The delivery layer between an origin and a client: proxies, caches, redirects, status codes and the headers that decide what a crawler actually received.

[Content-Language header](/glossary/content-language)[HTTP/3](/glossary/http-3)[Alt-Svc](/glossary/alt-svc)[HTTPS DNS record](/glossary/https-dns-record)[Link header](/glossary/link-header)[content negotiation](/glossary/content-negotiation)[Markdown negotiation](/glossary/markdown-negotiation)[+65 more](/glossary/area/http-edge-and-rendering)

13 53 terms

### [Linked data and entities](/glossary/area/linked-data-and-entities)

The RDF and JSON-LD machinery behind structured data, and the schema.org types a business page actually needs.

[SearchAction](/glossary/searchaction)[Dataset (schema.org)](/glossary/dataset-schema)[DataDownload](/glossary/datadownload)[DataCatalog](/glossary/datacatalog)[variableMeasured](/glossary/variable-measured)[TechArticle](/glossary/techarticle)[CollectionPage](/glossary/collectionpage)[+46 more](/glossary/area/linked-data-and-entities)

14 45 terms

### [Content architecture and SEO](/glossary/area/content-architecture-and-seo)

How pages, links and intent are organised so an entity is findable, and what a link or a structure can and cannot transfer.

[lang attribute](/glossary/lang-attribute)[x-default](/glossary/x-default)[Search intent](/glossary/search-intent)[Query class](/glossary/query-class)[Navigational query](/glossary/navigational-query)[Informational query](/glossary/informational-query)[Transactional query](/glossary/transactional-query)[+38 more](/glossary/area/content-architecture-and-seo)

15 33 terms

### [Measurement and evaluation](/glossary/area/measurement-and-evaluation)

How a measurement earns trust: evaluation sets, error rates, calibration, reproducibility, and the biases that make a number look better than it is.

[AI answer reading](/glossary/ai-answer-reading)[name confusion](/glossary/name-confusion)[hedged answer](/glossary/hedged-answer)[Benchmark](/glossary/benchmark)[Evaluation set](/glossary/evaluation-set)[Golden dataset](/glossary/golden-dataset)[Ground truth](/glossary/ground-truth)[+26 more](/glossary/area/measurement-and-evaluation)

16 65 terms

### [Agents and protocols](/glossary/area/agents-and-protocols)

How autonomous agents plan, remember, call tools and talk to each other, including MCP and A2A, and where each step can go wrong.

[OpenSearch description](/glossary/opensearch-description)[traceparent](/glossary/traceparent)[JSON Schema](/glossary/json-schema)[protected-resource metadata](/glossary/protected-resource-metadata)[token audience](/glossary/token-audience)[PKCE](/glossary/pkce)[refresh token](/glossary/refresh-token)[+58 more](/glossary/area/agents-and-protocols)

17 68 terms

### [Security and privacy](/glossary/area/security-and-privacy)

Where an agent, a tool or a page can be turned against its operator, and the controls that limit the damage without proving safety.

[Cookie prefix](/glossary/cookie-prefix)[Partitioned cookie](/glossary/partitioned-cookie)[SameSite](/glossary/samesite)[Public suffix](/glossary/public-suffix)[Schemeful same-site](/glossary/schemeful-same-site)[Subresource Integrity](/glossary/subresource-integrity)[Integrity metadata](/glossary/integrity-metadata)[+61 more](/glossary/area/security-and-privacy)

18 32 terms

### [Governance and rights](/glossary/area/governance-and-rights)

Who may use content for what: rights reservations, opt-outs, licences, model documentation and the accountability vocabulary around them.

[per-use rights](/glossary/per-use-rights)[rights feed](/glossary/rights-feed)[Text and data mining](/glossary/text-and-data-mining)[Rights reservation](/glossary/rights-reservation)[Machine-readable opt-out](/glossary/machine-readable-opt-out)[Usage preference](/glossary/usage-preference)[Content-Usage header](/glossary/content-usage-header)[+25 more](/glossary/area/governance-and-rights)

19 22 terms

### [Accessibility and provenance](/glossary/area/accessibility-and-provenance)

What a page owes a keyboard, a screen reader and a fact-checker: the accessibility basics and the C2PA vocabulary for proving where a file came from.

[Keyboard accessibility](/glossary/keyboard-accessibility)[Focus order](/glossary/focus-order)[Focus indicator](/glossary/focus-indicator)[ARIA](/glossary/aria)[Accessible description](/glossary/accessible-description)[Color contrast](/glossary/color-contrast)[Reduced motion](/glossary/reduced-motion)[+15 more](/glossary/area/accessibility-and-provenance)

How to read these

## Three questions to ask of any term

**What is the denominator?**A rate without one is decoration.

**Was absence measured or assumed?**Unverifiable is not forged, and not found is not the same as not there.

**What would it look like if it were wrong?**A term that cannot fail a test is not measuring anything.

New · measured weekly

## The AI classifier atlas

659 labels a reasoning model writes while it picks a local business, from `hidden_intent_possible` to `trust_risk`, defined in the order it uses them, with what a business can do about each and a weekly count of which ones it actually uses. [Open the atlas](/glossary/ai-classifiers).

Every term

## A to Z

#
[202 interstitial](/glossary/202-interstitial)[301 redirect](/glossary/301-redirect)[302 redirect](/glossary/302-redirect)[304 response](/glossary/304-response)[307 redirect](/glossary/307-redirect)[308 redirect](/glossary/308-redirect)[404 and 410 status codes](/glossary/404-and-410-status-codes)[404 response](/glossary/404-response)[410 response](/glossary/410-response)[429 response](/glossary/429-response)[500 response](/glossary/500-response)[502 response](/glossary/502-response)[503 response](/glossary/503-response)[504 response](/glossary/504-response)[@context](/glossary/schema-context)

A
[A2A](/glossary/a2a)[A2A client](/glossary/a2a-client)[A2A protocol](/glossary/a2a-protocol)[A2A server](/glossary/a2a-server)[Abstention](/glossary/abstention)[accessibility structure](/glossary/accessibility-structure)[Accessible description](/glossary/accessible-description)[accessible name](/glossary/accessible-name)[Accountability](/glossary/accountability)[Accuracy](/glossary/accuracy)[action assurance](/glossary/action-assurance)[AEO](/glossary/aeo)[Age header](/glossary/age-header)[agent arrival](/glossary/agent-arrival)[Agent artifact](/glossary/agent-artifact)[Agent authorization](/glossary/agent-authorization)[agent card](/glossary/agent-card)[agent conduct record](/glossary/agent-conduct-record)[Agent discovery](/glossary/agent-discovery)[Agent handoff](/glossary/agent-handoff)[Agent identity](/glossary/agent-identity)[Agent loop](/glossary/agent-loop)[agent manifest](/glossary/agent-manifest)[Agent memory](/glossary/agent-memory)[Agent orchestration](/glossary/agent-orchestration)[Agent sandbox](/glossary/agent-sandbox)[Agent skill](/glossary/agent-skill)[Agent task](/glossary/agent-task)[agent task canary](/glossary/agent-task-canary)[agent traffic index](/glossary/agent-traffic-index)[agentic knowledge graph](/glossary/agentic-knowledge-graph)[agentic traffic](/glossary/agentic-traffic)[agentic web](/glossary/agentic-web)[agents.md](/glossary/agents-md)[AggregateRating](/glossary/aggregaterating)[AI agent](/glossary/ai-agent)[AI answer reading](/glossary/ai-answer-reading)[AI Mode](/glossary/ai-mode)[AI opt-out](/glossary/ai-opt-out)[AI Overview](/glossary/ai-overview)[AI referral attribution](/glossary/ai-referral-attribution)[AI training license](/glossary/ai-training-license)[AI visibility score](/glossary/ai-visibility-score)[ai.txt](/glossary/ai-txt)[Algorithmic transparency](/glossary/algorithmic-transparency)[alias domain](/glossary/alias-domain)[Allow header](/glossary/allow-header)[Allowlist](/glossary/allowlist)[alt attribute](/glossary/alt-attribute)[alt text](/glossary/alt-text)[Alt-Svc](/glossary/alt-svc)[Anchor text](/glossary/anchor-text)[anchoring](/glossary/anchoring)[answer block](/glossary/answer-block)[answer engine](/glossary/answer-engine)[answer stability](/glossary/answer-stability)[Answer synthesis](/glossary/answer-synthesis)[apex domain](/glossary/apex-domain)[API key](/glossary/api-key)[Approval gate](/glossary/approval-gate)[Approximate nearest neighbor](/glossary/approximate-nearest-neighbor)[ARIA](/glossary/aria)[Article](/glossary/article)[as-of answer](/glossary/as-of-answer)[ASN blocking](/glossary/asn-blocking)[Attention](/glossary/attention)[Audio description](/glossary/audio-description)[Audit log](/glossary/audit-log)[Author relationship](/glossary/author-relationship)[Authority weighting](/glossary/authority-weighting)[Autoregressive model](/glossary/autoregressive-model)

B
[base-uri](/glossary/base-uri)[Bearer token](/glossary/bearer-token)[Benchmark](/glossary/benchmark)[BibTeX](/glossary/bibtex)[Blank node](/glossary/blank-node)[blocked render resource](/glossary/blocked-render-resource)[BlogPosting](/glossary/blogposting)[BM25](/glossary/bm25)[boilerplate](/glossary/boilerplate)[bot management platform](/glossary/bot-management)[Bot score](/glossary/bot-score)[Bot spoofing](/glossary/bot-spoofing)[brand mention](/glossary/brand-mention)[BreadcrumbList](/glossary/breadcrumblist)[broken embed](/glossary/broken-embed)[Broken link](/glossary/broken-link)[Brotli](/glossary/brotli)[byline](/glossary/byline)[Bytespider](/glossary/bytespider)

C
[C2PA](/glossary/c2pa)[C2PA assertion](/glossary/c2pa-assertion)[C2PA claim](/glossary/c2pa-claim)[CAA record](/glossary/caa-record)[Cache hit](/glossary/cache-hit)[Cache key](/glossary/cache-key)[Cache miss](/glossary/cache-miss)[cache split](/glossary/cache-split)[cache-buster](/glossary/cache-buster)[Cache-Control](/glossary/cache-control)[Calibration](/glossary/calibration)[canary](/glossary/canary)[Candidate generation](/glossary/candidate-generation)[canonical](/glossary/canonical)[Canonical cluster](/glossary/canonical-cluster)[canonical origin](/glossary/canonical-origin)[Canonical RDF](/glossary/canonical-rdf)[canonical URL](/glossary/canonical-url)[Capability negotiation](/glossary/capability-negotiation)[Captions](/glossary/captions)[case consistency](/glossary/case-consistency)[CAWG training and data mining assertion](/glossary/cawg-training-assertion)[CCBot](/glossary/ccbot)[CDN](/glossary/cdn)[Certificate Transparency](/glossary/certificate-transparency)[Chain of thought](/glossary/chain-of-thought)[challenge page at 200](/glossary/challenge-page-at-200)[change receipt](/glossary/change-receipt)[changefreq and priority](/glossary/changefreq-and-priority)[Chunk boundary](/glossary/chunk-boundary)[Chunk overlap](/glossary/chunk-overlap)[chunking](/glossary/chunking)[citation](/glossary/citation)[citation anchor](/glossary/citation-anchor)[Citation mapping](/glossary/citation-mapping)[citation rate](/glossary/citation-rate)[cite-as link](/glossary/cite-as-link)[Claim signature](/glossary/claim-signature)[ClaudeBot vs Claude-User](/glossary/claudebot-claude-user)[cloaking](/glossary/cloaking)[CLS](/glossary/cls)[cohort asymmetry](/glossary/cohort-asymmetry)[CollectionPage](/glossary/collectionpage)[Color contrast](/glossary/color-contrast)[Commercial investigation](/glossary/commercial-investigation)[Compression](/glossary/compression)[Conditional request](/glossary/conditional-request)[Confidence interval](/glossary/confidence-interval)[Confidence score](/glossary/confidence-score)[Connection timeout](/glossary/connection-timeout)[Consent record](/glossary/consent-record)[Constrained decoding](/glossary/constrained-decoding)[ContactPoint](/glossary/contactpoint)[Content binding](/glossary/content-binding)[Content Credential](/glossary/content-credential)[Content decay](/glossary/content-decay)[Content encoding](/glossary/content-encoding)[Content hash](/glossary/content-hash)[Content hub](/glossary/content-hub)[content negotiation](/glossary/content-negotiation)[Content refresh](/glossary/content-refresh)[Content Signals](/glossary/content-signals)[Content type](/glossary/content-type)[content-addressed storage](/glossary/content-addressed-storage)[content-change clock](/glossary/content-change-clock)[Content-Digest](/glossary/content-digest)[Content-Language header](/glossary/content-language)[Content-Security-Policy](/glossary/content-security-policy)[Content-Signal](/glossary/content-signal)[Content-Usage header](/glossary/content-usage-header)[Context caching](/glossary/context-caching)[Context compaction](/glossary/context-compaction)[Context poisoning](/glossary/context-poisoning)[context window](/glossary/context-window)[contrast ratio](/glossary/contrast-ratio)[Cookie prefix](/glossary/cookie-prefix)[Copyright provenance](/glossary/copyright-provenance)[Core Web Vitals](/glossary/core-web-vitals)[Corpus](/glossary/corpus)[Cosine similarity](/glossary/cosine-similarity)[cosmetic row](/glossary/cosmetic-row)[Coverage](/glossary/coverage)[Crawl anomaly](/glossary/crawl-anomaly)[crawl budget](/glossary/crawl-budget)[Crawl demand](/glossary/crawl-demand)[Crawl frequency](/glossary/crawl-frequency)[Crawl frontier](/glossary/crawl-frontier)[Crawl queue](/glossary/crawl-queue)[Crawl rate](/glossary/crawl-rate)[Crawl scheduler](/glossary/crawl-scheduler)[Crawl-delay](/glossary/crawl-delay)[crawler classification](/glossary/crawler-classification)[Crawler identity](/glossary/crawler-identity)[Crawler trap](/glossary/crawler-trap)[Credential scope](/glossary/credential-scope)[cross-encoder](/glossary/cross-encoder)[CrUX](/glossary/crux)[CSP hash](/glossary/csp-hash)[CSP nonce](/glossary/csp-nonce)[CSP report-only](/glossary/csp-report-only)

D
[daily seal](/glossary/daily-seal)[dangling @id](/glossary/dangling-id)[Dangling CNAME](/glossary/dangling-cname)[data aggregator](/glossary/data-aggregator)[Data Axle](/glossary/data-axle)[Data connector](/glossary/data-connector)[Data controller](/glossary/data-controller)[Data exfiltration](/glossary/data-exfiltration)[Data leakage](/glossary/data-leakage)[Data lineage](/glossary/data-lineage)[Data minimization](/glossary/data-minimization)[Data processor](/glossary/data-processor)[Data residency](/glossary/data-residency)[Data sheet](/glossary/data-sheet)[data snapshot](/glossary/data-snapshot)[DataCatalog](/glossary/datacatalog)[datacenter IP](/glossary/datacenter-ip)[DataDownload](/glossary/datadownload)[Dataset (schema.org)](/glossary/dataset-schema)[Dataset license](/glossary/dataset-license)[Datatype](/glossary/datatype)[dated series](/glossary/dated-series)[dateline](/glossary/dateline)[dead anchor](/glossary/dead-anchor)[decision receipt](/glossary/decision-receipt)[Deduplication](/glossary/deduplication)[Deep link](/glossary/deep-link)[Default graph](/glossary/default-graph)[default scope request](/glossary/default-scope-request)[DefinedTerm](/glossary/definedterm)[DefinedTermSet](/glossary/definedtermset)[Deindexing](/glossary/deindexing)[Delegation](/glossary/delegation)[delivery comparison](/glossary/delivery-comparison)[denominator note](/glossary/denominator-note)[dense retrieval](/glossary/dense-retrieval)[Dense vector](/glossary/dense-vector)[Denylist](/glossary/denylist)[Deterministic decoding](/glossary/deterministic-decoding)[Developer message](/glossary/developer-message)[Direct prompt injection](/glossary/direct-prompt-injection)[DKIM](/glossary/dkim)[DMARC](/glossary/dmarc)[DMARC alignment](/glossary/dmarc-alignment)[DNS resolution](/glossary/dns-resolution)[DNS timeout](/glossary/dns-timeout)[DNSSEC](/glossary/dnssec)[Document ingestion](/glossary/document-ingestion)[Dofollow](/glossary/dofollow)[DOM depth](/glossary/dom-depth)[domainIncludes](/glossary/domainincludes)[Dot-product similarity](/glossary/dot-product-similarity)[double-counted defect](/glossary/double-counted-defect)[Duplicate cluster](/glossary/duplicate-cluster)[duplicate content](/glossary/duplicate-content)[duplicate id](/glossary/duplicate-id)[Dynamic rendering](/glossary/dynamic-rendering)

E
[E-E-A-T](/glossary/e-e-a-t)[E-E-A-T proxies](/glossary/e-e-a-t-proxies)[edge cache](/glossary/edge-cache)[edge cache pinning](/glossary/edge-cache-pinning)[Edge node](/glossary/edge-node)[Edge redirect](/glossary/edge-redirect)[Edge transformation](/glossary/edge-transformation)[embedding](/glossary/embedding)[Encrypted Client Hello](/glossary/encrypted-client-hello)[End-anchor rule](/glossary/end-anchor-rule)[entity collision](/glossary/entity-collision)[entity disambiguation](/glossary/entity-disambiguation)[entity graph](/glossary/entity-graph)[entity map](/glossary/entity-map)[entity map parity](/glossary/entity-map-parity)[entity resolution](/glossary/entity-resolution)[Entity-first content](/glossary/entity-first-content)[Episodic memory](/glossary/episodic-memory)[ETag](/glossary/etag)[Evaluation set](/glossary/evaluation-set)[evidence level](/glossary/evidence-level)[Exact-match retrieval](/glossary/exact-match-retrieval)[Excessive agency](/glossary/excessive-agency)[Executor](/glossary/executor)[EXIF](/glossary/exif)[Explainability](/glossary/explainability)[External link](/glossary/external-link)

F
[F1 score](/glossary/f1-score)[faceted navigation](/glossary/faceted-navigation)[Fair use](/glossary/fair-use)[False negative](/glossary/false-negative)[false positive](/glossary/false-positive)[False positive rate](/glossary/false-positive-rate)[FAQPage](/glossary/faqpage)[FAQPage schema](/glossary/faqpage-schema)[FCrDNS](/glossary/fcrdns)[fetcher disagreement](/glossary/fetcher-disagreement)[field data](/glossary/field-data)[finding age](/glossary/finding-age)[finding fingerprint](/glossary/finding-fingerprint)[finding state](/glossary/finding-state)[Fine-tuning](/glossary/fine-tuning)[fix list](/glossary/fix-list)[fix receipt](/glossary/fix-receipt)[Fixed-size chunking](/glossary/fixed-size-chunking)[Focus indicator](/glossary/focus-indicator)[Focus order](/glossary/focus-order)[forged crawler identity](/glossary/forged-crawler-identity)[form control name](/glossary/form-control-name)[form-action](/glossary/form-action)[Foundation model](/glossary/foundation-model)[founder edge](/glossary/founder-edge)[foundingDate](/glossary/foundingdate)[frame-ancestors](/glossary/frame-ancestors)[Freshness boosting](/glossary/freshness-boosting)[freshness window](/glossary/freshness-window)[Function calling](/glossary/function-calling)

G
[gated row](/glossary/gated-row)[General-purpose AI model](/glossary/general-purpose-ai-model)[GEO](/glossary/geo)[geocoding](/glossary/geocoding)[GeoCoordinates](/glossary/geocoordinates)[GET request](/glossary/get-request)[Golden dataset](/glossary/golden-dataset)[Google Business Profile](/glossary/google-business-profile)[Google-Agent](/glossary/google-agent)[Google-Extended](/glossary/google-extended)[GPTBot vs OAI-SearchBot](/glossary/gptbot-oai-searchbot)[graded through a wall](/glossary/graded-through-a-wall)[Graph merge](/glossary/graph-merge)[Graph normalization](/glossary/graph-normalization)[Ground truth](/glossary/ground-truth)[grounding](/glossary/grounding)

H
[hallucinated attribution](/glossary/hallucinated-attribution)[hallucination](/glossary/hallucination)[Hard binding](/glossary/hard-binding)[hard bounce](/glossary/hard-bounce)[hash-prefix lookup](/glossary/hash-prefix-lookup)[HEAD request](/glossary/head-request)[Header parity](/glossary/header-parity)[Heading hierarchy](/glossary/heading-hierarchy)[headroom](/glossary/headroom)[hedged answer](/glossary/hedged-answer)[high-risk field](/glossary/high-risk-field)[Historical optimization](/glossary/historical-optimization)[HNSW](/glossary/hnsw)[homepage refused](/glossary/homepage-refused)[homoglyph](/glossary/homoglyph)[honeypot](/glossary/honeypot)[host duplicate](/glossary/host-duplicate)[Host politeness](/glossary/host-politeness)[hreflang](/glossary/hreflang)[HSTS](/glossary/hsts)[HSTS preload list](/glossary/hsts-preload-list)[HTTP Message Signatures](/glossary/http-message-signatures)[HTTP/3](/glossary/http-3)[HTTPS DNS record](/glossary/https-dns-record)[Human evaluation](/glossary/human-evaluation)[Human-in-the-loop](/glossary/human-in-the-loop)[Human-on-the-loop](/glossary/human-on-the-loop)[Hybrid search](/glossary/hybrid-search)[HyDE](/glossary/hyde)[Hydration mismatch](/glossary/hydration-mismatch)

I
[Identification string](/glossary/identification-string)[If-None-Match](/glossary/if-none-match)[image placeholder](/glossary/image-placeholder)[Image sitemap](/glossary/image-sitemap)[Impact assessment](/glossary/impact-assessment)[Inbound link](/glossary/inbound-link)[incident switch](/glossary/incident-switch)[inclusion proof](/glossary/inclusion-proof)[index coverage](/glossary/index-coverage)[Index freshness](/glossary/index-freshness)[Index lag](/glossary/index-lag)[Index selection](/glossary/index-selection)[Indexing pipeline](/glossary/indexing-pipeline)[IndexNow](/glossary/indexnow)[Indirect prompt injection](/glossary/indirect-prompt-injection)[Inference](/glossary/inference)[Information architecture](/glossary/information-architecture)[information gain](/glossary/information-gain)[Informational query](/glossary/informational-query)[Ingredient](/glossary/ingredient)[INP](/glossary/inp)[Input validation](/glossary/input-validation)[Insecure output handling](/glossary/insecure-output-handling)[Instruction tuning](/glossary/instruction-tuning)[Integrity metadata](/glossary/integrity-metadata)[intent coverage](/glossary/intent-coverage)[Inter-rater agreement](/glossary/inter-rater-agreement)[Internal link](/glossary/internal-link)[invented value](/glossary/invented-value)[Inverse property](/glossary/inverse-property)[IRI](/glossary/iri)[ItemList](/glossary/itemlist)

J
[Jailbreak](/glossary/jailbreak)[JSON Schema](/glossary/json-schema)[JSON-LD](/glossary/json-ld)[JSON-LD compaction](/glossary/json-ld-compaction)[JSON-LD expansion](/glossary/json-ld-expansion)[JSON-LD flattening](/glossary/json-ld-flattening)[JSON-RPC](/glossary/json-rpc)

K
[k-anonymity](/glossary/k-anonymity)[Keyboard accessibility](/glossary/keyboard-accessibility)[Knowledge distillation](/glossary/knowledge-distillation)[knowledge graph](/glossary/knowledge-graph)[knowsAbout](/glossary/knowsabout)

L
[lab data](/glossary/lab-data)[landmark](/glossary/landmark)[landmark region](/glossary/landmark-region)[lang attribute](/glossary/lang-attribute)[Language tag](/glossary/language-tag)[Last-Modified](/glossary/last-modified)[Late interaction](/glossary/late-interaction)[Lawful basis](/glossary/lawful-basis)[Layout-aware parsing](/glossary/layout-aware-parsing)[lazy loading](/glossary/lazy-loading)[LCP](/glossary/lcp)[Least privilege](/glossary/least-privilege)[Lexical retrieval](/glossary/lexical-retrieval)[Lighthouse](/glossary/lighthouse)[Link context](/glossary/link-context)[Link equity](/glossary/link-equity)[Link extraction](/glossary/link-extraction)[Link header](/glossary/link-header)[Link neighborhood](/glossary/link-neighborhood)[listing audit](/glossary/listing-audit)[listing supplier](/glossary/listing-supplier)[Literal](/glossary/literal)[LLM-as-judge](/glossary/llm-as-judge)[llms-full.txt](/glossary/llms-full-txt)[llms.txt](/glossary/llms-txt)[local citation](/glossary/local-citation)[local pack](/glossary/local-pack)[LocalBusiness](/glossary/localbusiness)[log file analysis](/glossary/log-file-analysis)[log witness](/glossary/log-witness)[Long-term memory](/glossary/long-term-memory)[Longest-match rule](/glossary/longest-match-rule)[lookalike domain](/glossary/lookalike-domain)

M
[machine layer](/glossary/machine-layer)[Machine-readable opt-out](/glossary/machine-readable-opt-out)[main landmark](/glossary/main-landmark)[mainEntity](/glossary/mainentity)[Manifest store](/glossary/manifest-store)[Manifest tampering](/glossary/manifest-tampering)[Markdown negotiation](/glossary/markdown-negotiation)[max-image-preview](/glossary/max-image-preview)[max-snippet](/glossary/max-snippet)[MCP](/glossary/mcp)[MCP client](/glossary/mcp-client)[MCP host](/glossary/mcp-host)[MCP preflight](/glossary/mcp-preflight)[MCP prompt](/glossary/mcp-prompt)[MCP resource](/glossary/mcp-resource)[MCP rug pull](/glossary/mcp-rug-pull)[MCP server](/glossary/mcp-server)[MCP tool](/glossary/mcp-tool)[MCP transport](/glossary/mcp-transport)[Mean reciprocal rank](/glossary/mean-reciprocal-rank)[Merkle tree](/glossary/merkle-tree)[Metadata filtering](/glossary/metadata-filtering)[microformats (h-card)](/glossary/h-card)[MIME sniffing](/glossary/mime-sniffing)[MinHash](/glossary/minhash)[Mixture of experts](/glossary/mixture-of-experts)[Model card](/glossary/model-card)[Model drift](/glossary/model-drift)[Model routing](/glossary/model-routing)[Model version](/glossary/model-version)[MTA-STS](/glossary/mta-sts)[mTLS](/glossary/mtls)[Multi-agent system](/glossary/multi-agent-system)[Multi-query retrieval](/glossary/multi-query-retrieval)[Multimodal model](/glossary/multimodal-model)

N
[name confusion](/glossary/name-confusion)[name drift](/glossary/name-drift)[named entity recognition](/glossary/named-entity-recognition)[Named graph](/glossary/named-graph)[NAP](/glossary/nap)[NAP consistency](/glossary/nap-consistency)[NAP schema](/glossary/nap-schema)[Navigational query](/glossary/navigational-query)[near-duplicate](/glossary/near-duplicate)[Near-duplicate detection](/glossary/near-duplicate-detection)[News sitemap](/glossary/news-sitemap)[NewsArticle](/glossary/newsarticle)[Nofollow](/glossary/nofollow)[noimageindex](/glossary/noimageindex)[noindex](/glossary/noindex)[nonexistent-path control](/glossary/nonexistent-path-control)[Normalized discounted cumulative gain](/glossary/normalized-discounted-cumulative-gain)[nosnippet](/glossary/nosnippet)

O
[OAuth scope](/glossary/oauth-scope)[Object](/glossary/object)[object-src](/glossary/object-src)[OCR](/glossary/ocr)[off-host redirect](/glossary/off-host-redirect)[Offer](/glossary/offer)[Ontology](/glossary/ontology)[Open license](/glossary/open-license)[OpenAPI specification](/glossary/openapi)[OpeningHoursSpecification](/glossary/openinghoursspecification)[OpenSearch description](/glossary/opensearch-description)[OpenTimestamps](/glossary/opentimestamps)[operator feed](/glossary/operator-feed)[operator origin](/glossary/operator-origin)[Organization schema](/glossary/organization-schema)[Origin bypass](/glossary/origin-bypass)[Origin server](/glossary/origin-server)[orphan page](/glossary/orphan-page)[Orphaned entity](/glossary/orphaned-entity)[Output validation](/glossary/output-validation)[own-host loop](/glossary/own-host-loop)

P
[PageRank](/glossary/pagerank)[Pairwise evaluation](/glossary/pairwise-evaluation)[Parameter](/glossary/parameter)[Parent-child retrieval](/glossary/parent-child-retrieval)[parked domain](/glossary/parked-domain)[Parsing](/glossary/parsing)[Partitioned cookie](/glossary/partitioned-cookie)[Passage ranking](/glossary/passage-ranking)[payload ratio](/glossary/payload-ratio)[payload text share](/glossary/payload-text-share)[per-use rights](/glossary/per-use-rights)[percentile rank](/glossary/percentile-rank)[Permissions-Policy](/glossary/permissions-policy)[PerplexityBot vs Perplexity-User](/glossary/perplexitybot-perplexity-user)[Person schema](/glossary/person-schema)[PII](/glossary/pii)[Pillar page](/glossary/pillar-page)[PKCE](/glossary/pkce)[Place](/glossary/place)[placeholder content](/glossary/placeholder-content)[placeholder in record](/glossary/placeholder-in-record)[Planner](/glossary/planner)[Pointwise evaluation](/glossary/pointwise-evaluation)[population baseline](/glossary/population-baseline)[PostalAddress](/glossary/postaladdress)[Predicate](/glossary/predicate)[Preference optimization](/glossary/preference-optimization)[preflight](/glossary/preflight)[presence rate](/glossary/presence-rate)[Pretraining](/glossary/pretraining)[Product](/glossary/product)[Product token](/glossary/product-token)[prompt injection](/glossary/prompt-injection)[prompt set](/glossary/prompt-set)[protected-resource metadata](/glossary/protected-resource-metadata)[Public domain](/glossary/public-domain)[Public suffix](/glossary/public-suffix)[publication floor](/glossary/publication-floor)[published crawler ranges](/glossary/published-crawler-ranges)[Publisher relationship](/glossary/publisher-relationship)[Purpose limitation](/glossary/purpose-limitation)

Q
[Quantization](/glossary/quantization)[Query class](/glossary/query-class)[Query expansion](/glossary/query-expansion)[Query fan-out](/glossary/query-fan-out)[Query rewriting](/glossary/query-rewriting)

R
[RAG](/glossary/rag)[rangeIncludes](/glossary/rangeincludes)[rate limit](/glossary/rate-limit)[rate limiting](/glossary/rate-limiting)[Rate-limit policy](/glossary/rate-limit-policy)[RCE](/glossary/rce)[RDF](/glossary/rdf)[RDF dataset](/glossary/rdf-dataset)[RDFa and Microdata](/glossary/rdfa-microdata)[Re-ranking](/glossary/re-ranking)[read equivalence](/glossary/read-equivalence)[Read timeout](/glossary/read-timeout)[Reasoning trace](/glossary/reasoning-trace)[receipt header](/glossary/receipt-header)[Reciprocal rank fusion](/glossary/reciprocal-rank-fusion)[reciprocated sameAs](/glossary/reciprocated-sameas)[Recrawl interval](/glossary/recrawl-interval)[Red teaming](/glossary/red-teaming)[Redirect chain](/glossary/redirect-chain)[Redirect loop](/glossary/redirect-loop)[Redirected internal link](/glossary/redirected-internal-link)[Reduced motion](/glossary/reduced-motion)[Referrer leakage](/glossary/referrer-leakage)[Referrer-Policy](/glossary/referrer-policy)[Referring domain](/glossary/referring-domain)[refresh token](/glossary/refresh-token)[refusal](/glossary/refusal)[regression](/glossary/regression)[Relevance score](/glossary/relevance-score)[Remote context](/glossary/remote-context)[Removal request](/glossary/removal-request)[render dependence](/glossary/render-dependence)[render dependence gap](/glossary/render-dependence-gap)[render gap](/glossary/render-gap)[render-blocking resource](/glossary/render-blocking-resource)[Rendered HTML](/glossary/rendered-html)[Rendering queue](/glossary/rendering-queue)[Repeatability](/glossary/repeatability)[Reproducibility](/glossary/reproducibility)[residential proxy](/glossary/residential-proxy)[resolve answer](/glossary/resolve-answer)[Response parity](/glossary/response-parity)[Retention period](/glossary/retention-period)[retrieval](/glossary/retrieval)[retrieval crawler](/glossary/retrieval-crawler)[Retrieval cutoff](/glossary/retrieval-cutoff)[Retrieval latency](/glossary/retrieval-latency)[Retrieval precision](/glossary/retrieval-precision)[Retrieval recall](/glossary/retrieval-recall)[Retry-After](/glossary/retry-after)[Reverse proxy](/glossary/reverse-proxy)[Review](/glossary/review)[revocation delta](/glossary/revocation-delta)[rights feed](/glossary/rights-feed)[Rights reservation](/glossary/rights-reservation)[RIS citation format](/glossary/ris-format)[Risk assessment](/glossary/risk-assessment)[rival roots](/glossary/rival-roots)[RLHF](/glossary/rlhf)[Robots cache](/glossary/robots-cache)[Robots group merging](/glossary/robots-group-merging)[Robots parse error](/glossary/robots-parse-error)[Robots redirect](/glossary/robots-redirect)[Robots unavailable](/glossary/robots-unavailable)[robots.txt](/glossary/robots-txt)[row review](/glossary/row-review)[Rubric](/glossary/rubric)[RUM](/glossary/rum)

S
[sameAs](/glossary/sameas)[SameSite](/glossary/samesite)[Sample size](/glossary/sample-size)[Sampling bias](/glossary/sampling-bias)[Schema property](/glossary/schema-property)[Schema type](/glossary/schema-type)[Schemeful same-site](/glossary/schemeful-same-site)[score version](/glossary/score-version)[Search intent](/glossary/search-intent)[SearchAction](/glossary/searchaction)[Secret leakage](/glossary/secret-leakage)[security.txt](/glossary/security-txt)[Seed URL](/glossary/seed-url)[Selection bias](/glossary/selection-bias)[Self-attention](/glossary/self-attention)[self-audit](/glossary/self-audit)[self-audit invariant](/glossary/self-audit-invariant)[self-claim ledger](/glossary/self-claim-ledger)[Semantic cache](/glossary/semantic-cache)[Semantic chunking](/glossary/semantic-chunking)[Semantic HTML](/glossary/semantic-html)[Semantic memory](/glossary/semantic-memory)[Semantic search](/glossary/semantic-search)[Sensitive information disclosure](/glossary/sensitive-information-disclosure)[Server-Timing](/glossary/server-timing)[Service](/glossary/service)[service area](/glossary/service-area)[SHA-256](/glossary/sha-256)[share of voice](/glossary/share-of-voice)[Shared IP range](/glossary/shared-ip-range)[shingle](/glossary/shingle)[signed absence](/glossary/signed-absence)[signed agent](/glossary/signed-agent)[signed business record](/glossary/signed-business-record)[signed tree head](/glossary/signed-tree-head)[signed verdict](/glossary/signed-verdict)[Sitemap](/glossary/sitemap)[Sitemap directive](/glossary/sitemap-directive)[Sitemap hreflang](/glossary/sitemap-hreflang)[sitemap index](/glossary/sitemap-index)[Sitemap lastmod](/glossary/sitemap-lastmod)[sitemap origin error](/glossary/sitemap-origin-error)[Sitemap URL set](/glossary/sitemap-url-set)[SKILL.md](/glossary/skill-md)[skip link](/glossary/skip-link)[Small language model](/glossary/small-language-model)[Soft 404](/glossary/soft-404)[soft 404 on a machine file](/glossary/soft-404-machine-file)[Soft binding](/glossary/soft-binding)[Source diversity](/glossary/source-diversity)[Sparse vector](/glossary/sparse-vector)[speakable](/glossary/speakable)[SPF](/glossary/spf)[SPF lookup limit](/glossary/spf-lookup-limit)[split-view attack](/glossary/split-view-attack)[Sponsored link](/glossary/sponsored-link)[SSRF](/glossary/ssrf)[stable @id](/glossary/stable-id)[staging leak](/glossary/staging-leak)[Stale response](/glossary/stale-response)[Stale-while-revalidate](/glossary/stale-while-revalidate)[stapled verdict](/glossary/stapled-verdict)[Status message](/glossary/status-message)[strict-dynamic](/glossary/strict-dynamic)[structured data](/glossary/structured-data)[structured data validation](/glossary/structured-data-validation)[Structured output](/glossary/structured-output)[Subdomain takeover](/glossary/subdomain-takeover)[Subject](/glossary/subject)[Subprocessor](/glossary/subprocessor)[Subresource Integrity](/glossary/subresource-integrity)[Supervised fine-tuning](/glossary/supervised-fine-tuning)[Survivorship bias](/glossary/survivorship-bias)[System prompt](/glossary/system-prompt)[Systemic risk](/glossary/systemic-risk)

T
[Taxonomy](/glossary/taxonomy)[TDM exception](/glossary/tdm-exception)[tdmrep.json](/glossary/tdmrep-json)[TechArticle](/glossary/techarticle)[technographic data](/glossary/technographic-data)[Temperature](/glossary/temperature)[Tenant isolation](/glossary/tenant-isolation)[Test contamination](/glossary/test-contamination)[Text and data mining](/glossary/text-and-data-mining)[thin content](/glossary/thin-content)[Threat model](/glossary/threat-model)[threshold verdict](/glossary/threshold-verdict)[TLS downgrade](/glossary/tls-downgrade)[TLS handshake](/glossary/tls-handshake)[TLS-RPT](/glossary/tls-rpt)[token](/glossary/token)[token audience](/glossary/token-audience)[Tokenizer](/glossary/tokenizer)[Tool call](/glossary/tool-call)[Tool definition](/glossary/tool-definition)[Tool description](/glossary/tool-description)[tool lockfile](/glossary/tool-lockfile)[Tool permission](/glossary/tool-permission)[tool poisoning](/glossary/tool-poisoning)[Tool result](/glossary/tool-result)[Tool schema](/glossary/tool-schema)[Tool selection](/glossary/tool-selection)[tool shadowing](/glossary/tool-shadowing)[Top-k](/glossary/top-k)[Top-p sampling](/glossary/top-p-sampling)[Topic cluster](/glossary/topic-cluster)[topical authority](/glossary/topical-authority)[topicality](/glossary/topicality)[traceparent](/glossary/traceparent)[training data](/glossary/training-data)[Training-data summary](/glossary/training-data-summary)[Transactional query](/glossary/transactional-query)[Transformative use](/glossary/transformative-use)[Transformer](/glossary/transformer)[transparency log](/glossary/transparency-log)[triple](/glossary/triple)[True negative](/glossary/true-negative)[True positive](/glossary/true-positive)[Trust boundary](/glossary/trust-boundary)[trust center](/glossary/trust-center)[Trust list](/glossary/trust-list)[TTFB](/glossary/ttfb)

U
[unavailable_after](/glossary/unavailable-after)[uniform refusal](/glossary/uniform-refusal)[unreciprocated profile](/glossary/unreciprocated-profile)[unscored measurement](/glossary/unscored-measurement)[unverifiable](/glossary/unverifiable)[upgrade-insecure-requests](/glossary/upgrade-insecure-requests)[URI](/glossary/uri)[URL](/glossary/url)[URL discovery](/glossary/url-discovery)[URL inspection](/glossary/url-inspection)[Usage preference](/glossary/usage-preference)[user agent](/glossary/user-agent)[user-agent group](/glossary/user-agent-group)[User-generated link](/glossary/user-generated-link)[user-triggered fetch](/glossary/user-triggered-fetch)[user-triggered fetch exemption](/glossary/user-triggered-fetch-exemption)

V
[vantage point](/glossary/vantage-point)[variableMeasured](/glossary/variable-measured)[Variance](/glossary/variance)[Vary header](/glossary/vary-header)[Vector database](/glossary/vector-database)[Vector index](/glossary/vector-index)[verdict over DNS](/glossary/verdict-over-dns)[verified crawler](/glossary/verified-crawler)[Video sitemap](/glossary/video-sitemap)[Vision-language model](/glossary/vision-language-model)[visitor resolve](/glossary/visitor-resolve)[Vocabulary term](/glossary/vocabulary-term)

W
[WAF](/glossary/waf)[Web Bot Auth](/glossary/web-bot-auth)[WebMCP](/glossary/webmcp)[WebPage](/glossary/webpage)[well-known URI](/glossary/well-known-uri)[Wikidata](/glossary/wikidata)[Wildcard rule](/glossary/wildcard-rule)[Working memory](/glossary/working-memory)[WWW-Authenticate](/glossary/www-authenticate)

X
[x-default](/glossary/x-default)[X-Frame-Options](/glossary/x-frame-options)[X-Robots-Tag](/glossary/x-robots-tag)

## What a glossary cannot settle

No engine publishes its ranking function, so every definition here describing influence is a description of a mechanism, not a lever. Terms marked as proposals — llms.txt, agents.md, entity maps — are bets with a low cost, not standards anyone has committed to reading. Where adoption figures exist they are on [the dataset page](/data), with the population they were counted against.

Written by

[Emmanuel Orta](/author) · VSNARY

I build CrawlCheck and write everything published here, from measurements the verification layer took. Where a post got something wrong, the correction stays in the post.

[Every post, and where I got it wrong →](/author) · [Follow on LinkedIn →](https://www.linkedin.com/in/emmanuelorta)

## Questions about this page

#### Q What is an AI visibility score?

A 0–100 measure of whether an answer engine can reach a site, read it and quote a fact about it, computed from scored sections in that order. It measures what a site controls; it does not measure whether an engine actually cites the site.

#### Q What is llms.txt?

A plain-text or Markdown file at /llms.txt that names a site, what it does and the pages worth reading, for language-model agents. Adoption is low; CrawlCheck reports it as absent when the path returns HTML or a challenge page with a 200.

#### Q What is a verified crawler?

A request whose user-agent names a crawler and whose source IP falls inside the ranges that operator publishes. A user-agent alone is a claim; only the address can confirm it.

#### Q Why do the definitions state limits?

Because terms like citation, visibility and forged crawler are routinely used to mean more than any measurement supports. Each entry says what evidence the term can rest on.

We read your site the way crawlers do, from our own network, and every report says exactly what was asked for and what came back. [Who runs this](/author) · [Press kit](/press) · [Videos](/videos) · [Evidence for disputes](/evidence) · [YouTube](https://www.youtube.com/@crawlcheck) · [hello@crawlcheck.io](mailto:hello@crawlcheck.io)

#### How we measure, and our own machine files

CrawlCheck reads every site as itself. Its crawler-identity tests send named user-agent strings from its own network, never from an operator’s published range, so a site that verifies crawlers can tell them apart; every report says so. Results reflect what an anonymous non-browser client receives. [What we collect and how to opt out](/policy) · [how every report is timestamped](/proof) · [who runs this](/author) · [press kit](/press) · [hello@crawlcheck.io](mailto:hello@crawlcheck.io).

**Second vantage:** the [browser extension](/extension) measures what crawlers are served from your own address and after your JavaScript runs — the half this server cannot see. From 1.0.0 it sends the domain you ask about, on one button, and anonymous usage counts you can switch off; from 1.0.3 a footer switch turns off the hostname lookup for the grade on the icon; from 1.0.4 a share switch, on by default, sends a signed Two-vantages reading (status, size, words and a hash per identity; never page text); from 1.0.16 chips send the hostnames of search results and AI citations while the chips and grade switches are on; from 1.0.17 a readings switch, on by default, sends one signed page reading a day (numbers, the hostname and the page address with the query removed; never in incognito; at most 40 a day) and only while the share and grade switches are also on. Earlier versions send nothing.

**Our own machine layer for the agentic web:** [llms.txt](/llms.txt) · [llms-full.txt](/llms-full.txt) · [agents.md](/agents.md) · [SKILL.md](/SKILL.md) · [entitymap.json](/entitymap.json) · [media kit](/.well-known/media-kit.json) · [MCP server card](/.well-known/mcp/server-card.json) · [API catalog](/.well-known/api-catalog) · [OpenAPI](/openapi.json) · [robots.txt](/robots.txt) · [sitemap.xml](/sitemap.xml) · [security.txt](/.well-known/security.txt) — every one named in robots.txt, which is exactly what we check for on yours.

**Registry:** [MCP: Operator](/registry/mcp/operator) · [MCP](/registry/mcp) · [AI policy](/registry/aipolicy) | **Pages:** [IP ranges](/data/ip-ranges) · [WordPress](/fix/wordpress) · [Conduct: Mistralai user](/agents/conduct/mistralai-user) | **Blog:** [Claude-User uses two published address lists. We checked one](/blog/claude-user-comes-from-two-published-address-lists-we-checked-one) | **Glossary:** [Conditional request](/glossary/conditional-request) · [Retrieval latency](/glossary/retrieval-latency) · [Information architecture](/glossary/information-architecture) · [Captions](/glossary/captions) · [incident switch](/glossary/incident-switch)

[Contact](/contact) · [Partnerships](/partnerships) · [Terms of service](/terms) · [Refund policy](/refunds) · [Trust center](/trust) · [Subprocessors](/subprocessors) · [Changelog](/changelog) · [Report an error](/report-error)
