CrawlCheck

Glossary · Machine files

HTTP Message Signatures

A standard, RFC 9421, for signing HTTP requests so a server can verify who sent them. Web Bot Auth applies it to crawlers: the operator publishes a key directory, signs each request, and a site can verify the signature instead of trusting a user-agent string or an IP list.

What HTTP Message Signatures means

A standard, RFC 9421, for signing HTTP requests so a server can verify who sent them. Web Bot Auth applies it to crawlers: the operator publishes a key directory, signs each request, and a site can verify the signature instead of trusting a user-agent string or an IP list. It answers who sent the request, not whether they may have what they asked for.

What HTTP Message Signatures can and cannot support

It can supportIt cannot support
A standard, RFC 9421, for signing HTTP requests so a server can verify who sent them. Web Bot Auth applies it to crawlers: the operator publishes a key directory, signs each request, and a site can verify the signature instead of trusting a user-agent string or an IP list.It answers who sent the request, not whether they may have what they asked for.

Terms this definition uses

Web Bot Auth

Related terms in Machine files

The files addressed to machines rather than readers, and what publishing one does and does not buy.

llms.txt · llms-full.txt · agents.md · entity map · entity map parity · machine layer · well-known URI · MCP · agent card · sitemap index · OpenAPI specification · ai.txt · Tool definition · SKILL.md · sitemap origin error · soft 404 on a machine file · tdmrep.json · TDM exception · security.txt · Content Signals · changefreq and priority · staging leak · host duplicate

Questions about HTTP Message Signatures

What is HTTP Message Signatures?

A standard, RFC 9421, for signing HTTP requests so a server can verify who sent them. Web Bot Auth applies it to crawlers: the operator publishes a key directory, signs each request, and a site can verify the signature instead of trusting a user-agent string or an IP list.

What does HTTP Message Signatures not show or guarantee?

It answers who sent the request, not whether they may have what they asked for.

Which area of the glossary does HTTP Message Signatures belong to?

Machine files: The files addressed to machines rather than readers, and what publishing one does and does not buy.

← Content Signals  ·  changefreq and priority →

See it in the full glossary · 668 terms across 19 areas.