CrawlCheck

Glossary · HTTP, edge and rendering

410 response

“Gone,” indicating the resource is intentionally unavailable and likely permanently removed. Crawlers may process it faster than a 404, but timing is not guaranteed.

HTTP, edge and rendering

The delivery layer between an origin and a client: proxies, caches, redirects, status codes and the headers that decide what a crawler actually received.

Origin server · Reverse proxy · CDN · Edge node · WAF · Bot score · TLS handshake · DNS resolution · DNS timeout · Connection timeout · Read timeout · Redirect chain · Redirect loop · 301 redirect · 302 redirect · 307 redirect · 308 redirect · 304 response · 404 response · 429 response · 500 response · 502 response · 503 response · 504 response · HEAD request · GET request · Content type · MIME sniffing · Content encoding · Compression · Brotli · ETag · Last-Modified · If-None-Match · Cache hit · Cache miss · Cache key · Vary header · Stale response · Stale-while-revalidate · Edge redirect · Origin bypass · Response parity · Header parity

404 response  ·  429 response

See it in the full glossary · 579 terms across 19 areas. Scan a site to see which of these apply to it.