# robots.txt for veed.io — corrected by CrawlCheck, 2026-11-03 # Source: the file the site served at the time. Every change is listed here; nothing else was touched. # Changes: # - Read from https://www.veed.io/robots.txt (the apex redirected there); publish at the host you serve. # - 1 named group lacked Disallow lines the * group carries. A crawler obeys only its own most specific group and ignores *, so those rules never applied to it. Copied in. # Verify after publishing: curl -sI https://veed.io/robots.txt must show text/plain, on the bare URL, with no cache-buster. User-agent: OAI-AdsBot Allow: / Disallow: /cdn-cgi/ Disallow: /api/v1 Disallow: /api/v2 Disallow: /api/graphql Disallow: /login$ Disallow: /signup$ Disallow: /new$ User-agent: * Allow: /cdn-cgi/image/ Disallow: /cdn-cgi/ Disallow: /api/v1 Disallow: /api/v2 Disallow: /api/graphql Disallow: /login$ Disallow: /signup$ Disallow: /new$ Sitemap: https://www.veed.io/sitemap.xml