CrawlCheck

Directory · ML platforms and experiment tracking

DataRobot — what a crawler receives from datarobot.com

Website measurement, not a product review. This page records what machine clients received from datarobot.com. It does not evaluate DataRobot software, features, pricing or support.

82
Cgrade

AI assistants like ChatGPT can read your site, but find little to quote about your product.

AI visibility 82/100, measured 2026-11-06 05:32 UTC from outside its network, as each named AI and search crawler and as a browser. An engine can reach and read this site. What is left is giving it something specific to say.

Held at C A medium-severity finding sets the letter.

Open the full report datarobot.com ↗

4

findings

✓

llms.txt on its host

✗

entity map on its host

11

scans on record since 2026-09-18

The three stages, in the order an engine hits a site

Each stage gates the next: an engine that cannot reach a page never reads it, and one that cannot read it never quotes it.

92

Stage 1 · 40% of the score

Reach

Can a named answer-engine crawler get your pages at all?

Every named crawler was served the same page a browser gets.

79

Stage 2 · 30% of the score

Read

Once it has the bytes, can it find the words?

The content is buried in markup, or the files that guide a crawler are missing.

73

Stage 3 · 30% of the score

Quote

Is there a specific fact it can state and attribute?

An assistant would have to paraphrase your page instead of quoting a fact.

The stage to fix first is quote, the lowest of the three.

What to fix

4 findings on this site, 1 of them scored serious. Every one is named below; the top one is shown in full, free. What we saw, why each matters, how to fix it and the bytes behind it are what a licence buys — or we implement the whole list for you.

1 Medium 3 Note

1The ARD manifest breaks the specification10 specification errors in 10 entries: [datarobot-model-training] Identifier 'urn:ai:github.com:datarobot-oss:datarobot-agent-skills:model-training' does not maMediumDeveloper

What we saw

10 specification errors in 10 entries: [datarobot-model-training] Identifier 'urn:ai:github.com:datarobot-oss:datarobot-agent-skills:model-training' does not match RFC 8141 URN pattern 'urn:air:<publisher>:<namespace>:<agent-name>'. [datarobot-model-deployment] Identifier 'urn:ai:github.com:datarobot-oss:datarobot-agent-skills:model-deployment' does not match RFC 8141 URN pattern 'urn:air:<publisher>:<namespace>:<agent-name>'. [datarobot-predictions] Identifier 'urn:ai:github.com:datarobot-oss:datarobot-agent-skills:predictions' does not match RFC 8141 URN pattern 'urn:air:<publisher>:<namespace>:<agent-name>'. (+7 more) /.well-known/ai-catalog.json

Why it matters

The site's ARD manifest (/.well-known/ard.json or the source it advertises) fails the rules every entry must meet: a urn:air:<publisher>:<name> identifier, a displayName, a media type, and exactly one of url or data. A registry rejects the entry, so the resource it describes is not discoverable; where the trust manifest names an identity that does not belong to the URN's publisher, the entry is rejected as namespace squatting.

How to fix it

Give every entry an identifier of the form urn:air:<your-domain>:<name>, a displayName, a type (an IANA media type such as application/mcp-server-card+json) and exactly one of url or data. If an entry carries a trustManifest, its identity must belong to the same domain as the URN. Validate with the official tester: npx ajv-cli validate -s ard-entry.schema.json -d ard.json.

Developer: a template, theme or application change — a developer touches it

Fix it with a coding agent

Paste into Claude Code, Cursor or Codex, opened on the code that serves this site.

Show prompt
You are working in the codebase (or server/CDN configuration) that serves datarobot.com. A crawler audit found the problem below. Fix it at the source, keep the change as small as possible, and do not change anything a visitor sees unless the fix requires it.

Finding: The ARD manifest breaks the specification (ARD_MANIFEST_INVALID, medium severity)
Page: https://datarobot.com/.well-known/ai-catalog.json
What was measured: 10 specification errors in 10 entries: [datarobot-model-training] Identifier 'urn:ai:github.com:datarobot-oss:datarobot-agent-skills:model-training' does not match RFC 8141 URN pattern 'urn:air:<publisher>:<namespace>:<agent-name>'. [datarobot-model-deployment] Identifier 'urn:ai:github.com:datarobot-oss:datarobot-agent-skills:model-deployment' does not match RFC 8141 URN pattern 'urn:air:<publisher>:<namespace>:<agent-name>'. [datarobot-predictions] Identifier 'urn:ai:github.com:datarobot-oss:datarobot-agent-skills:predictions' does not match RFC 8141 URN pattern 'urn:air:<publisher>:<namespace>:<agent-name>'. (+7 more)
Why it matters: The site's ARD manifest (/.well-known/ard.json or the source it advertises) fails the rules every entry must meet: a urn:air:<publisher>:<name> identifier, a displayName, a media type, and exactly one of url or data. A registry rejects the entry, so the resource it describes is not discoverable; where the trust manifest names an identity that does not belong to the URN's publisher, the entry is rejected as namespace squatting.
How to fix it: Give every entry an identifier of the form urn:air:<your-domain>:<name>, a displayName, a type (an IANA media type such as application/mcp-server-card+json) and exactly one of url or data. If an entry carries a trustManifest, its identity must belong to the same domain as the URN. Validate with the official tester: npx ajv-cli validate -s ard-entry.schema.json -d ard.json.
Detected platform: WordPress. Make the change where this platform controls it.
Rule and examples: https://crawlcheck.io/rules/ARD_MANIFEST_INVALID
Full report (markdown): https://crawlcheck.io/r/zhqwtnaw23.md

When you are done:
1. Tell me which files or settings you changed and why.
2. If the fix is a server, CDN or cache setting you cannot change from the code, say exactly where it is set.
3. After deploying, re-scan at https://crawlcheck.io/?url=datarobot.com . The fix counts when ARD_MANIFEST_INVALID is gone from the new report.
Why this was decided
RuleARD_MANIFEST_INVALID · rule pack sv34 · scope core
Evaluated2026-11-06 05:32:17 UTC
Decisiond1:4dadc8c5a71f6a3bc491 · finding f1:381d1a137a28ef133f90 resolve either at /api/graph/<id> with a key
Observercrawlcheck-worker build c699237f from LHR · audit v4

What it rests on

IdentityStatusBytessha256Kept
crawlcheck200167,38453459ead8c33kept
crawlcheck2003,340d62267392e84already_held
crawlcheck2001,94451eeb3c0f1d1kept
crawlcheck2001,94451eeb3c0f1d1kept
crawlcheck2001,94451eeb3c0f1d1kept
crawlcheck2005,473331d6af5de4aalready_held
crawlcheck404140,4895eeae1a5f1fakept
anon200167,38453459ead8c33kept

No recorded fetch names this path; these are the scan’s recorded fetches.

15 more in the manifest.

Artifacts: manifest:5bd6c1864a35012961ab4187c54f5db4960a68e92c22bf99548e781923799f41

Grade effect Caps the letter at C. Without it the letter would be B (score 82). Moves no points; the score is built from the measured rows.

Rule ARD_MANIFEST_INVALID under scoring rules v34 · fingerprint ARD_MANIFEST_INVALID|/.well-known/ai-catalog.json · new on this scan (seen in 1 of 11 readings)

Verify the fix Re-scan after the change. This finding is cleared when ARD_MANIFEST_INVALID is no longer reported at /.well-known/ai-catalog.json; if it comes back after clearing, it is marked a regression, not a new finding.

ARD_MANIFEST_INVALID · full transcript

2The homepage is reached through a chain of redirects🔒 What we saw, why it matters, the fix and the evidence bytes are in the licensed report.Note
3The ARD manifest is served only under its old name/.well-known/ai-catalog.json🔒 What we saw, why it matters, the fix and the evidence bytes are in the licensed report.Note
4A missing agent file answers with a full page🔒 What we saw, why it matters, the fix and the evidence bytes are in the licensed report.Note

Open the other 3

What we saw, why it matters, how to fix it, and the evidence bytes for every finding — on this report and every re-scan. From $29 a month.

Unlock the findings

Or have them fixed

We implement the list on your site and re-measure, with a sealed before and after. One site, $100 once.

Get this fixed — $100

Sites like yours that fixed the same problem

Matched on WordPress + Fastly + Pantheon. A fix counts once two readings confirm it; sites are not named. Read from the technology record of 2026-10-27.

serves no /agents.md (optional)

3 sites on a stack like yours have made this fix (Published agents.md); 3 still hold it. 16 on any stack.

  • similar site A (WordPress + Cloudflare + Cloudflare Bot Management; shares platform): fixed 2026-11-04, still fixed today
  • similar site B (WordPress + Cloudflare + Cloudflare Bot Management; shares platform): fixed 2026-10-31, still fixed today
  • similar site C (WordPress; shares platform): fixed 2026-10-30, still fixed today

serves no /entitymap.json (optional)

1 site on a stack like yours has made this fix (Published an entity map); 1 still hold it. 2 on any stack.

  • similar site A (WordPress + Cloudflare; shares platform): fixed 2026-10-29, still fixed today

The signed answer lists every match.

The chain a crawler follows

The chain breaks at entity graph. Everything after that point is only reachable by a crawler guessing the conventional path.

✓robots.txtEvery crawler reads this first
✓sitemapNamed in robots.txt and resolves
✓llms.txtPoints onward, links stay on this host
✗entity graphParses as JSON and points home

A machine file that does not parse is worth less than one that is absent, because it looks present. Rows behind this.

Where the bytes go

Of the 167,329 decompressed bytes the homepage delivers, 5.2% is text a reader or a model can actually use. Most of what a crawler downloads here is not words.

  • Readable text 5.2% · 8,679 B
  • Markup & attributes 74.9% · 125,269 B
  • Structured data (JSON-LD) 1.1% · 1,904 B
  • Inline CSS 3.2% · 5,436 B
  • Inline JavaScript 4.8% · 7,991 B
  • Inline SVG 10.5% · 17,487 B
  • HTML comments 0.3% · 563 B

The individual blocks that weigh the most, so the fix is a search rather than a hunt:

  • Inline SVG · 6,597 B (3.9%) — unnamedopens <path d="M37.5282 14.6924C38.8792 13.486 39.5653 11.743 39.5653 9.5145
  • Inline SVG · 6,518 B (3.9%) — unnamedopens <path d="M37.5282 14.6924C38.8792 13.486 39.5653 11.743 39.5653 9.5145
  • Inline CSS · 4,354 B (2.6%) — WordPress theme or block stylesopens :root{--wp-block-synced-color:#7a00df;--wp-block-synced-color--rgb:122
  • Inline JavaScript · 2,451 B (1.5%) — jQueryopens window._vwo_code || (function() { var account_id=841954, version=2.0,
  • Inline SVG · 2,195 B (1.3%) — unnamedopens <rect width="390" height="26" transform="translate(0 0.253418)" fill="
Section average over 2 scans on this score version · held at 82 since 2026-11-06
0501002026-10-27 02:04 · 81/100 · 2 findings2026-11-06 05:32 · 82/100 · 4 findings2026-10-272026-11-06

What changed

  1. 2026-11-06score moved 81 → 83score-up
  2. 2026-11-06a new finding appeared: the ard manifest breaks the specificationappeared
  3. 2026-11-06a new finding appeared: the ard manifest is served only under its old nameappeared
  4. 2026-10-14the homepage began taking more than one redirect to reachappeared
  5. 2026-10-03score moved 81 → 82score-up

11 scans on record, first 2026-09-18. Score movements are shown only between readings taken under the same score version; finding codes are comparable across all of them.

Is this your site?

This listing reads the public record. Claim it and Watch keeps that record on your terms: every list this page holds back, finding age, and twice-daily re-measurement. Free for 30 days, no card.

Claim the record for datarobot.com

Is this your company?

This listing came from a fetch, not a submission, so nobody was asked. If datarobot.com is yours you can prove it in a minute and for nothing — a DNS TXT record or one meta tag on the home page, both checkable by anyone.

Claiming marks you as the verified owner and lets you correct the category and the company name. It moves no score, no rank and no position.

Claim this listing — free or ask about a verified profile

Dispute or re-scan

If a reading here is wrong, it is our instrument that is wrong, and we want to know. Tell us what is wrong and the site is re-measured; the result is published as measured. Nothing about a listing, a payment or a request changes a number.

By email instead: hello@crawlcheck.io. Category “ML platforms and experiment tracking” is our label for what the product is primarily sold as; it is not scored. Back to the directory.