# See your website the way AI bots see it. > Free AI crawler checker. See the raw HTML GPTBot gets from any page, how much of your content bots miss, and the robots.txt, schema, and SEO gaps that keep… Source: https://contengi.com/ai-bot-view Free tool / AI bot view ChatGPT, Claude, and Perplexity read the raw HTML of your page and skip the JavaScript. Paste a URL to see what they get, how much of your page they miss, and the robots, schema, and SEO gaps between you and an AI answer. ## What the scan checks Four questions decide whether an AI engine can quote your page: can its crawler get in, can it read the content, does it understand what the page is, and can it find the rest of your site. ### Bot readable content We load your page twice: once as GPTBot, with no JavaScript, and once in a real browser. Every heading, paragraph, and link a visitor sees gets matched against the bot’s copy. The score is the share of words that made it across. ### AI crawler access Your robots.txt, read the way crawlers read it, for 14 bots across AI search and AI training. The scan also catches firewalls that turn GPTBot away before robots.txt comes into it. ### SEO signals Title, meta description, H1, canonical, language, internal links, alt text, and link previews. All read from the HTML a bot downloads, so a title that only appears once scripts run gets flagged. ### Machine-readable extras Schema.org structured data, llms.txt, and your sitemap. Schema injected by a tag manager or a script shows up as a warning, because crawlers that don’t render never see it. ## Why AI bots miss half a page Google renders JavaScript, given time. Most AI crawlers skip it. When Vercel studied crawler traffic across its network in late 2024, none of the major AI crawlers ran JavaScript, OpenAI’s and Anthropic’s included. They downloaded script files and never executed them. So a page assembled in the browser can look perfect to you and blank to ChatGPT. Single-page apps, reviews pulled in by a widget, pricing that loads on click, and FAQs hidden behind a script all fall into the gap. The fix is to put the content in the HTML your server sends, through server-side rendering or a static build. After that, the job is giving bots something worth quoting. [Request beta access](https://contengi.com/beta-access) [Check your AI visibility](https://contengi.com/llm-tracker) FAQ ## Common questions ### Which bot does the scan pretend to be? GPTBot, OpenAI’s crawler, using its published user agent. If your server refuses that request, we fetch the page again as a normal browser so you still get a result, and flag the refusal as a warning. ### Does Google see my JavaScript content? Usually. Googlebot renders pages in a headless Chrome, though rendering can trail the first crawl. ChatGPT, Claude, and Perplexity crawlers download the HTML and stop there, which is why the score is based on the raw HTML. ### Should I block AI training bots? That’s your call. Blocking GPTBot or ClaudeBot keeps your pages out of future training data. Blocking the search bots (OAI-SearchBot, Claude-SearchBot, PerplexityBot) keeps you out of the answers themselves, which is rarely what anyone wants. The scan lists the two groups separately for that reason. ### What counts as a missing block? Any heading, paragraph, list item, link, or button a visitor can see that isn’t in the raw HTML. Cookie banners, chat widgets, and pop-ups are left out of the score, since every site injects those with JavaScript and nobody wants them quoted. ### What is llms.txt? A plain-text file at yoursite.com/llms.txt that summarises your site and links to its key pages, written for AI models to read. It’s still a proposal and support across AI engines is patchy, but it’s a small file to write, so the scan suggests one. ### Could the real GPTBot see something different? Sometimes. Some firewalls let the real GPTBot through from OpenAI’s IP addresses while blocking everyone else using its name, and a few sites serve bots a different page altogether. The scan shows what a bot gets from an ordinary request, which is the right thing to fix against. ### Is the page I scan stored? No. We fetch it, analyse it, and send you the result. The only thing we log is your IP address and the time, so one person can’t run hundreds of scans an hour. ## Readable is step one. Worth quoting is the rest. Contengi writes on-brand, well-structured content from your knowledge base, so when AI engines read your site they find something to cite. [Request beta access](https://contengi.com/beta-access) [Try it out](https://contengi.com/demo) Also free: the [LLM visibility check](https://contengi.com/llm-tracker) and the [AI slop cleaner](https://contengi.com/ai-slop-cleaner).