Is Your Website Visible to ChatGPT? How to Check and Fix It

Here is a question most site owners have never thought to ask: can ChatGPT actually see your website?

Not "can a user paste your URL into ChatGPT." Can OpenAI's crawler — GPTBot — access your site, read your content, and include it in the training data and real-time search results that power ChatGPT's responses?

For a surprising number of websites, the answer is no. And the site owner has no idea.

ChatGPT's knowledge comes from two sources:

1. Training data — a massive snapshot of the web, collected by OpenAI's crawlers before the model's knowledge cutoff
2. Real-time web search — when ChatGPT has browsing enabled, it uses live web results to answer current questions

Both rely on OpenAI's web crawler, GPTBot, being able to access your site.

If GPTBot is blocked — or if your site's content is hidden behind JavaScript that the crawler cannot execute — your site effectively does not exist to ChatGPT.

The most common reason sites are invisible to AI engines is a misconfigured robots.txt file.

robots.txt is a text file at the root of your domain (e.g., yourdomain.com/robots.txt) that tells crawlers which parts of your site they can and cannot access.

Many sites block AI crawlers without realizing it. This happens because:

• A developer added a blanket "block all unknown bots" rule
• A security plugin or WAF is blocking unfamiliar user agents
• An old robots.txt template was used that predates AI crawlers
• Someone intentionally blocked AI crawlers during a controversy and forgot to reverse it

1. Go to yourdomain.com/robots.txt in your browser
2. Look for any of these entries:

User-agent: GPTBot
Disallow: /

User-agent: *
Disallow: /

The first explicitly blocks ChatGPT. The second blocks all crawlers — including AI ones.

To allow GPTBot to crawl your entire site, add this to your robots.txt:

User-agent: GPTBot
Allow: /

To allow all AI crawlers:

User-agent: GPTBot
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: ClaudeBot
Allow: /

User-agent: Google-Extended
Allow: /

ChatGPT is not the only AI engine that matters. Here are the major AI crawlers and what they power:

Each of these can be independently allowed or blocked in your robots.txt. Most sites should allow all of them unless there is a specific reason not to.

Even if your robots.txt is clean, your content might still be invisible to AI crawlers.

Many modern websites render their content using JavaScript — the page loads a blank shell, then JavaScript fills in the actual content. This works fine for human visitors, but many crawlers (including some AI crawlers) cannot execute JavaScript. They see the blank shell and index nothing.

Signs your site might have this problem:

• Your site is built with a JavaScript framework (React, Vue, Angular) without server-side rendering
• When you view the page source (Ctrl+U), the main content is not visible in the HTML
• Google Search Console shows your pages as "Crawled — currently not indexed"

The fix is server-side rendering (SSR) or static site generation (SSG) — ensuring your content exists in the initial HTML response, not just after JavaScript runs.

Even if your site is fully crawlable, some types of content are effectively invisible to AI engines:

• Content behind login walls — AI crawlers cannot authenticate
• Content in PDFs or images — not indexed as text
• Content in iframes — often not attributed to your domain
• Content loaded via API calls — not present in the initial page HTML

If your most important content lives in any of these formats, consider adding a text version on a crawlable page.

Beyond checking robots.txt manually, the most reliable way to assess your AI search visibility is a full audit that checks all the signals that matter — not just crawlability, but schema markup, entity clarity, content structure, and more.

Our free audit checks 40+ SEO and AEO signals and tells you:

• Whether your site is accessible to major AI crawlers
• Your AIReady™ Score (0–100)
• A prioritized list of exactly what to fix

Get your free audit → Find out if ChatGPT can actually see your site.

Further reading:

• How ChatGPT and Perplexity decide which sites to cite
• Perplexity SEO: how to get your site cited in Perplexity answers
• Schema markup: the beginner guide for site owners