Can I allow ChatGPT search while blocking GPTBot?

Yes. OpenAI documents OAI-SearchBot and GPTBot as independent robots.txt controls. It gives the example of allowing OAI-SearchBot for search while disallowing GPTBot to indicate that crawled content should not be used to train its generative-AI foundation models.

Does blocking OAI-SearchBot stop an URL appearing anywhere in ChatGPT?

Not necessarily. OpenAI says an otherwise-disallowed URL may still appear as a title-only navigational link when it is discovered elsewhere and deemed relevant. It recommends noindex to prevent that outcome, while noting that its crawler must be able to crawl the page to read the tag.

Does allowing OAI-SearchBot guarantee a ChatGPT citation or search placement?

No. OpenAI describes it as an eligibility and discovery requirement, and says ChatGPT search uses multiple ranking factors and does not guarantee placement.

Is ChatGPT-User the control for ChatGPT search?

No. OpenAI says ChatGPT-User is for certain user-initiated actions, is not used for automatic web crawling and does not determine whether content can appear in Search. Use OAI-SearchBot to manage Search opt-outs and automatic crawl.

OAI-SearchBot, GPTBot and noindex: Choose the Right OpenAI Control

OpenAI's publisher guidance separates three decisions that are often collapsed into one: whether a page can be used in ChatGPT search, whether it is available for potential foundation-model training, and whether a discovered URL can be shown as a navigational link.

For a public page that you want eligible for ChatGPT search, allow OAI-SearchBot and make sure your CDN or host accepts OpenAI's published search-bot IP ranges. If you want to express that content should not be used for potential generative-AI foundation-model training, disallow GPTBot. If a page must not surface even as a title-only navigational link, use noindex and permit the relevant crawler to read it. These settings are independent, and none is a promise of placement, a summary, a citation, traffic, conversions or revenue.

That is the documented control plane. The workflow below is AEO Landscape's editorial interpretation of how a small team can make and verify the decision without turning a robots.txt edit into folklore.

Start with the outcome, not the bot name

The two OpenAI crawlers are not interchangeable.

OAI-SearchBot is the search setting

OpenAI says OAI-SearchBot is used to surface websites in ChatGPT search features. Its publisher FAQ says that a site must not block the bot for content to be included in summaries and snippets. The ChatGPT Search help page adds a second operational condition: the host or CDN must allow traffic from OpenAI's published search-bot IP ranges.

This makes OAI-SearchBot a search-access decision. It is not a switch that requests a citation. OpenAI says ChatGPT search uses multiple factors to find relevant, reliable information and that placement is not guaranteed. Keep that distinction in a decision record so a later result is not misread as proof that a robots rule "worked" or "failed."

GPTBot is the training-preference setting

OpenAI describes GPTBot as a crawler for content that may be used to train its generative-AI foundation models. Its crawler documentation says an operator can allow OAI-SearchBot for search while disallowing GPTBot for training; the controls are independent.

That is a useful separation for a publisher whose policy is "search discovery yes, potential training no." It is not an instruction to block GPTBot for every site. Treat it as a content-governance choice, document the scope and preserve the search access you still intend to allow.

noindex is about page presentation, with a crawl caveat

OpenAI's publisher FAQ covers a less intuitive case. If OpenAI learns about a disallowed page from another search provider or from crawling other pages, and considers it relevant, it may show the page title and link in ChatGPT Atlas without a summary or snippet. Its stated remedy is the noindex meta tag.

There is a constraint: a crawler has to be allowed to crawl the relevant page in order to read its noindex tag. A page policy that says "do not crawl this URL" and "read this tag on this URL" needs to be resolved deliberately. Do not assume that a broad bot block achieves both outcomes.

A page-by-page workflow

Apply the policy to groups of pages, then check representative URLs. A global default may be appropriate for public editorial content, while licensed, member-only or transactional pages may need a different decision.

1. Write three explicit answers for each page group

Record the answer before changing a configuration:

Should this group be eligible for ChatGPT search summaries and snippets?
Should this group be available to GPTBot for potential foundation-model training?
Should this group be allowed to appear as a title-only navigational link if it is discovered elsewhere?

The answers may differ. A clear policy prevents the familiar mistake of using a training control to solve a search-presentation question.

2. Check the live rules and the actual response

Read the production robots.txt, not a repository template or a CDN dashboard screenshot. Check for the specific OAI-SearchBot and GPTBot groups as well as broader wildcard rules that may override the intended access pattern.

For page groups intended to be eligible for ChatGPT search, confirm that the origin, WAF and CDN do not reject requests from OpenAI's published OAI-SearchBot IP ranges. OpenAI recommends allowing both the bot in robots.txt and the published IP ranges. Store the check date with the rule change, because infrastructure policies can move separately from SEO settings.

3. Inspect the rendered noindex tag where it is required

For a URL that must not become a navigational link, confirm that the response seen by a permitted crawler contains a valid noindex meta tag. This is not a reason to expose a sensitive page. It is a reason to decide whether the page should be public and crawlable at all, rather than relying on a contradictory combination of access rules.

The OpenAI guidance is specific to this page-presentation outcome. It does not say that noindex is a general control over model training, search ranking or all ChatGPT interactions.

4. Allow time for the system to adjust, then use the right evidence

OpenAI says its systems can take about 24 hours to adjust after a robots.txt update for search results. Treat that as an adjustment window for the stated access control, not as a promise about discovery, ranking or answer appearance.

After the change, keep evidence separate:

Server and CDN logs can show whether requests arrived and were served.
Analytics can identify ChatGPT referrals. OpenAI says ChatGPT appends utm_source=chatgpt.com to referral URLs when publishers allow OAI-SearchBot.
A fixed prompt sample can show what a user-facing answer happened to cite or link at a moment in time.

For that last check, CiteCue can keep a fixed buyer-question set and record cited sources across supported engines. Disclosure: AEO Landscape and CiteCue have common ownership. Read the full disclosure. CiteCue observations are not OpenAI crawler logs, ranking data, referral analytics or training records. They are useful for repeatable prompt sampling after a policy change, not for proving the cause of an outcome.

5. Recheck the policy on the next release

Put crawler settings on the release checklist when a site changes CDN, WAF, robots generation or page templates. Also recheck after a change to access control, because a noindex tag that disappears in a template refactor can undo a carefully made page-level decision.

Use the broader technical AEO crawler guide for origin and bot-mitigation checks. For a wider policy that separates discovery from training across providers, see Cloudflare's Disallow AI Training setting.

Do not use ChatGPT-User as a search signal

ChatGPT-User adds another source of confusion. OpenAI says this agent is used for certain user-initiated actions in ChatGPT and Custom GPTs. It is not used for automatic web crawling, and OpenAI says its robots rules may not apply because those actions begin with a user request.

Most importantly for an AEO policy, OpenAI says ChatGPT-User is not used to determine whether content can appear in Search. Use OAI-SearchBot for the search decision, and evaluate user-triggered agent access as its own operational and security question.

The practical rule

Do not ask whether a page is "allowed for AI." Ask which documented use you are deciding: ChatGPT search discovery, potential GPTBot training, or title-only navigational presentation. Set the relevant control, verify the live configuration and collect evidence from the layer that can actually observe it.

That will not manufacture visibility. It does leave your team with a testable access policy instead of an all-or-nothing bot block whose consequences nobody can explain.

Sources and verification

Sources checked 2 October 2026. The crawler purposes, access conditions, approximate robots update timing, noindex caveat and referral parameter are documented by OpenAI in the sources above. The page-group workflow, evidence separation and operational interpretation are AEO Landscape editorial guidance. This article does not claim that any setting guarantees placement, summaries, citations, traffic, conversions or revenue.