GPTBot

GPTBot is the token most site owners have heard of, and the one most often blocked for the wrong reason. It collects material that may be used to train future models; it plays no part in ChatGPT answering a question today. If you have blocked GPTBot and are wondering why ChatGPT still quotes your site, this is why — that is ChatGPT-User, a different crawler with a different rule.

OpenAI runs four separate crawlers and gives each one its own token, which is more considerate than it first appears: it means you can refuse the training crawl and keep the one that puts you in ChatGPT's answers, using two lines in a robots file. Most operators on this site do not offer that choice.

On this site GPTBot is counted as training, which is the row it appears in on your dashboard. The category matters because it decides which number moves when this crawler visits, and the four are reported separately for exactly that reason.

Broad and slow: many pages, few repeats, spread over days or weeks rather than concentrated. Training crawls tend to want everything once, so the long tail of a site sees traffic it never gets from search — old posts, deep archive pages, anything reachable by a link. Bandwidth is the usual reason people notice these at all.

Checked as ip ranges + reverse dns. The user agent has to carry the token, and then the address is compared against what OpenAI publishes — refetched every six hours, so a newly announced range is picked up without a deploy.

A request that matches neither is recorded as unverified rather than as a forgery: a published list can be incomplete, and an accusation needs evidence. Only a reverse DNS record resolving to somebody else produces a spoofed verdict, which is proof rather than the absence of it.

Nothing today. GPTBot only collects material for future model training — blocking it does not remove you from ChatGPT search or stop ChatGPT citing you when someone asks. It is the cleanest single block available if your objection is to training.

Nothing you see today changes if you block this. Training crawls feed models that ship months later, so the decision is about whether your work becomes part of a commercial model, not about this quarter's traffic.

The honest counter-argument is that being in the training data is part of how a model comes to mention you at all. There is no measurement that settles this either way — anyone who tells you there is has not looked — so it is a decision about your terms rather than about your numbers.

OpenAI sends 5 crawlers and gives each its own token, so they can be allowed and refused separately. The tokens differ by a few characters and what the fetch means does not:

ChatGPT-User — ai assistant. Fetches a page while ChatGPT is answering someone's question. ChatGPT-Agent — ai assistant. ChatGPT's agent mode browsing on a person's behalf. OAI-SearchBot — search indexing. Builds the index behind ChatGPT search. OAI-AdsBot — other bots. Checks pages submitted as ChatGPT ads.

Confusing two of these is the most common mistake with this operator, and it is expensive in one direction: a rule meant for the training crawler that lands on the search crawler removes you from results without stopping any training.