Google-CloudVertexBot

Google runs more crawlers than any other operator here, and the important thing is that they are genuinely separate. Search indexing, AI training, research fetches and the agents that act for a person all carry different tokens and appear in different published address files. Treating them as one 'Google' decision is the most expensive mistake available on this page.

On this site Google-CloudVertexBot is counted as training, which is the row it appears in on your dashboard. The category matters because it decides which number moves when this crawler visits, and the four are reported separately for exactly that reason.

Broad and slow: many pages, few repeats, spread over days or weeks rather than concentrated. Training crawls tend to want everything once, so the long tail of a site sees traffic it never gets from search — old posts, deep archive pages, anything reachable by a link. Bandwidth is the usual reason people notice these at all.

Checked as ip ranges + reverse dns. The user agent has to carry the token, and then the address is compared against what Google publishes — refetched every six hours, so a newly announced range is picked up without a deploy.

A request that matches neither is recorded as unverified rather than as a forgery: a published list can be incomplete, and an accusation needs evidence. Only a reverse DNS record resolving to somebody else produces a spoofed verdict, which is proof rather than the absence of it.

Google stops collecting your pages as training material. Nothing about how you appear in search or in live answers changes — those are separate crawlers with separate rules.

Nothing you see today changes if you block this. Training crawls feed models that ship months later, so the decision is about whether your work becomes part of a commercial model, not about this quarter's traffic.

The honest counter-argument is that being in the training data is part of how a model comes to mention you at all. There is no measurement that settles this either way — anyone who tells you there is has not looked — so it is a decision about your terms rather than about your numbers.

Google sends 7 crawlers and gives each its own token, so they can be allowed and refused separately. The tokens differ by a few characters and what the fetch means does not:

Google-NotebookLM — ai assistant. Fetches a page a person added to a NotebookLM notebook. Google-Agent — ai assistant. A Google agent fetching on a person's behalf. Gemini-Deep-Research — ai assistant. Gemini's deep research mode gathering sources. Google-Read-Aloud — ai assistant. Fetches a page so it can be read out loud. Googlebot — search indexing. Google's search crawler, and the input to AI Overviews. GoogleOther — ai training. Google product teams fetching public content, including research.

Confusing two of these is the most common mistake with this operator, and it is expensive in one direction: a rule meant for the training crawler that lands on the search crawler removes you from results without stopping any training.