PerplexityBot

The most common mistake with this crawler is blocking it to prevent training. Perplexity's own documentation says it builds a search index and does not train foundation models on what it fetches, so a block costs you citations in Perplexity's answers and prevents nothing you were worried about.

Perplexity's product is an answer with citations, so its crawling exists to build a source list rather than a model. The company is unusual in saying so explicitly: its documentation states that it does not train foundation models on what it fetches.

On this site PerplexityBot is counted as indexing, which is the row it appears in on your dashboard. The category matters because it decides which number moves when this crawler visits, and the four are reported separately for exactly that reason.

A steady crawl over time rather than a burst: robots.txt first, then pages in waves, revisiting the ones that change often. Volume tracks how large and how frequently updated your site is. A sudden increase usually follows a sitemap change or a burst of new pages, and a sudden stop is worth investigating — it often means a robots rule or a server error rather than a decision by the operator.

Perplexity publishes neither a machine-readable list of the addresses this crawler uses nor a reverse DNS record pointing back at itself. The user-agent header is therefore the only evidence, and anyone can send it — so a visit is recorded as an unverified claim rather than as a confirmed fetch.

That is not a gap in the measurement so much as an honest ceiling. A report that showed these as verified would be inventing a confidence nobody can support, and the number it produced would be the first thing to fall apart when somebody asked where it came from.

You leave Perplexity's index, so you stop being offered as a source in its answers. Perplexity's documentation is explicit that this bot builds a search index rather than training a model, so blocking it to prevent training costs visibility and prevents nothing.

Treat this as a search-engine decision rather than an AI one, because that is what it is. The question is whether the people you want are looking through this engine, and your own analytics answer it better than any general advice can.

It is worth checking what you actually lose before writing the rule. Blocking an index crawler is one of the few changes here with an effect that compounds quietly: nothing breaks on the day, and traffic simply stops arriving over the following weeks.

Perplexity sends 2 crawlers and gives each its own token, so they can be allowed and refused separately. The tokens differ by a few characters and what the fetch means does not:

Perplexity-User — ai assistant. Fetches a page Perplexity is about to cite in an answer.

Confusing two of these is the most common mistake with this operator, and it is expensive in one direction: a rule meant for the training crawler that lands on the search crawler removes you from results without stopping any training.