Allen AI
AI2Bot
The Allen Institute's crawler for open AI research datasets.
When you see it: A non-profit research institute collected your page for an open dataset.
How we check it is really Allen AI
- 01
The user agent has to contain AI2Bot.
Allen AI publishes neither IP ranges nor a reverse DNS record, so this is the honest ceiling: the visit is recorded as a claim. We would rather show you an unverified row than a tick we cannot back.
What each result means
- verified
- The address matched Allen AI's own published records. This really was them.
- unverified
- We could not confirm it, and that is all it means. The operator may publish nothing to check against, a published list may be behind the addresses actually in use, or DNS may simply have been slow. It is not an accusation.
- spoofed
- The address belongs to somebody else — its own reverse DNS record resolves to a different owner. That is positive evidence of a forgery rather than a failure to find evidence, which is why it is reported as its own result and never folded into the row above.
The distinction costs nothing to make and is the reason these numbers can be shown to someone who will ask where they came from. A report that cannot separate “we do not know” from “this was faked” invites exactly one question it cannot answer.
No list to check against
Allen AI publishes neither an address file nor a reverse DNS record for this crawler. There is nothing to check a request against, which is why every visit from it is recorded as an unverified claim.
- Operator
- Allen AI
- Token
- AI2Bot
- Counted as
- AI training
- Verification
- User agent only
- Runs JavaScript
- No
- Operator docs
- Published
User agent
AI2BotMatched as a token, not as a full browser string. Allen AI adds and changes the version and product text around it without notice, so a filter pinned to the whole header stops working the first time they bump a version while one matching AI2Bot keeps working. The same string is what Allen AI names in robots.txt.
One real request
Mozilla/5.0 (compatible) AI2Bot (+https://www.allenai.org/crawler)Your analytics never mentioned AI2Bot because it cannot see it.
This crawler takes your HTML and leaves without running a line of JavaScript, so a browser tag records nothing. TrueStat reads it server-side, checks the address, and shows you which pages it fetched — including the ones that returned a 404.