---
title: "AI crawler access checker"
description: "Check if GPTBot, ClaudeBot, PerplexityBot, Google-Extended and other AI crawlers can access your site, and whether you publish llms.txt."
url: "https://mozolite.com/tools/ai-crawlers"
---

# AI crawler access checker

See whether ChatGPT, Claude, Perplexity, Gemini and other AI crawlers are allowed to read a page, and whether the site has an llms.txt file.

Use the tool: https://mozolite.com/tools/ai-crawlers

## Why this matters now

A growing share of people ask AI assistants instead of searching. Those assistants rely on their own crawlers to read and cite pages. If robots.txt blocks them, your content cannot be quoted or linked in their answers.

## Training and search are separate

Several companies use different user agents for model training and for live answers. For example OAI-SearchBot and ChatGPT-User fetch pages for answers, while GPTBot is used for training. You can allow one and block the other.

## About llms.txt

llms.txt is a proposed plain-text file at the root of a site that gives language models a short summary and a list of key pages. It is not a standard yet and no major assistant has committed to reading it, so treat it as optional.
