Your site is either being crawled by AI bots or ignored by them. The Crawler Log shows you which, with names, timestamps, and page-level data for 14 AI crawlers.
Download PluginThe AI Crawler Log detects 18 known AI bot user-agent patterns and records every visit to your WordPress database. You see which bots are crawling which pages, when they visited, and what response code they received.
Last 24 hours
/blog/what-is-aeo/ · 2 min ago
/services/hvac-repair/ · 8 min ago
/about/ · 15 min ago
/blog/seo-guide/ · 22 min ago
/pricing/ · 45 min ago
WordPress gives you no visibility into AI crawler traffic. Google Analytics does not track bots. Server logs require SSH access and pattern knowledge. The AI Crawler Log brings this data into your WordPress dashboard.
| bot_name | Identified bot (e.g. GPTBot, ClaudeBot) |
| user_agent | Full user-agent string (max 500 chars) |
| url | Requested URL (max 2083 chars) |
| response_code | HTTP status (200, 403, 404, etc.) |
| ip_address | Bot IP address |
| created_at | Visit timestamp |
Each bot visit is stored in the a dedicated log table table. The detection runs early in the WordPress request lifecycle (on the WordPress action, priority 1) so every page view by an AI bot is captured before any caching layers interfere.
Bot identification uses case-insensitive pattern matching against the user-agent string. The 18 patterns cover every major AI company that currently deploys web crawlers.
AI Crawler Log is included free in every AEO God Mode installation. Pair it with the AI Crawler Allowlist to control which bots you allow and which you block.
Fourteen: GPTBot, ChatGPT-User, PerplexityBot, ClaudeBot, Anthropic-AI, Google-Extended, CCBot, FacebookBot, Applebot-Extended, Amazonbot, Bytespider, Cohere-AI, DeepSeekBot, and Meta-ExternalAgent.
No. The plugin only writes to the database when it detects an AI bot in the user agent. Regular visitors trigger zero database writes. A single lightweight INSERT per bot visit.
Default is 90 days. After that, old entries are automatically purged. You can also clear the entire log manually from the dashboard.
Yes. The dashboard shows your top 20 most-visited URLs by AI bots over the last 30 days, plus a per-bot breakdown and daily visit chart.