About Known Agents

The Company

Known Agents is an AI agent and bot traffic analytics platform that helps website owners track, control, and monetize the growing wave of AI agents, crawlers, and scrapers visiting their websites. It was founded in New York City by Gavin King (previously Meta/Instagram, Vimeo, PayPal, Intuit). If you want to reach out, feel free to send us an email.

We Are Not "Anti-Bot"

Our mission is to accelerate the transition to an agentic internet where trusted AI traffic unlocks new levels of human value.

• Accelerate the agentic internet: With AI, the internet is inevitably transitioning to a place where more agents visit websites than humans.
• Build trust between humans and agents: Create a transparent ecosystem where humans and agents are easily able to self-identify.
• Turn agent traffic into human value: Maximize the productivity of humans operating AI agents, and the value for humans creating content that AI agents interact with.

Frequently Asked Questions for Website Owners

Is Known Agents free?

Known Agents is a freemium service. The free tier gives you immediate visibility into your hidden bot traffic with no credit card required. You can upgrade to unlock higher event volumes, additional projects for multiple websites, CSV export, API access, and priority support. Full details are on the pricing page.

What types of AI agents and bots does Known Agents track?

Known Agents tracks thousands of known AI agents, web crawlers, and scrapers by inspecting the user agent string and other signals on each request. This includes GPTBot (OpenAI), ClaudeBot (Anthropic), PerplexityBot, Googlebot, Bytespider (TikTok), CCBot (Common Crawl), and many more. See the full agent directory for the complete list.

Why should I care about bot and AI agent traffic to my website?

AI agent traffic has a direct and growing impact on your business. It determines the following:

• Whether your content is included in AI model training datasets
• Whether AI shopping agents can successfully complete purchases on your site
• How accurately AI assistants like ChatGPT and Perplexity summarize and recommend your content to users
• How your pages rank in traditional and AI-powered search engine results
• How much server bandwidth and cost bot traffic consumes
• Whether bad actors and unauthorized scrapers are stealing your content

With AI adoption accelerating, bot traffic is becoming one of the most important segments to understand and optimize for.

How much of my website traffic comes from bots and AI agents?

More than you think. On average, 40% or more of website visits come from AI agents and bots, a volume that is completely invisible to traditional analytics tools like Google Analytics. This has been observed consistently across websites of all sizes and industries.

Can Known Agents help me block scrapers and protect my content?

Yes. Known Agents provides automatic robots.txt management and web scraping protection that lets you block entire categories of AI crawlers and scrapers including AI training crawlers, SEO bots, and unauthorized scrapers with a single setting. Your robots.txt updates automatically as new agents emerge, so you never have to manage it manually. The WordPress plugin can also enforce those rules with active crawler detection, blocking any bot that ignores them.

What is an LLM referral?

An LLM referral is a visit to your website from a human who clicked a link or followed a recommendation from an AI chat or search platform such as ChatGPT, Perplexity, Gemini, or DeepSeek. Unlike bot visits, LLM referrals are real human visitors driven by AI recommendations. Known Agents tracks them separately so you can measure how many visitors and conversions AI platforms are sending you, distinct from your bot traffic data.

How does Known Agents work and how do I install it?

Known Agents works by receiving visit events from your website and identifying which ones come from AI agents and bots. You can connect your website in a few minutes using any of our integrations, including WordPress, Cloudflare, Node.js, Python, PHP, AWS, Fastly, Shopify, and more. Once connected, data starts appearing in your dashboard immediately.

How is Known Agents different from traditional analytics platforms?

Traditional analytics tools like Google Analytics intentionally filter out bot traffic, so they give you no visibility into it at all. Other platforms can detect some bots at the network level but do not identify specific AI agents, track their behavior, or let you manage access by agent category. Known Agents is purpose-built for the bot and AI agent layer, giving you agent-by-agent identification, session replays, traffic trends, LLM referral attribution, and automated robots.txt management in one place.

Does Known Agents affect my human visitors or site performance?

No. Known Agents only processes data about visits that have already occurred and does not add anything to your front-end page load. The WordPress plugin enforces robots.txt rules server-side before a full page render, which reduces load on your server for blocked bots rather than adding to it. Human visitors are unaffected.

Is my data private and secure?

Yes. Known Agents only collects the data necessary to identify and analyze bot and AI agent activity. This includes request metadata such as user agent strings, IP addresses, timestamps, and URLs. We do not collect personal information about your human visitors and do not sell data to third parties.

Frequently Asked Questions for AI Agent Operators

How do I add my agent to the Known Agents directory?

Please send us an email with details about your agent and we'll create a dedicated listing for it. The directory is a comprehensive public index of AI agents, crawlers, and bots.

Why should I list my agent on Known Agents?

A listing on Known Agents increases your agent's credibility and discoverability. It helps website owners understand what your agent does so they can make informed decisions about allowing access rather than blocking it by default. It also signals transparency, a key factor in building trust between your agent and the websites it visits.

Does listing my agent cost anything?

No. Adding your agent to the Known Agents directory is free. Please send us an email and we'll get it listed.