Set Up Automatic Robots.txt for Your WordPress Website

Overview

Use the WordPress plugin to connect your website to Automatic Robots.txt in just a few seconds. Please contact us if you need help getting set up.

Step 1: Install the Plugin

• Navigate to the Dark Visitors Projects page and open your project
• Copy your access token from the Settings page
• Log in to your website's WordPress dashboard
• Click Plugins in the sidebar
• Search for Dark Visitors
• Click Install Now
• Click Activate
• Click Dark Visitors in the sidebar
• Paste in your access token
• Select the agent types you want to block. Allowed categories include:
  • AI Agent
  • AI Assistant
  • AI Data Scraper
  • AI Search Crawler
  • Archiver
  • Developer Helper
  • Fetcher
  • Automated Agent
  • Intelligence Gatherer
  • Scraper
  • SEO Crawler
  • Search Engine Crawler
  • Security Scanner
  • Undocumented AI Agent
  • Uncategorized
• Click Save Changes

Step 2: Test Your Integration

If your website is correctly connected, you should see the new rules in your website's robots.txt.