Set Up Agent Analytics

Overview

Track the activity of all known artificial agents crawling your website. Connect your website in just a few seconds using the WordPress plugin or API.

Half of your traffic probably comes from artificial agents, and they're becoming more intelligent every day.

1. Create a New Project

Navigate to the Projects page
Click the New Project button
Enter your website details
Click Create

2. Copy Your Access Token

Click on your project
Click Settings
Copy your access token

3. Start Sending Visit Events

There are 2 ways to send visit events from your website to your project.

Option 1: Using the WordPress Plugin

Use this method for WordPress websites. Adding the plugin is quick and easy.

Log in to your website's WordPress dashboard
Click Plugins in the sidebar
Search for "Dark Visitors" or download the plugin directly
Click Install Now
Click Activate
Click Dark Visitors in the sidebar
Paste your access token
Click Save Changes

Option 2: Using the API

Use this method for server-side rendered websites. Simply make a request to the Visits endpoint every time a page view request is made to your website.

The Request

Endpoint
URL	`https://api.darkvisitors.com/visits`
HTTP Method	`POST`
Headers
`Authorization`	A bearer token with your project's access token (e.g. `Bearer 48d7dcbd-fc44-4b30-916b-2a5955c8ee42`).
`Content-Type`	This needs to be set to `application/json`
Body
`request_path`	The URL path of your visitor's request
`request_method`	The HTTP method of your visitor's request (e.g. `GET`, `POST`, etc.)
`request_headers`	The HTTP headers of your visitor's request, as a key-value object

Tips

Add this call in middleware to cover all of your pages from one place.
Don't wait for the API response (i.e. make it non-blocking) to avoid adding latency to your page response.
Ensure that errors are handled in a way that does not make your page response fail too.

Example

This cURL example sends a visit. If the visit is from an artificial agent, it will show up in agent analytics.

curl -X POST https://api.darkvisitors.com/robots-txts \
-H 'Authorization: Bearer ${ACCESS_TOKEN}' \
-H 'Content-Type: application/json' \
-d '{
    "request_path": "${request.path}",
    "request_method": "${request.method}",
    "request_headers": "${request.headers}"
}'

Here's an example of how to use this in practice for a Node.js backend:

// For each page view, send a visit without awaiting

fetch("https://api.darkvisitors.com/visits", {
    method: "POST",
    headers: {
        "Authorization": "Bearer " + ACCESS_TOKEN,
        "Content-Type": "application/json"
    },
    body: JSON.stringify({
        request_path: request.path,
        request_method: request.method,
        request_headers: request.headers
    })
}).catch((error) => {
    // Silently log the error
})

// Then render and return the page's HTML response

You can follow these examples to call the API in any language.

4. Test Your Integration

Navigate to the Projects page
Click on your project
Click Settings
Click Visit With Dark Visitor
Click realtime

If your website is correctly sending visit events to the project, you should see a new visit from the Dark Visitor agent in the project's realtime timeline.