Cloudflare Worker Setup

Track bot and AI crawler visits on any website using Cloudflare. Works with all Cloudflare plans (Free, Pro, Business, Enterprise).

Time to set up: ~5 minutes

Prerequisites

  • A Cloudflare account with your domain added and proxied (orange cloud enabled)
  • A BotSights account with a project created for your domain
  • Your API Key (found in BotSights → Account → Projects → API Key)

Step 1: Create the Worker

  1. Log in to your Cloudflare Dashboard
  2. Select your domain (e.g. yourdomain.com)
  3. In the left sidebar, go to Workers & Pages (under Compute / Workers)
  4. Click Create application
  5. Select Start with Hello World!
  6. Name the worker (e.g. botsights) and click Deploy

Step 2: Edit the Worker Code

  1. After deploying, click Edit Code in the top right
  2. Select all existing code in the worker.js file and delete it
  3. Paste the following script:
const API_KEY = 'YOUR_API_KEY_HERE';
const API_URL = 'https://www.botsights.com/api/log';
 
export default {
  async fetch(request, env, ctx) {
    const startTime = Date.now();
    const response = await fetch(request);
    ctx.waitUntil(sendAnalytics(request, response, startTime).catch(() => {}));
    return response;
  },
};
 
async function sendAnalytics(request, response, startTime) {
  const url = new URL(request.url);
  const ext = url.pathname.split('.').pop();
  const skip = [
    'css','js','png','jpg','jpeg','gif','svg','webp','avif',
    'ico','woff','woff2','ttf','eot','map','mp4','webm','mp3',
  ];
  if (ext && skip.includes(ext.toLowerCase())) return;
 
  await fetch(API_URL, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      api_key: API_KEY,
      ua: request.headers.get('user-agent') || '',
      url: url.pathname + url.search,
      ip: request.headers.get('cf-connecting-ip') || '',
      referer: request.headers.get('referer') || '',
      status_code: response.status,
      duration_ms: Date.now() - startTime,
      content_type: response.headers.get('content-type') || '',
      http_method: request.method,
      source: 'cloudflare_worker',
    }),
  });
}
  1. Replace YOUR_API_KEY_HERE with your actual API key from BotSights
  2. Click Deploy

Step 3: Add a Route to Your Domain

  1. Navigate back to your Cloudflare dashboard and select your domain
  2. In the left sidebar, click Workers Routes
  3. Click Add route in the HTTP Routes section
  4. Under Route, enter your domain followed by /*:
yourdomain.com/*
  1. Select the botsights worker you just created
  2. Under Failure mode, select Fail open (proceed) — this ensures your website keeps working even if the Worker has an error
  3. Click Save

Important: Use yourdomain.com/* (without a *. prefix). The *. prefix only matches subdomains, not the root domain. If you also need to track subdomains, add a second route: *.yourdomain.com/*

Step 4: Verify It Works

  1. Visit your website in a browser
  2. Open your BotSights dashboard → Realtime page
  3. Your visit will appear as a "Human" visit

To test bot detection, run this in your terminal:

curl -H "User-Agent: ChatGPT-User" https://yourdomain.com/

This simulates a ChatGPT bot visit.

Troubleshooting:

  • No visits appearing? Check that your domain is proxied in Cloudflare (orange cloud in DNS)
  • Still nothing? Verify the API key is correct and the route uses yourdomain.com/*
  • Check Worker logs: Workers & Pages → your Worker → Metrics
  • Set failure mode to "Fail open" so your site isn't affected by Worker errors

How It Works

The Worker sits between Cloudflare and your origin server. Every request passes through it transparently. After serving the response, analytics data is sent asynchronously — your visitors experience zero added latency.

Static assets (images, CSS, JS, fonts) are automatically skipped.

FAQ

Will this slow down my website? No. Analytics are sent after the response is delivered using waitUntil(). Zero latency added.

Does this work with Cloudflare's free plan? Yes. The free plan includes 100,000 Worker requests per day. Set failure mode to "Fail open" so your site works if the limit is reached.

What about cached pages? The Worker runs on every request, including cached pages.

Can I use this alongside the WordPress plugin? We recommend one or the other to avoid double-counting. The Worker is preferred for Cloudflare sites.