Node.js Integration

Track bot and AI crawler visits from any Node.js application. Works with Express, Fastify, Next.js, Koa, and any Node.js HTTP server.

Time to set up: ~2 minutes

Prerequisites

A Node.js application (v18+ for native fetch)
A BotSights account with a project created for your domain
Your API Key (found in BotSights → Account → Projects → API Key)

Express / Connect

Add the middleware to your Express app:

const BOTSIGHTS_API_KEY = 'YOUR_API_KEY_HERE';
const BOTSIGHTS_API_URL = 'https://www.botsights.com/api/log';
 
const SKIP_EXT = new Set([
  'css','js','png','jpg','jpeg','gif','svg','webp','avif',
  'ico','woff','woff2','ttf','eot','map','mp4','webm','mp3',
]);
 
function botsights(req, res, next) {
  const startTime = Date.now();
  const ext = req.path.split('.').pop();
  if (ext && SKIP_EXT.has(ext.toLowerCase())) return next();
 
  const originalEnd = res.end;
  res.end = function (...args) {
    originalEnd.apply(res, args);
    fetch(BOTSIGHTS_API_URL, {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({
        api_key: BOTSIGHTS_API_KEY,
        ua: req.headers['user-agent'] || '',
        url: req.originalUrl || req.url,
        ip: req.headers['cf-connecting-ip']
          || req.headers['x-forwarded-for']?.split(',')[0]?.trim()
          || req.socket?.remoteAddress || '',
        referer: req.headers['referer'] || '',
        status_code: res.statusCode,
        duration_ms: Date.now() - startTime,
        content_type: res.getHeader?.('content-type') || '',
        http_method: req.method,
        source: 'nodejs',
      }),
    }).catch(() => {});
  };
  next();
}
 
// Use it:
app.use(botsights);

Next.js (Middleware)

For Next.js apps not hosted on Cloudflare, add to middleware.ts:

import { NextResponse } from 'next/server';
import type { NextRequest } from 'next/server';
 
const BOTSIGHTS_API_KEY = 'YOUR_API_KEY_HERE';
const BOTSIGHTS_API_URL = 'https://www.botsights.com/api/log';
 
export function middleware(request: NextRequest) {
  const response = NextResponse.next();
  const startTime = Date.now();
 
  // Skip static assets
  const ext = request.nextUrl.pathname.split('.').pop() || '';
  const skip = ['css','js','png','jpg','svg','ico','woff','woff2'];
  if (skip.includes(ext)) return response;
 
  // Send analytics after response (non-blocking via waitUntil if available)
  const data = {
    api_key: BOTSIGHTS_API_KEY,
    ua: request.headers.get('user-agent') || '',
    url: request.nextUrl.pathname + request.nextUrl.search,
    ip: request.headers.get('x-forwarded-for')?.split(',')[0] || '',
    referer: request.headers.get('referer') || '',
    http_method: request.method,
    source: 'nodejs_nextjs',
  };
 
  fetch(BOTSIGHTS_API_URL, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify(data),
  }).catch(() => {});
 
  return response;
}
 
export const config = {
  matcher: ['/((?!_next/static|_next/image|favicon.ico).*)'],
};

How It Works

The middleware intercepts every incoming request, measures the response time, and sends analytics to BotSights after the response is sent. Static assets are automatically skipped.

Non-blocking — analytics are sent after res.end(), no impact on response time
Server-side — captures all visitors including bots that don't execute JavaScript
No cookies — nothing stored in the visitor's browser
No client-side code — zero JavaScript added to your pages

Troubleshooting

No data appearing?

Check your API key is correct
Verify your server can make outbound HTTPS requests
Check that the domain in your BotSights project matches your site's domain