AI Visibility Setup
Send request metadata to Hardal to analyze AI agent, AI search bot, and crawler traffic.
What AI Visibility does
AI Visibility shows which AI agents, search bots, and crawlers reach your pages. Send metadata from the original incoming request, and Hardal classifies bot traffic such as GPTBot, ClaudeBot, and PerplexityBot.
Hardal drops requests that it does not classify as bot traffic before storing them for AI Visibility.
How AI Visibility collection works
For each incoming server request you want analyzed, send a corresponding request to the agent-events endpoint. Run this logic in middleware, an edge function, or a CDN worker where you can read the original request.
Collect on the server
Create AI Visibility events on the server side. Run the collection at the layer that receives the original website request, before the page renders in the browser:
- Your CDN layer
- Your application server
Do not rely on a browser-side script for AI Visibility. Many bots and crawlers do not run client-side scripts.
On high-volume websites, send AI Visibility events asynchronously so Hardal does not delay the original request. Build the event from the incoming request, then send it in the background.
POST https://analytics.yourdomain.com/agent-events
Use the same first-party collection URL shown in Setup in the Dashboard.
Required headers:
| Header | Value |
|---|---|
Content-Type | application/json |
tenant-id | Your Tenant ID from Setup in the Dashboard |
AI Visibility payload
| Field | Type | Required | Description |
|---|---|---|---|
path | string | Yes | Requested path on the site. |
ip | string | Yes | IP address from the original incoming request. |
userAgent | string | Yes | Raw User-Agent header from the request. |
eventTime | string | Yes | Request time in ISO 8601 format. |
sourceId | string | Yes | Your Source ID from Setup in the Dashboard. |
Example AI Visibility request
{
"path": "/pricing",
"ip": "66.249.66.1",
"userAgent": "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)",
"eventTime": "2026-04-29T15:12:17.836Z",
"sourceId": "your-source-id"
}
Next.js middleware example
Read the IP and user agent from the original incoming request. Do not rebuild them in the browser.
import { NextResponse } from "next/server";
import type { NextRequest } from "next/server";
const VIRGO_URL = "https://analytics.yourdomain.com";
const VIRGO_TENANT_ID = "your-tenant-id";
const VIRGO_SOURCE_ID = "your-source-id";
export function middleware(request: NextRequest) {
const forwardedFor = request.headers.get("x-forwarded-for");
const realIp = request.headers.get("x-real-ip");
const ip = (forwardedFor?.split(",")[0]?.trim() || realIp || "").trim();
void fetch(VIRGO_URL + "/agent-events", {
method: "POST",
headers: {
"Content-Type": "application/json",
"tenant-id": VIRGO_TENANT_ID
},
body: JSON.stringify({
path: request.nextUrl.pathname + request.nextUrl.search,
ip,
userAgent: request.headers.get("user-agent") || "",
eventTime: new Date().toISOString(),
sourceId: VIRGO_SOURCE_ID
})
}).catch(() => {});
return NextResponse.next();
}
export const config = {
matcher: [
"/((?!_next/static|_next/image|favicon.ico|.*\\.(?:svg|png|jpg|jpeg|gif|webp|ico|woff|woff2|ttf|eot)$).*)",
]
};
If your CDN sets a different trusted client-IP header, read that header instead. Do not accept an arbitrary forwarded IP header unless your proxy overwrites it.
Verify AI Visibility data
Send a test request with a known crawler user agent. A recognized and allowlisted crawler returns 202 Accepted. An ignored request returns 204 No Content. Then open the AI Visibility report and confirm that the agent and requested path appear.
Send Agent Events API
Full reference for the agent-events endpoint.