AI Visibility Setup
Send request metadata to Hardal to analyze AI agent, AI search bot, and crawler traffic.
What AI Visibility does
AI Visibility shows which AI agents, search bots, and crawlers reach your pages. Send metadata from the original incoming request, and Hardal classifies bot traffic such as GPTBot, ClaudeBot, and PerplexityBot.
Hardal drops requests that it does not classify as bot traffic before storing them for AI Visibility.
How AI Visibility collection works
For each incoming server request you want analyzed, send a corresponding request to the agent-events endpoint. Run this logic in middleware, an edge function, or a CDN worker where you can read the original request.
Collect on the server
Create AI Visibility events on the server side. Run the collection at the layer that receives the original website request, before the page renders in the browser:
- Your CDN layer
- Your application server
Do not rely on a browser-side script for AI Visibility. Many bots and crawlers do not run client-side scripts.
On high-volume websites, send AI Visibility events asynchronously so Hardal does not delay the original request. Build the event from the incoming request, then send it in the background.
POST https://collect.yourdomain.com/agent-events
Use the same first-party collection URL shown in Setup in the Dashboard.
Required headers:
| Header | Value |
|---|---|
Content-Type | application/json |
tenant-id | Your Tenant ID from Setup in the Dashboard |
AI Visibility payload
| Field | Type | Required | Description |
|---|---|---|---|
path | string | Yes | Requested path on the site. |
ip | string | Yes | IP address from the original incoming request. |
userAgent | string | Yes | Raw User-Agent header from the request. |
eventTime | string | Yes | Request time in ISO 8601 format. |
sourceId | string | Yes | Your Source ID from Setup in the Dashboard. |
Example AI Visibility request
{
"path": "/pricing",
"ip": "66.249.66.1",
"userAgent": "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)",
"eventTime": "2026-04-29T15:12:17.836Z",
"sourceId": "your-source-id"
}
Choose your platform
Open Setup, choose your source, and expand AI Visibility. Choose a platform and copy its generated example:
| Platform | Where to add it |
|---|---|
| HTTP API | Send a POST from the server for each page request, using the original request details. |
| Next.js | Add the generated code to proxy.ts, or merge it into your existing proxy. It uses event.waitUntil to keep forwarding alive after the response. |
| Node.js | Register the generated hardalAgentEvents Express middleware with app.use(hardalAgentEvents) before your routes. Configure trusted proxies so Express reads the real client IP. |
| Cloudflare | Add the generated Worker to your website route. It reads CF-Connecting-IP and uses ctx.waitUntil for background forwarding. |
Use the generated collection URL and IDs for your source. The Node.js and Cloudflare examples skip static assets and requests other than GET or HEAD, and log forwarding failures.
If your CDN sets a different trusted client-IP header, read that header instead. Do not accept an arbitrary forwarded IP header unless your proxy overwrites it.
Verify AI Visibility data
Use Check incoming data on Onboarding to check agent verification. A genuine recognized crawler from an allowlisted IP returns 202 Accepted; ignored requests return 204 No Content. Spoofing a crawler user agent alone is not enough. Then open the AI Visibility report and confirm that the agent and requested path appear.
Send Agent Events API
Full reference for the agent-events endpoint.