Install AI crawler tracking

This page walks through each install path click by click. For what crawler tracking measures and how verification works, see AI Crawlers. The same install also reports human visits referred by AI answers, so completing it powers AI Traffic too; browser-only options for that page live in Install AI Traffic tracking.

Pick the guide that matches where your site runs:

Before you start: generate a site key

Every install needs a site key, whichever platform you use:

1. In the app, go to Settings → Connections → Website tracking.
2. Click Generate site key. Give the key a name and, ideally, the domain it will report for; the domain also strengthens signed-crawler verification for that site.
3. Copy both values from the confirmation dialog: the key ID and the signing secret (it starts with ctk_). The secret is shown once; if you lose it, use Rotate on the key to generate a replacement.

Keep the signing secret private: anyone who has it can submit forged traffic reports for your site. The Cloudflare, Vercel, and custom server snippets read it from an environment variable named CITLYZE_SIGNING_SECRET, so it never appears in your code; the WordPress plugin keeps it in its settings. Never commit it to a repository or paste it into client-side code.

Cloudflare Worker

Use this when your DNS is on Cloudflare and the site is proxied (the orange-cloud toggle). It is the most complete option: it sees every request at the edge, reports real HTTP status codes, and works no matter what runs behind Cloudflare.

1. On Settings → Connections → Website tracking → Install snippet, open the Cloudflare tab and copy the snippet.
2. Replace SITE_KEY_ID in the snippet with your key ID. The signing secret does not go in the code; you add it as a Worker secret in step 5.
3. In the Cloudflare dashboard, go to Workers & Pages → Create → Create Worker. Give it a name (for example ai-crawler-tracker) and click Deploy to create the empty worker.
4. Click Edit code, replace the generated example with your snippet, and click Deploy again.
5. Add the signing secret: in the worker, open Settings → Variables and Secrets → Add, choose the type Secret, name it CITLYZE_SIGNING_SECRET, paste your signing secret, and click Deploy. Cloudflare stores it encrypted, and it never appears in the worker's code.
6. Connect the worker to your site: open the worker's Settings → Domains & Routes → Add → Route, pick your zone, and add the route example.com/*. If your site is also served on www.example.com, add a second route for www.example.com/*.
7. Done: no code changes on your site. The worker passes every request through to your origin unchanged and reports matching visits in the background.

Two things to check:

• The route must cover all paths (/*), otherwise crawler visits to unrouted pages are invisible.
• If you already run a worker on those routes, Cloudflare only executes one worker per route; merge the snippet's logic into your existing worker instead of adding a second one.

Vercel / Next.js middleware

Use this for a Next.js app deployed on Vercel. The same file also works on self-hosted Next.js.

1. On Settings → Connections → Website tracking → Install snippet, open the Vercel / Proxy tab and copy the snippet.
2. In your Next.js repository, create middleware.ts in the project root (or in src/ if your app lives there) and paste the snippet.
3. Replace SITE_KEY_ID with your key ID. The signing secret does not go in the file.
4. In your Vercel project, open Settings → Environment Variables and add CITLYZE_SIGNING_SECRET with your signing secret for every environment you want tracked. Mark it Sensitive. Self-hosting Next.js? Set the same variable in your server's environment.
5. If you already have a middleware.ts, don't add a second file; Next.js only runs one. Copy the snippet's helper block into your existing file, call its reporting logic at the start of your middleware function, and make sure your matcher config still covers all paths.
6. Commit and deploy.

Notes:

• Middleware runs before the response exists, so events report their HTTP status as unknown. That is expected. For status codes, use the Vercel log drain instead of this middleware (running both counts every visit twice), or use the Cloudflare or custom server install, which also report redirects.
• On Vercel the visitor IP is taken from a platform-set header and is trustworthy. If you self-host Next.js behind your own proxy, make sure the forwarded-IP header the snippet reads is set by your proxy, not passed through from the client.

WordPress plugin

Use this for any WordPress site where you can install plugins.

The same plugin also checks, for free, which AI crawlers your robots.txt lets in and what blocks the rest; see WordPress plugin.

1. In WordPress admin, go to Plugins → Add New and search for AI Crawler Control by Citlyze, then click Install Now and Activate. The plugin is listed on the official WordPress.org directory, so no zip upload is needed.

2. Prefer a manual install? On Settings → Connections → Website tracking → Install snippet, open the WordPress tab, click Download plugin, and upload the zip under Plugins → Add New → Upload Plugin.

3. Go to Citlyze → Connect Citlyze and fill in all three fields:

   • Tracker base URL: https://app.citlyze.com
   • Key ID: your key ID
   • Signing secret: your ctk_... secret The plugin does not report anything until the base URL is filled in.

4. Click Send test event. You should see a "connected" confirmation; this proves your credentials and connectivity end to end.

5. Behind a proxy or load balancer other than Cloudflare? Under Visitor IP behind a proxy, choose the header your proxy sets and list the proxy's addresses, so crawlers can be verified by their real IP. Behind Cloudflare, nothing is needed.

The plugin reports after each page is served, with the exact status code, including requests another plugin redirects before the page renders. Events wait in your WordPress database and are sent in batches; the Connect Citlyze screen shows how many are waiting and when they were last delivered.

Caveat: pages served from a full-page cache, redirects done by your web server or CDN, and requests a firewall blocks before WordPress loads never reach the plugin. The Connect Citlyze screen tells you when it detects a page cache. If your site is heavily cached, prefer the Cloudflare Worker install or upload your server logs.

Custom server / Node

Use this for custom-coded sites you host yourself: AWS, Google Cloud, Azure, a VPS, or containers. The snippet is an Express-style middleware for Node 20 or newer (it also runs on Bun and Deno).

1. On Settings → Connections → Website tracking → Install snippet, open the Custom server / Node tab and copy the snippet.
2. Save it as aeo-tracker.mjs next to your server entry point.
3. Replace SITE_KEY_ID with your key ID, and set the environment variable CITLYZE_SIGNING_SECRET to your signing secret on the server: in your host's secret settings, or in a .env file that is never committed. The secret never goes in the file itself.
4. Register the middleware before your routes:

   import { aeoCrawlerTracker } from "./aeo-tracker.mjs";
   app.use(aeoCrawlerTracker());

5. If your server sits behind a load balancer or reverse proxy (nginx, ALB), configure Express to trust it (for example app.set("trust proxy", 1)) so the middleware reports the real client IP instead of the proxy's. Crawler identity verification checks that IP.
6. Deploy. The middleware reports after each response finishes, so it adds no latency and includes real HTTP status codes.

Not on Express? The returned handler takes plain (req, res, next) arguments and adapts to Fastify, Koa, or Node's http server. For other languages entirely (Python, Go, PHP, Ruby, ...), implement the signed-beacon protocol; each signed HTTPS POST carries a batch of up to 50 matching requests.

Webflow

Webflow's own hosting cannot run server-side code, and browser-side scripts cannot see AI training crawlers, so tracking installs at a proxy layer in front of Webflow:

1. Webflow supports self-managed reverse proxies on Enterprise plans; arrange it with your Webflow contact and route your domain through your proxy (Cloudflare, CloudFront, Fastly, or your own nginx).
2. Install the tracker at that proxy layer:
   • Proxy on Cloudflare → follow the Cloudflare Worker guide.
   • Your own Node proxy → follow the Custom server / Node guide.
   • Anything else → implement the signed-beacon protocol in your proxy.
3. Verify as described below.

Webflow Cloud apps are not a path to whole-site tracking: they only serve requests under their mount path, so they never see crawler visits to your regular pages.

Shopify

Standard Shopify storefronts cannot run this tracker: Shopify does not let merchants run server-side code on storefront requests and does not support putting a proxy (such as Cloudflare) in front of a store. Browser-side scripts are not a workaround: AI training crawlers never execute JavaScript, and a browser script would expose your signing secret.

What you can do:

• Headless storefronts built on Hydrogen and hosted on Oxygen (or any self-hosted setup) run server-side code; wire the Custom server / Node snippet's logic into your storefront's server request handler.
• For standard stores, your other Citlyze measurements (prompt tracking, citations, GEO audits) work regardless of where the store is hosted.

Verify your install

• On Settings → Connections → Website tracking, the site keys table has a Last event column. It updates once your install delivers its first accepted report; crawler visits and AI-referral clicks both count.
• WordPress users can confirm delivery immediately with Send test event (test events verify credentials and connectivity but do not update Last event).
• Give it time: on low-traffic sites the first real crawler visit can take hours or a few days. Once events arrive, Diagnose → AI crawler activity → Overview fills in.
• If nothing arrives after a couple of days, re-check that the key ID in your snippet (or the plugin settings) belongs to an active, non-revoked key, that CITLYZE_SIGNING_SECRET holds that key's current signing secret, and that your route or middleware covers all paths. Without a valid secret the snippet reports nothing and logs "Citlyze crawler tracking is off" once.
• The site keys table also shows which collector version reports with each key, and tells you when a newer snippet or plugin version is available or when the WordPress plugin detected a page cache.