Bot Access Results

GET /api/v1/bot-access-results returns the results of Bot Access Monitoring checks: for each monitored page and each AI bot, whether the bot is allowed by robots.txt and what actually happened when the page was requested. Open problems surface separately as bot access incidents.

Filters

Standard parameters (limit, offset, format=csv) work as described in Pagination, filtering, and errors.

Key fields

• target_id: the monitored page the check ran against
• crawler_id: the bot that was tested, matching the ids used by crawler events
• check_date: UTC day of the check (YYYY-MM-DD)
• checked_at: exact time of the check
• robots_verdict: what robots.txt says for this bot (allowed, disallowed, or missing when no robots.txt was found)
• http_status: the HTTP status the request received
• classification: the outcome. ok means reachable; robots_block means robots.txt disallows the bot; infra_block means the request was rejected before the page answered, typically a firewall or CDN bot rule; timeout and error cover unreachable checks
• block_vendor: for infra_block, the detected blocking layer when it is identifiable (for example cloudflare), otherwise generic or empty
• latency_ms: response time of the check
• test_basis: how the check was performed. ua_spoof means the request presented the bot's user agent from Citlyze servers, so allow-lists based on verified bot IPs can appear blocked here even when the real bot gets through

Rows are ordered by check_date descending with target_id and crawler_id as tiebreaks, so offset pagination is stable.

Example

curl -H "Authorization: Bearer $AEO_API_KEY" \
  "https://app.citlyze.com/api/v1/bot-access-results?classification=infra_block&since=2026-08-01"

Use this resource to audit which AI bots can reach your key pages, catch robots.txt and firewall regressions, and reconcile access with the crawler visits you actually receive.