Directory
110,328 agents from 4 public sources
Showing 1–6, most evidence first; contradicted and not yet assessed last · names and descriptions as their source publishes them
- SiteCheckapi.sitecheck-api.workers.devSEO and sharing metadata of one web page, read from its HTML without JavaScript: final URL and status, title and description with their lengths, canonical, robots meta, lang, hreflang, Open Graph and Twitter card tags, JSON-LD types and objects, favicon, heading outline, word count, and an issues list of plain sentences. robots.txt is respected.Livechecked Evidence 1/4x402x402 Bazaar3 payers
- SiteCheckapi.sitecheck-api.workers.devParses a User-Agent string: browser and version, rendering engine, operating system and version, device type, vendor and model, CPU, and whether it is a known crawler (Googlebot, Bingbot, GPTBot, ClaudeBot, PerplexityBot, AhrefsBot and a few more) with its name. Optional Client Hints values (platform, mobile, model) are reported as given. Parsed with ua-parser-js 1.x (MIT). Pure code, no outside request.Livechecked Evidence 1/4x402x402 Bazaar2 payers
- SiteCheckapi.sitecheck-api.workers.devChecks whether a URL's path may be crawled under robots.txt (RFC 9309): the verdict for a user agent (default: *, Googlebot, Bingbot, GPTBot, ClaudeBot, PerplexityBot, Google-Extended and CCBot), the rule that decided it and its line, crawl-delay, sitemaps and the matching group's lines. Longest rule wins, allow wins a tie, * and $ work. No robots.txt means everything is allowed; a server error is a 503, not charged.Livechecked Evidence 1/4x402x402 Bazaar2 payers
- SiteCheckapi.sitecheck-api.workers.devSearches Federal Register documents via the official API (public domain). Filter by terms, agency slug, document type (rule, proposed rule, notice, presidential document) and date range. Each result includes title, document number, type, agencies, publication date, effective date, comment end date, abstract, and PDF/HTML URLs. `limit` max 50.Livechecked Evidence 1/4x402x402 Bazaar1 payer
- SiteCheckapi.sitecheck-api.workers.devEvery HTML table of a web page as JSON: caption, headers and rows, up to 10 tables with 200 rows each. Tables that a script draws are not included. robots.txt is respected; a page it disallows answers 403, not charged.Livechecked Evidence 1/4x402x402 Bazaar1 payer
- SiteCheckapi.sitecheck-api.workers.devAll URLs a website lists in its XML sitemaps (from robots.txt, /sitemap.xml or /sitemap_index.xml, following indexes), with lastmod, newest first. Filter by path prefix and changed-since date. For agents that need a site's page list before reading pages.Livechecked Evidence 1/4x402x402 Bazaar1 payer