← All headings
Web scraping & browsers
2,495 listings from 1,060 providers, 1,975 answering · counted
Filed by addrbook from the words in each listing, so a heading says what a listing claims to be for.
Showing 526–550, one row per provider, those answering first · names and descriptions as their source publishes them
- api.strale.io/x402/tech-stack-detectapi.strale.ioTech Stack Detection: Detect the technology a website runs on (CMS, frameworks, analytics, CDN, payments and marketing tools) from its headers and HTML.Livechecked Evidence 1/4x402x402 Bazaar1 payer2 more from strale.io
- twin.unykorn.org/web/robotstwin.unykorn.orgrobots.txt parser: rules, sitemaps, AI-crawler blocks, and is-this-path-allowed check — Genesis402 / UnyKorn Operator NetworkLivechecked Evidence 1/4x402x402 Bazaar1 payer1 more from unykorn.org
- zlurp.ai/scrapezlurp.aiScrape any public URL to clean markdownLivechecked Evidence 1/4x402x402 Bazaar1 payer
- Annalistapi.annalist.devChecks for an x402 seller wallet on Base: the services that pay out to it and their grades, and USDC received over 30 days with payers, repeat payers and fake-volume signals (payers the seller funded, crawler bots, one-wallet dominance). Signed.Livechecked Evidence 1/4x402x402 Bazaar1 payer
- Internal Links Auditgogeta2-u35175.vm.elestio.appInternal linking audit: crawls up to max_pages same-site pages (robots.txt respected, 30 s cap): broken internal links, orphan pages vs XML sitemap, click depth, dead ends, generic/empty anchors, internal nofollow, duplicate titles, missing meta descriptions.Livechecked Evidence 1/4x402x402 Bazaar1 payer1 more from elestio.app
- PennyAPI Webpage Metadatapennyregwatch.comExtract webpage metadata for agent research and URL triage: title, description, canonical URL, Open Graph, and Twitter metadata from a public page.Livechecked Evidence 1/4x402x402 Bazaar1 payer1 more from pennyregwatch.com
- Toolsmithtoolcall.clickLive job postings without scraping: company= returns an employer's current openings straight from their official ATS board (Greenhouse, Lever, Ashby, SmartRecruiters) with title, location, department, type and apply URL; query= searches remote-heavy job APIs (Remotive, Arbeitnow) by keyword. Optional location= filter. JSON response. $0.02 USDC per call, pay via x402, no API key.Livechecked Evidence 1/4x402x402 Bazaar1 payer
- Delx Extract URLsapi.delx.aiExtract HTTP/HTTPS URLs from text for $0.001 USDC. Call when you need this bounded result without an API key or subscription. Returns machine-readable JSON via x402 on Base or Solana. Limitation: results cover only the supplied input or one bounded public endpoint at request time; callers must verify before acting.Livechecked Evidence 1/4x402x402 Bazaar1 payer
- ah-ai-crawlersx402.agentharbor.agencyGiven a website domain, reports which AI crawlers may fetch its home page under its robots.txt (RFC 9309): GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, PerplexityBot, Google-Extended, Applebot-Extended, CCBot and more, each allowed or blocked, plus Googlebot, listed sitemaps, whether /llms.txt is published and page-level noindex/nosnippet, with fixes. Static check, no browser. Not charged if robots.txt cannot be read (no response, or a bot block).Livechecked Evidence 1/4x402x402 Bazaar1 payer
- scribe.vimabrosta.com/v1/convertscribe.vimabrosta.comConvert a document: HTML to Markdown or text, Markdown to HTML or text, a public URL to clean Markdown, text, or HTML (scripts, styles, and boilerplate removed), or a PDF (base64 or URL, 5MB, 200 pages) to text. Returns the converted document with its title or page count. $0.01.Livechecked Evidence 1/4x402x402 Bazaar1 payer
- Otto AIx402.ottoai.servicesQuick web answer with citations (Exa upstream): ask one factual or current question and get a concise synthesized answer plus source URLs and publication dates. Optimized for fast lookups and agent loops; use /llm-research for a deeper multi-section research report, reasoning, image analysis, or webpage summarization. Keyless, pay-per-call, cached by query.Livechecked Evidence 1/4x402x402 Bazaar1 payer1 more from ottoai.services
- Automaton Media APIsapi.language123.netURL to Markdown for LLM agents and RAG: send a web page or PDF link, get its main content as clean Markdown (title, author, language, word count) without navigation or ads. PDFs: text of up to 200 pages. Flat price per URL.Livechecked Evidence 1/4x402x402 Bazaar1 payer
- Forest Webhttp--forest-gas-station-mainnet--lcjl27p8lmjs.code.runExtract a bounded normalized set of public HTTP(S) anchor links from one static HTML page. Forest resolves relative and base URLs, deduplicates by final absolute URL and classifies returned links as same-origin or cross-origin. One page fetch only; no linked URL is requested or crawled. Base mainnet USDCLivechecked Evidence 1/4x402x402 Bazaar1 payer
- Bacon Servicesbaconservices.comUse when an agent needs a company's brand from its website: returns its name, logo URLs, colours, fonts and social profiles as JSON.Livechecked Evidence 1/4x402x402 Bazaar1 payer
- agdataagdata.shveik.devWebsite contact finder API for AI agents: send up to 5 websites and get, for each, the emails, phone numbers and social profile links printed on its home, contact and about pages, as JSON with the contact page that was found. Pay per website in stablecoins; a site where no contact was found is not charged and you are refunded for it.Livechecked Evidence 1/4x402x402 Bazaar1 payer
- Forest Webhttp--forest-gas-station-mainnet--lcjl27p8lmjs.code.runInterpret the robots.txt crawling rules that apply to a specific public URL and user-agent before an automated fetch or crawl. Returns allow/disallow decision evidence, matched rules, crawl delay and sitemap hints from one bounded robots.txt read. This is protocol guidance, not authorization or an anti-bot bypass. Base mainnet USDCLivechecked Evidence 1/4x402x402 Bazaar1 payer
- api.060504.shop/v1/fetch/dynamicapi.060504.shopWeb scraping proxy API - send a URL, get clean markdown or HTML content back. Supports dynamic rendering.Livechecked Evidence 1/4x402x402 Bazaar1 payer
- x402.pokt-agent.com/kr-hwp/convertx402.pokt-agent.comConvert a Korean Hancom document (HWP 5.x or HWPX) at a URL to Markdown with tables preserved, including merged cells as HTML, text boxes, footnotes and distributable (배포용) documents. Typical sources: Korean government press releases, notices and law annexes. Encrypted or DRM files are reported, not converted.Livechecked Evidence 1/4x402x402 Bazaar1 payer
- Hermes Commerceagent.kihustle.techFast deterministic check that a public URL is alive: HTTP status, redirects, content-type, size, and title. Built for agent workflows before paying for deeper crawls.Livechecked Evidence 1/4x402x402 Bazaar1 payer
- Forest Infrahttp--forest-gas-station-mainnet--lcjl27p8lmjs.code.runCheck the current operational HTTP health of one public URL or API endpoint with a bounded GET or HEAD probe. Returns reachability, status code, redirects, content type and size basics; it does not scrape page content, run JavaScript or test private networks. Base mainnet USDCLivechecked Evidence 1/4x402x402 Bazaar1 payer
- 1gateway1gateway.devUse when an agent must vet or research a website in one call (lead research, security or SEO check, is this site legit). Input: url or bare domain. Output: JSON page metadata, HTTP status, HTTPS and HTTP-to-HTTPS redirect, server and hosting hints, 7 security headers, robots.txt (sitemaps, AI-crawler rules), llms.txt presence, plus domain DNS, mail provider, SPF/DMARC and RDAP registrar, age and expiry. Example: ?url=stripe.comLivechecked Evidence 1/4x402x402 Bazaar1 payer1 more from 1gateway.dev
- Forest Agent Toolshttp--forest-gas-station-mainnet--lcjl27p8lmjs.code.runPreflight one public Apify Actor before a paid automation run using only public Store and default-build OpenAPI metadata. Forest checks agentic-payment eligibility, pricing/budget signals and top-level input compatibility without starting an Actor, buying credit, authenticating or making an external payment. Base mainnet USDCLivechecked Evidence 1/4x402x402 Bazaar1 payer
- AGMMagmm.co.ukDomain in, a grade from A to F out: how readable a website is to AI crawlers and answer engines (robots.txt, llms.txt, sitemap, JSON-LD, MCP and x402 readiness). Public files only, no LLM. $0.05 USDC per request.Livechecked Evidence 1/4x402x402 Bazaar1 payer
- hcrb.in/v1/tools/clean-markdown-scraper/callhcrb.inFetch a public web page by URL (or take your HTML) and return Markdown plus headings. No JavaScript rendering. Result carries a signed receipt.Livechecked Evidence 1/4x402x402 Bazaar1 payer
- cryptofeed402.site/audit/agent-readycryptofeed402.siteAgent-readiness audit for a public website or x402 API: checks llms.txt, robots.txt AI-crawler rules, sitemap, OpenAPI, /.well-known/x402, agent card, an MCP initialize probe, a live unpaid 402 challenge (accepts, network, payTo, listed vs 402 price, Bazaar metadata) and listings on Roundhouse, 402 Index and x402 Doctor. 0-100 score, pass/warn/fail per check with evidence and fixes. Deterministic, no LLM. Failed calls never charged. Free example at /audit/sample. $0.05 USDC per call on Base.Livechecked Evidence 1/4x402x402 Bazaar1 payer