CLI reference
Every command and flag the scrape CLI accepts.
scrape crawl
Crawl one or more URLs through the escalation pipeline.
filed under · bash.bash
scrape crawl <URL...> [OPTIONS]
--file, -f PATH File of URLs (one per line; '-' for stdin)
--max-tier INT 0=HTTP, 1=Browser, 2=Captcha, 3=Unblock (default: 1)
--schema PATH YAML or JSON schema for LLM extraction
--schema-name TEXT Logical name for the schema (default: 'default')
--no-browser Disable browser tier even if available
--llm Enable LLM extraction (requires ANTHROPIC_API_KEY)
--metrics-port INT Prometheus metrics port (0 = disabled)scrape stats
Show aggregate stats from the local SQLite store.
filed under · bash.bash
scrape statsscrape selftest
Quick smoke test against a couple of public, scraper-friendly URLs. Use this to verify your install.
filed under · bash.bash
scrape selftestscrape-api
Run the FastAPI HTTP server (multi-tenant web service).
filed under · bash.bash
scrape-api
Environment variables:
SCRAPE_API_HOST Bind host (default: 127.0.0.1)
SCRAPE_API_PORT Bind port (default: 8000)
SCRAPE_JWT_SECRET Required in prod; HS256 secret for cookie tokens
SCRAPE_CORS_ORIGINS Comma-separated allowed origins