Scrapeless CLI

Overview

The Scrapeless CLI provides a single scrapeless command for web extraction, website crawling, AI answers, proxy URL generation, and cloud browser operations. Run it in a terminal or invoke it from your automation scripts.

Features

The api namespace is reserved and not implemented.

Get Started

• Quickstart — install, configure your key, and make your first request.
• Usage Examples — commands for each feature.
• Configuration and Tasks — credentials, output, polling, and browser requirements.

Quickstart

Requirements

Use Node.js 22 or later and npm. Node.js is the CLI runtime; you do not need to write JavaScript or TypeScript. A Scrapeless account and API key are required for authenticated services.

1. Install the CLI

npm install -g scrapeless
scrapeless --version
scrapeless --help

2. Configure Your API Key

Copy your API key from the Scrapeless dashboard and replace YOUR_API_TOKEN:

1. Open Settings in the dashboard.
2. Select API Key Management.
3. Use an available key, or choose Create API Key if you need one. Copy the key for the configuration command below.

scrapeless config set api_token YOUR_API_TOKEN
scrapeless auth status

auth status checks authentication and account balance. Cloud requests and proxy traffic consume account credits.

3. Extract a Page

scrapeless unlock https://example.com --type markdown

The command prints the page’s extracted Markdown.

Inspect the Full Response

Add --json to inspect the full response, including its code and page content:

scrapeless unlock https://example.com --type markdown --json

Usage Examples

Install the CLI and configure your API key before running these examples.

Web Unlocker

Extract a selected region as HTML, or save a full JSON response:

scrapeless unlock https://example.com --type html --selector body
scrapeless unlock https://example.com --type markdown --json -o page.json

scrape is an alias for unlock. The default output is HTML, with JavaScript rendering enabled.

Crawl

scrapeless crawl https://example.com --limit 3 --max-depth 2 --formats markdown --json -o crawl.json

--formats markdown selects page content format; the overall task result remains JSON. Scope a crawl with --include-paths and --exclude-paths.

AI Scraper

scrapeless ai platforms
scrapeless ai chatgpt "Explain what Scrapeless is used for" -o answer.md
scrapeless ai perplexity "What web scraping techniques are available?" --web-search true --json -o answer.json

Successful tasks return Markdown by default. Options vary by platform; use ai platforms for the current list. English prompts are examples: the CLI accepts Unicode text, and language availability depends on the selected service.

Proxies

scrapeless proxy gateways
scrapeless proxy create --type residential --country US --gateway us

URL generation happens locally without testing connectivity. Supported types are residential, ipv6, and datacenter; gateways accept us, eu, ap, or hostname:port. The URL contains credentials; keep it private.

Cloud Browser

Replace SESSION_ID with the session ID returned by new-session:

scrapeless browser new-session --name demo --ttl 300 --json
scrapeless browser --session-id SESSION_ID open https://example.com
scrapeless browser --session-id SESSION_ID snapshot -i
scrapeless browser --session-id SESSION_ID screenshot page.png
scrapeless browser --session-id SESSION_ID close

For click @e1 or fill @e2 "hello world", use actual interactive references from a fresh snapshot while the session is open. By default, first use, including browser --help, installs the upstream CLI; see Browser Requirements in Configuration and Tasks.

For command options, run scrapeless unlock --help, scrapeless crawl --help, scrapeless ai --help, or scrapeless proxy create --help. The first-use installation behavior of browser --help is described above.

Full command reference and source-build instructions are in the CLI repository. AI service behavior and billing are covered in the AI Scraper documentation.

Configuration and Tasks

Credentials

The default file is ~/.scrapeless/config.json. Key precedence is configuration file → SCRAPELESS_API_TOKEN → SCRAPELESS_API_KEY. To use an environment variable:

export SCRAPELESS_API_KEY="YOUR_API_TOKEN"

A saved key takes precedence. The CLI does not load .env automatically. This is POSIX syntax; PowerShell uses $env:SCRAPELESS_API_KEY="YOUR_API_TOKEN".

A custom config path does not change Browser’s default file. The wrapper passes its resolved key via an environment variable, but Browser’s default saved key can override it. Use the shared default file to keep authentication consistent.

Output and Request Bodies

HTTP commands support --json for the full response and -o, --output <file> to save output. By default, results go to stdout; with -o, they are saved to the file instead. Progress, task-ID notices during polling, and errors go to stderr. With --no-wait, the creation response, including its task ID, is part of the result output.

Unlocker, Crawl, and AI creation accept --data '<JSON>' or --data @request.json. Precedence is explicit flags → body fields → CLI defaults; nested objects merge and arrays replace.

Task Handling

Crawl and AI creation poll every 5 seconds for up to 300 seconds by default. Add --no-wait to return after creation. Query an existing task with its returned ID:

scrapeless crawl result TASK_ID --wait --timeout 300 --interval 5
scrapeless ai result TASK_ID --platform chatgpt --wait --timeout 300 --interval 5

Use the ID for the corresponding service. A polling timeout does not cancel the server task. Check completed for Crawl or success for AI and inspect actual content; exit code 0 alone is insufficient.

--timeout and --interval use seconds; scraping options --wait-for and --scrape-timeout use milliseconds. Each HTTP request has a separate 60-second timeout. Task-creation POSTs are not retried automatically.

Browser Requirements

The wrapper needs npm, access to npm and GitHub Releases, and cache write permissions. Sessions run in the cloud; the wrapper runs locally without a separate Chromium install.

The pinned release supports macOS x64/arm64, Linux x64/arm64, and Windows x64; no Windows arm64 binary is bundled.

Troubleshooting