Scrapeless Skills

Scrapeless Skills lets your AI coding assistant use Scrapeless through natural-language instructions. Extract a webpage, collect content across a website, query AI platforms, interact with a cloud browser, generate proxy URLs, or retrieve structured Google Search, Amazon, and TikTok data.

Each skill includes task instructions and the references or scripts needed to carry out that task. Your assistant must support Agent Skills and be able to execute local commands.

View the Skills repository

Available skills

SkillUse it to
web-unlockerExtract a single webpage as HTML, Markdown, text, or other supported output, including JavaScript-rendered pages.
crawlCollect multiple pages with page limits, depth controls, path filters, and output formats.
ai-scraperCollect answers and available citations from ChatGPT, Gemini, Copilot, Perplexity, Grok, Alexa, Google AI Mode, and Google AI Overview.
agent-browserNavigate pages, click elements, fill forms, inspect page snapshots, and capture screenshots in a cloud browser session.
proxyGenerate authenticated residential, IPv6, or datacenter proxy URLs with location and session settings.
scraping-apiRetrieve structured Google Search, Amazon, and TikTok data.

Use Web Unlocker for one-page extraction, Crawl for multi-page collection, and Agent Browser for workflows that need ongoing interaction. Use AI Scraper for AI-generated answers and Scraping API for standard Google Search results or supported Amazon and TikTok datasets.

Quickstart

1. Prepare your environment

You need:

  • Node.js 22.20 or later and npm to use the installation workflow below.

  • An AI coding assistant that supports Agent Skills and local command execution.

  • A Scrapeless account and API key from the dashboard.

  • Network access to the services used by your selected skills. Agent Browser also downloads dependencies from npm and GitHub Releases on first use.

Check Node.js and install the Scrapeless CLI:

node --version
npm install -g scrapeless
scrapeless --version

Five skills use the CLI. The scraping-api skill uses its own bundled Node.js script and does not require a global CLI installation.

2. Install the skills

From your project directory, run:

npx skills add scrapeless-ai/scrapeless-skills

Follow the prompts to select the skills and target assistant.

To install only one skill:

npx skills add scrapeless-ai/scrapeless-skills --skill web-unlocker

To install globally for use across projects:

npx skills add scrapeless-ai/scrapeless-skills -g

See the Skills installer documentation for supported assistants and installation options.

3. Configure your API key

Copy your API key from the Scrapeless dashboard. Replace YOUR_API_TOKEN with your key and run these commands locally:

scrapeless config set api_token YOUR_API_TOKEN
scrapeless auth status

The default configuration file is ~/.scrapeless/config.json. The CLI, bundled Scraping API script, and cloud browser can use this shared configuration.

For environment-based configuration, use the following in Bash or Zsh:

export SCRAPELESS_API_KEY='YOUR_API_TOKEN'

Launch your assistant from that shell so it inherits the variable. An assistant launched separately, including from a desktop application, needs its own environment configuration or access to the saved configuration file.

If you installed only scraping-api, you can use the environment variable without installing the CLI. The config and auth commands above require the CLI.

Keep your key out of chat prompts, shared files, and logs. See Get and protect your API key for credential handling guidance.

4. Run your first task

Ask your assistant:

Use the web-unlocker skill to extract https://example.com as Markdown
and save the content to page.md.

If your assistant supports explicit $skill-name invocation, you can also write:

Use $web-unlocker to extract https://example.com as Markdown
and save the content to page.md.

Review the assistant’s reported result and generated file. A successful installation alone does not confirm that a data request has completed.

Example tasks

Specify the target URL or platform, the scope, and the output you want. Replace example URLs and search terms with your own inputs.

Extract a webpage

Use the web-unlocker skill to extract https://example.com as Markdown.
Save the extracted content to page.md.

Web Unlocker supports single-page extraction without maintaining an interactive browser session. JavaScript rendering is enabled by default.

Crawl a website

Use the crawl skill to collect up to 10 pages from https://example.com.
Keep external links disabled, extract only the main content as Markdown,
and save the crawl results to crawl.json.

The page content can be Markdown while the overall crawl result remains JSON. Include a page limit and path filters when you need a narrower collection.

Collect AI answers

Use the ai-scraper skill to ask ChatGPT:
"What are the main use cases for web scraping?"
Save the full response as JSON and preserve any returned citations.

Available fields and options depend on the platform. Full JSON output preserves structured fields; the default CLI output extracts Markdown. Citations are included only when returned by the service.

Automate a browser

Use the agent-browser skill to create a cloud browser session,
open https://example.com, inspect the page, and save a screenshot
to page.png. Close the session when finished.

For interactive tasks, the assistant uses element references from the current page snapshot and keeps the same session ID throughout the workflow.

Generate a proxy URL

Use the proxy skill to generate a US residential proxy URL
through the Asia-Pacific gateway. Save it to a private local file
named proxy.txt and mask the password in your summary.

Proxy URL generation happens locally. It does not test connectivity, validate the API key, or confirm available balance. The generated URL contains credentials.

Retrieve structured data

Use the scraping-api skill to search Google for "coffee" with
hl=en and gl=us. Save the full JSON response to google-results.json.

The same skill supports Amazon and TikTok datasets. For example:

Use the scraping-api skill to search Amazon.com for "coffee",
using the keywords request type and page 1. Save the JSON response.

The assistant uses the skill’s bundled scripts/scrape.mjs file and the relevant input reference. The script path must resolve inside the installed skill directory.

Configuration

Credential precedence

The CLI and bundled Scraping API script read credentials in this order:

  1. apiKey in the configuration file, with support for the legacy api_token field.

  2. SCRAPELESS_API_TOKEN.

  3. SCRAPELESS_API_KEY.

A saved key takes precedence over environment variables. When switching accounts, update or remove the saved key before using a different environment key:

scrapeless config remove api_token

Use scrapeless config path to locate the configuration file and scrapeless config list to inspect masked settings.

SCRAPELESS_CONFIG_PATH overrides the configuration path for the CLI and bundled script. It does not change the cloud browser’s own default configuration path. Use the shared default file to keep credentials consistent across all six skills.

Using a .env file

The CLI and bundled script do not load .env automatically. If you maintain a .env file containing SCRAPELESS_API_KEY, load it before starting your assistant. In Bash or Zsh:

set -a
. ./.env
set +a

Keep .env outside version control.

Long-running tasks

AI Scraper, Crawl, and Scraping API can return a task ID before results are ready. Keep that ID and retrieve the existing task instead of submitting the same request again.

For CLI-backed tasks, replace TASK_ID with the returned ID:

scrapeless ai result TASK_ID --platform chatgpt --wait --json
scrapeless crawl result TASK_ID --wait --json

Use the platform that created the AI task. Confirm success for AI Scraper or completed for Crawl. A polling timeout can return an unfinished response with exit code 0, so inspect the status and content.

To resume a Scraping API task, run the following from the installed scraping-api skill directory, or replace the script path with its absolute path:

node scripts/scrape.mjs --task-id TASK_ID --max-polls 20 --timeout 180

The script exits with code 1 on a polling timeout and includes the task ID when available. A client timeout does not cancel the server task.

Troubleshooting

IssueWhat to check
The assistant does not find a skillConfirm that you installed it for the correct assistant and scope. Use npx skills list to inspect installed skills and consult the installer documentation for scope options.
scrapeless is not foundInstall the CLI and make sure the assistant can access the npm executable directory through its PATH.
Missing key or HTTP 401/403Check the saved configuration and environment variables. If the CLI is installed, run scrapeless auth status.
A .env file has no effectLoad it explicitly and start the assistant from the configured environment.
Browser setup fails on first useCheck access to npm and GitHub Releases and permissions to write to the browser cache. Even scrapeless browser --help can trigger installation.
A task is still pendingPreserve its task ID and resume retrieval. Check the returned status before treating the task as complete.
The Scraping API script cannot be foundConfirm that the entire skill directory was installed. Resolve scripts/scrape.mjs relative to that directory, rather than your project directory.

For manual installation, copy each skill’s entire directory, including any scripts/, references/, and agents/ subdirectories. Copying only SKILL.md can leave required files behind.