Node.js SDK

Overview

The official Scrapeless Node.js SDK provides access to browser automation, scraping, crawling, proxies, search results, and AI chat extraction. This guide follows the SDK repository README, with runnable examples and configuration details.

Requirements

Use Node.js with npm, pnpm, or Yarn. The package manifest does not declare a minimum Node.js version; the repository’s publishing workflow uses Node.js 20. JavaScript and TypeScript are supported, with both ES module and CommonJS exports.

The examples below use ES modules. Save them as .mjs files, or set "type": "module" in your project’s package.json.

Installation

npm install @scrapeless-ai/sdk

You can also use pnpm add @scrapeless-ai/sdk or yarn add @scrapeless-ai/sdk.

Authentication / API Key

Log in to the Scrapeless dashboard and create an API key. Export it before running the examples:

export SCRAPELESS_API_KEY="YOUR_API_KEY"

Keep your API key in your environment or secret manager instead of committing it to source control.

new Scrapeless() reads SCRAPELESS_API_KEY. You can also pass an apiKey option when constructing the client.

Quick Start

Save this as quickstart.mjs and run node quickstart.mjs after setting your API key.

import { Scrapeless } from '@scrapeless-ai/sdk';
 
const client = new Scrapeless();
const result = await client.universal.scrape({
  actor: 'unlocker.webunlocker',
  input: { url: 'https://example.com', method: 'GET', redirect: false }
});
console.log(result);

Product Coverage Matrix

Usage Examples

Unless an example initializes its own client, reuse const client = new Scrapeless() from Quick Start.

Browser

Install Puppeteer for this example: npm install puppeteer-core. For Playwright and the SDK’s browser wrappers, see the browser integration examples.

Advanced browser session management supporting Playwright and Puppeteer frameworks, with configurable anti-detection capabilities (e.g., fingerprint spoofing, CAPTCHA solving) and extensible automation workflows:

import { Scrapeless } from '@scrapeless-ai/sdk';
import puppeteer from 'puppeteer-core';
 
const client = new Scrapeless();
 
// Create a browser session
const { browserWSEndpoint } = await client.browser.create({
  sessionName: 'my-session',
  sessionTTL: 180,
  proxyCountry: 'US'
});
 
// Connect with Puppeteer
const browser = await puppeteer.connect({
  browserWSEndpoint: browserWSEndpoint
});
 
try {
  const page = await browser.newPage();
  await page.goto('https://example.com');
  console.log(await page.title());
} finally {
  await browser.close();
}

Browser Profile

Manage browser profiles for persistent sessions.

const createResponse = await client.profiles.create('My Profile');
console.log('Profile created:', createResponse);
 
const profiles = await client.profiles.list({ page: 1, pageSize: 10 });
console.log('Profiles:', profiles.docs);
 
const profile = await client.profiles.get(createResponse.profileId);
console.log('Profile details:', profile);
 
// Delete the profile when it is no longer needed.
await client.profiles.delete(createResponse.profileId);

Scraping API

Direct data extraction APIs for websites (e.g., e-commerce, travel platforms). Retrieve structured product information, pricing, and reviews with pre-built connectors:

const result = await client.scraping.scrape({
  actor: 'scraper.google.search',
  input: {
    'q': 'coffee',
    'hl': 'en',
    'gl': 'us'
  }
});
 
console.log(result.data);

Web Unlocker

Extract data from websites using Web Unlocker (exposed as client.universal).

const result = await client.universal.scrape({
  actor: 'unlocker.webunlocker',
  input: { url: 'https://example.com', method: 'GET', redirect: false }
});
console.log(result);

Crawl

Extract data from single pages or traverse entire domains, exporting in formats including Markdown, JSON, HTML, screenshots, and links.

const result = await client.scrapingCrawl.scrapeUrl('https://example.com');
 
console.log(result);

Proxy

Generate a proxy URL using your gateway and session settings.

const proxyUrl = client.proxies.proxy({
  type: 'residential',
  country: 'US',
  sessionDuration: 30,
  sessionId: client.proxies.generateSessionId(),
  gateway: 'your-proxy-gateway:port'
});
console.log(proxyUrl);

AI Scraper

Create a task: client.aiScraper.createTask(request)

Pass the required actor and actor-specific input. An optional webhook object accepts a callback url. The promise resolves to the full API response, including task_id and status, and task_result when available.

Extract AI chat content in bulk to monitor brand mentions, compare answers, and analyze competitive intelligence from the latest models. Retrieve URLs, prompts, Markdown answers, citations, and more through one integration.

Supported actors include scraper.chatgpt, scraper.perplexity, scraper.copilot, scraper.gemini, scraper.aimode, scraper.overview, scraper.grok, and scraper.alexa. The input JSON depends on the actor; see the AI Scraper documentation for detailed parameters. The optional webhook JSON contains a callback url.

import { Scrapeless } from '@scrapeless-ai/sdk';
 
const client = new Scrapeless(); // Uses SCRAPELESS_API_KEY
 
const task = await client.aiScraper.createTask({
  actor: 'scraper.chatgpt',
  input: {
    prompt: 'Most reliable proxy service for data extraction',
    country: 'US',
    web_search: true
  },
  // Optional: webhook: { url: 'https://your-webhook.example.com' }
});
console.log('Created task:', task);

Get task status and result: client.aiScraper.getTaskResult(taskId)

Pass the task_id from creation. Continue in the same script, or store the ID and retrieve the result in a later request.

const result = await client.aiScraper.getTaskResult(task.task_id);
 
switch (result.status) {
  case 'success':
    console.log('Task result:', result.task_result);
    break;
  case 'failed':
    console.error('Task failed:', result.message);
    break;
  case 'running':
    console.log('Task is running. Retrieve the result again later.');
    break;
}

Both methods return the API JSON unchanged. Creation returns task_id, status, and, when available, task_result. Result retrieval returns status, task_result when available, and message on failure. Status is success, failed, or running; the SDK does not poll automatically.

Creation can already include a result. Inspect its status before scheduling further requests. If you implement polling, use a delay and an overall timeout.

Google Search API

const result = await client.deepserp.scrape({
  actor: 'scraper.google.search',
  input: { q: 'nike site:www.nike.com' }
});
console.log(result);

For more complete integrations, browse the repository’s examples directory.

Error Handling

Catch ScrapelessError for API request failures. Check an AI Scraper response’s status separately: a task can return failed without the HTTP request throwing an error.

import { Scrapeless, ScrapelessError } from '@scrapeless-ai/sdk';
 
try {
  const client = new Scrapeless();
  const result = await client.universal.scrape({
    actor: 'unlocker.webunlocker',
    input: { url: 'https://example.com', method: 'GET' }
  });
  console.log(result);
} catch (error) {
  if (error instanceof ScrapelessError) {
    console.error('Scrapeless error:', error.message);
    console.error('Status code:', error.statusCode);
  } else {
    throw error;
  }
}

Configuration / Environment Variables

The API key is required. Endpoint overrides are optional; the table shows their defaults.

import { Scrapeless } from '@scrapeless-ai/sdk';
 
const client = new Scrapeless({
  apiKey: process.env.SCRAPELESS_API_KEY,
  timeout: 30000, // Request timeout in milliseconds
  baseApiUrl: 'https://api.scrapeless.com',
  browserApiUrl: 'https://browser.scrapeless.com',
  scrapingCrawlApiUrl: 'https://api.scrapeless.com'
});

Explicit configuration takes precedence over environment variables. The default request timeout is 30,000 milliseconds.

Support

• SDK source and README
• Report an issue
• Scrapeless documentation
• Join the Discord community
• Email support

The SDK is released under the MIT License.

Related Projects

• Python SDK documentation · GitHub
• Go SDK documentation · GitHub
• SDK Overview