Proxies

Introduction

The proxy system is a component of the scraping browser that forwards connections through a proxy server. It is primarily used to prevent IP blocking, access geo-restricted content, and enhance anonymity. Scraping Browser provides a configurable proxy system that allows for precise control over automated traffic routing, including anonymity, geolocation targeting, and increased reliability, making it easy to integrate proxies into your workflow.

Core Capabilities

The core capability of the proxy system lies in hiding the real IP address through an intermediary server and automatically or on-demand rotating through different addresses in an IP pool. It also allows selecting proxy servers in specific geographic locations, simulating user access from various parts of the world.

Application Scenarios

Primarily applicable to scenarios requiring large-scale data scraping and bypassing target website IP restrictions or blocks. It is also a key tool for accessing geo-restricted content and maintaining online anonymity in tasks such as market research.

Classification

With Scrapeless, you have two proxy configuration options:

Using built-in proxies: Easily route traffic through our managed residential proxies.
Using custom proxies: Use custom HTTP/HTTPS/SOCKS5 proxies for more flexible control over network routing.

Operational Guidance

Using Built-in Proxies

Using Scraping Browser’s built-in proxies routes traffic through its managed residential proxies.

const { ScrapelessClient } = require('@scrapeless-ai/sdk');
const puppeteer = require('puppeteer-core');
const client = new ScrapelessClient({ apiKey: 'API Key' });
 
const { browserWSEndpoint } = client.browser.create({
  sessionName: 'sdk_test',
  sessionTTL: 180,
  proxyCountry: 'AU',
  proxyState: 'NSW',      // Optional: state code
  proxyCity: 'sydney', // Optional: city name
  sessionRecording: true,
});
 
(async () => {
  const browser = await puppeteer.connect({ browserWSEndpoint });
  const page = await browser.newPage();
  await page.goto('https://www.scrapeless.com');
  console.log(await page.title());
  await browser.close();
})();

const puppeteer = require('puppeteer-core');
const connectionURL = 'wss://browser.scrapeless.com/api/v2/browser?token=APIKey&sessionTTL=180&proxyCountry=AU&proxyState=NSW&proxyCity=sydney';
 
(async () => {
  const browser = await puppeteer.connect({ browserWSEndpoint: connectionURL });
  const page = await browser.newPage();
  await page.goto('https://www.scrapeless.com');
  console.log(await page.title());
  await browser.close();
})();

const { chromium } = require('playwright-core');
const connectionURL = 'wss://browser.scrapeless.com/api/v2/browser?token=APIKey&sessionTTL=180&proxyCountry=AU&proxyState=NSW&proxyCity=sydney';
 
(async () => {
  const browser = await chromium.connectOverCDP(connectionURL);
  const page = await browser.newPage();
  await page.goto('https://www.scrapeless.com');
  console.log(await page.title());
  await browser.close();
})();

Geographic Proxy Selection

Our built-in proxy network spans 195 countries and territories worldwide, enabling precise geographic targeting for your scraping operations. Each proxy location is optimized for performance and reliability, allowing you to simulate authentic user access from specific regions.

For comprehensive country and region selection, including state and city-level targeting, please refer to our detailed proxy documentation.

Custom Proxies

If you want to use your own proxy service/provider, you can specify the following parameters when creating a session. If you use your own proxy provider, you will not be charged for proxy usage.

💡Custom proxy functionality is currently only available to subscribers. Upgrade here

const { ScrapelessClient } = require('@scrapeless-ai/sdk');
const puppeteer =require('puppeteer-core');
const client = new ScrapelessClient({ apiKey: 'API Key' });
 
const proxyUrl = client.proxies.proxy({
  country: 'US',
  sessionDuration: 30,
  sessionId: 'test-session-123',
  gateway: 'gate.smartproxy.com:7000'
});
 
(async () => {
  const browser = await puppeteer.launch({ args: [`--proxy-server=${proxyUrl}`], headless: false });
  const page = await browser.newPage();
  await page.goto('API_URL');
  await browser.close();
})();

const puppeteer = require('puppeteer-core');
 
(async () => {
  try {
    const proxyUrl = 'http://gw-us.scrapeless.com:8789';
    const username = 'username';
    const password = 'password';
    const browser = await puppeteer.launch({ args: [`--proxy-server=${proxyUrl}`], headless: false });
    const page = await browser.newPage();
    await page.authenticate({ username, password });
    await page.goto('API_URL');
    await browser.close();
  } catch (e) {
    console.log(e)
  }
})();

const { chromium } = require('playwright-core');
 
(async () => {
  try {
    const proxyUrl = 'http://gw-us.scrapeless.com:8789';
    const username = 'username';
    const password = 'password';
    const browser = await chromium.launch({ args: [`--proxy-server=${proxyUrl}`], headless: false });
    const page = await browser.newPage();
    await page.authenticate({ username, password });
    await page.goto('API_URL');
    await browser.close();
  } catch (e) {
    console.log(e)
  }
})()

The built-in proxy of our cloud browser only supports the residential type, which is more suitable for large-scale data extraction operations. We also separately offer various proxy types including Residential proxies, Datacenter proxies, Static ISP Proxies, and IPv6 Proxies, with support for more refined configurations. You can also purchase our proxies independently and inject them into the cloud browser to match more diverse usage scenarios. For example, if you need an IP address that remains consistent and dedicated across sessions, you can purchase our Static ISP Proxies. Guide

Captcha Solving Browser Use