Scraping BrowserFeaturesLive Session

Live Session

Scraping browser’s live view feature allows you to view and control browser sessions in real time. Specifically, the live view feature includes viewing, clicking, typing, and scrolling within any active browser session. Therefore, you can easily monitor automated processes, debug automation scripts, and intervene manually in browser sessions.

In scrapeless, you can view or control browser sessions in two places: the playground and the session management interface.

How to Use

Create a Scrapeless Browser Session

First, you need to create a session. There are two ways to do this:

Create a Session via Playground

image1.png

Create a Session via API

You can also use our API to create a session. Refer to the API documentation: Scraping Browser API Docs. Our session feature will help you manage this session, including the live view function.

const puppeteer =require('puppeteer-core');
 
const token = 'API Key'
 
// custom fingerprint
const fingerprint = {
    platform: 'Windows',
}
 
const query = new URLSearchParams({
    session_ttl: 180,
    session_name: 'test_scraping', // session name
    proxy_country: 'ANY',
    token: token,
    fingerprint: encodeURIComponent(JSON.stringify(fingerprint)),
});
 
const connectionURL = `wss://browser.scrapeless.com/browser?${query.toString()}`;
 
(async () => {
    const browser = await puppeteer.connect({browserWSEndpoint: connectionURL});
    const page = await browser.newPage();
 
    await page.goto('https://www.scrapeless.com');
    await new Promise(res => setTimeout(res, 3000));
 
    await page.goto('https://www.google.com');
    await new Promise(res => setTimeout(res, 3000));
 
    await page.goto('https://www.youtube.com');
    await new Promise(res => setTimeout(res, 3000));
 
    await browser.close();
})();

View Live Session

In the scrapeless session management interface, you can easily view live sessions. There are also two ways to view them:

View Playground Session in Real Time

After creating a session in the playground, you can see the browser running in real time on the right side.

image2.png

View API Session in Real Time

After creating a session via the API, you can see the list of running sessions on the session page. Clicking on the Action details allows you to preview the browser’s operation in real time. Here you can choose to view the live session on-site or copy the session URL to view the live session. We provide two operation videos for your reference.

On-site Display

image3.gif

Live URL

You can copy the Live URL from the list of running sessions and paste it into a browser to access it directly.

image4.gif

Obtain Live URL via API

You can obtain the Live URL by calling the API. In the following code example, we first fetch all currently running sessions using the Running Sessions API, and then retrieve the Live URL of a specific session using the Live URL API:

const API_CONFIG = {
    host: 'https://api.scrapeless.com',
    headers: {
        'x-api-token': 'API Key',
        'Content-Type': 'application/json'
    }
};
 
const requestOptions = {
    method: 'GET',
    headers: new Headers(API_CONFIG.headers)
};
 
async function fetchBrowserSessions() {
    try {
        // Fetch running browser sessions
        const sessionResponse = await fetch(`${API_CONFIG.host}/browser/running`, requestOptions);
 
        if (!sessionResponse.ok) {
            throw new Error(`failed to fetch sessions: ${sessionResponse.status} ${sessionResponse.statusText}`);
        }
 
        const sessionResult = await sessionResponse.json();
 
        // Process sessions data
        const sessions = sessionResult.data;
        if (!sessions || !Array.isArray(sessions) || sessions.length === 0) {
            console.log("no active browser sessions found");
            return;
        }
 
        // Get first session task ID
        const taskId = sessions[0]?.taskId;
        if (!taskId) {
            console.log("task id not found in the session data");
            return;
        }
 
        // Fetch live URL for the task
        await fetchLiveUrl(taskId);
    } catch (error) {
        console.error("error fetching browser sessions:", error.message);
    }
}
 
async function fetchLiveUrl(taskId) {
    try {
        const liveResponse = await fetch(`${API_CONFIG.host}/browser/${taskId}/live`, requestOptions);
 
        if (!liveResponse.ok) {
            throw new Error(`failed to fetch live url: ${liveResponse.status} ${liveResponse.statusText}`);
        }
 
        const liveResult = await liveResponse.json();
        if (liveResult && liveResult.data) {
            console.log(`taskId: ${taskId}`);
            console.log(`liveUrl: ${liveResult.data}`);
        } else {
            console.log("no live url data available for this task");
        }
    } catch (error) {
        console.error(`error fetching live url for task ${taskId}:`, error.message);
    }
}
 
fetchBrowserSessions().then(r => { });
PlaywrightSession Replay