Real-time Signaling

Real-time Signaling is an advanced event-driven system for handling asynchronous communication in automated workflows. This signal-based architecture enables seamless interaction between automation scripts and external systems, with Multi-Factor Authentication (MFA) handling being one of its most critical applications.

Overview

The Real-time Signaling system provides a robust framework for managing asynchronous events in automation workflows. While MFA verification represents a primary use case, the system’s flexible architecture supports diverse event-driven scenarios.

Multi-Factor Authentication (MFA) is a critical security feature that often becomes a bottleneck for automated workflows, causing failures or account lockouts.

Why MFA Handling is Critical for Automation:

✅ Ensure uninterrupted access: Handle unexpected SMS codes, email OTPs, or TOTP authentication without workflow disruption

✅ Prevent workflow crashes: Traditional automation freezes when MFA prompts appear, while Scraping Browser handles these smoothly

✅ Maintain stable sessions: Long-running tasks stay logged in with secure verification state management

✅ Reduce account risk: Human-like verification behavior lowers security trigger likelihood

Beyond MFA: Universal Event System

The signaling system extends beyond authentication to handle:

• Form submissions and status updates
• Task progress notifications
• API callbacks and webhooks
• User interactions and state changes
• Process coordination across systems
• Real-time monitoring and alerts

Complete signaling solution:

• Reliable MFA verification (primary use case)
• Universal event handling for automation needs
• Multiple input methods for codes and data
• Asynchronous processing (non-blocking)
• Full CDP and HTTP API support
• Universal compatibility with authentication flows

Supported APIs

Scrapeless supports both CDP and HTTP interfaces for real-time signaling:

CDP APIs

The CDP Signal APIs allow you to send and receive signals through event channels in browser automation workflows. Learn more

Signal.send

Sends signal data to a specified event channel.

Request Format:

{
  "method": "Signal.send",
  "params": {
    "event": "string",
    "data": "object"
  }
}

Parameters:

Example:

await client.send('Signal.send', {
  event: 'mfa_code',
  data: { code: '123456', type: 'sms' }
});

Signal.wait

Waits for signal data on a specified event channel.

Request Format:

{
  "method": "Signal.wait",
  "params": {
    "event": "string",
    "timeout": 60000
  }
}

Parameters:

Example:

const result = await client.send('Signal.wait', {
  event: 'mfa_code',
  timeout: 60000
});
console.log('Received MFA code:', result.data);

Signal.list

Lists all pending event names in the queue.

Request Format:

{
  "method": "Signal.list",
  "params": {}
}

Parameters: None required

Example:

const list = await client.send('Signal.list');
console.log('Pending events:', list.events);
// Output: ["mfa_code", "captcha_result", "order_status"]
 
// Check for specific event
if (list.events.includes('mfa_code')) {
  console.log('MFA code in queue');
}

Signal.stats

Retrieves queue statistics and subscriber information.

Request Format:

{
  "method": "Signal.stats",
  "params": {}
}

Parameters: None required

Response Fields:

Example:

const client = await page.target().createCDPSession();
 
// Get statistics
const stats = await client.send('Signal.stats');
console.log('Pending events:', stats.events);
console.log('Waiting subscribers:', stats.waiters);

Signal.clear

Clears specified or all events from the queue.

Request Format:

{
  "method": "Signal.clear",
  "params": {
    "event": "string (optional)"
  }
}

Parameters:

HTTP REST APIs

For external systems, HTTP REST endpoints provide a simpler alternative to CDP connection setup. This eliminates the need to establish WebSocket connections for every data transmission. Learn more

Default Browser Endpoint Prefix: https://browser.scrapeless.com/browser/{taskId}

POST /signal/send

Sends a signal via HTTP.

Request Format:

POST https://browser.scrapeless.com/browser/{taskId}/signal/send?x-api-token={API_KEY}
Content-Type: application/json
 
{
  "event": "string",
  "data": "object"
}

Parameters:

Example:

curl -X POST 'https://browser.scrapeless.com/browser/{taskId}/signal/send?x-api-token={API_KEY}' \
  -H 'Content-Type: application/json' \
  -d '{
    "event": "mfa_code",
    "data": { "code": "123456", "type": "sms" }
  }'

GET /signal/wait

Waits for a signal via HTTP GET request.

Request Format:

GET https://browser.scrapeless.com/browser/{taskId}/signal/wait?x-api-token={API_KEY}&event={event}&timeout={timeout}

Parameters:

Example:

# Wait for MFA code with 60 second timeout
curl -X GET 'https://browser.scrapeless.com/browser/{taskId}/signal/wait?x-api-token={API_KEY}&event=mfa_code&timeout=60000'

GET /signal/list

Lists all pending event names.

Request Format:

GET https://browser.scrapeless.com/browser/{taskId}/signal/list?x-api-token={API_KEY}

Parameters: None required

Example:

# List all pending events
curl -X GET 'https://browser.scrapeless.com/browser/{taskId}/signal/list?x-api-token={API_KEY}'

GET /signal/stats

Retrieves queue statistics.

Request Format:

GET https://browser.scrapeless.com/browser/{taskId}/signal/stats?x-api-token={API_KEY}

Parameters: None required

Example:

# Get queue statistics
curl -X GET 'https://browser.scrapeless.com/browser/{taskId}/signal/stats?x-api-token={API_KEY}'

DELETE /signal/clear

Clears events from the queue.

Request Format:

DELETE https://browser.scrapeless.com/browser/{taskId}/signal/clear?x-api-token={API_KEY}
Content-Type: application/json
 
{
  "event": "string (optional)"
}

Parameters:

Example:

# Clear specific event
curl -X DELETE 'https://browser.scrapeless.com/browser/{taskId}/signal/clear?x-api-token={API_KEY}' \
  -H 'Content-Type: application/json' \
  -d '{"event": "mfa_code"}'
 
# Clear all events
curl -X DELETE 'https://browser.scrapeless.com/browser/{taskId}/signal/clear?x-api-token={API_KEY}'

Best Practices

Use Appropriate Timeouts

Set timeout values based on expected verification delays. Typical MFA delivery times range from 10-60 seconds.

Error Handling

Always check response status codes (200, 408, 400) to handle success, timeout, and error cases appropriately.

Queue Monitoring

Regularly check queue statistics to detect backlog conditions that might indicate system issues.

Event Channel Naming

Use descriptive event channel names (e.g., mfa_code, email_verification, totp_token) to avoid confusion in multi-event scenarios.

Asynchronous Processing

Leverage async signal handling to prevent script blocking while waiting for user input.

Queue Cleanup

Clear obsolete events from the queue to maintain system performance and prevent memory issues.

Complete Example

const puppeteer = require('puppeteer-core');
 
(async () => {
    const API_TOKEN = 'API Key';
    const API_URL = 'https://api.scrapeless.com/api/v2/browser'; // Create session task API endpoint
 
    try {
        // Step 1: Get session taskId via HTTP API
        const sessionResponse = await fetch(API_URL, {
            method: 'GET',
            headers: {'x-api-token': API_TOKEN},
        });
 
        const {taskId} = await sessionResponse.json();
        console.log('Session created with task ID:', taskId);
 
        // Step 2: Connect to browser by taskId
        const browser = await puppeteer.connect({
            browserWSEndpoint: `wss://api.scrapeless.com/browser/${taskId}`,
            headers: {'x-api-token': API_TOKEN},
        });
 
        // Step 3: Navigate to page and wait for signal
        const page = await browser.newPage();
        await page.goto("https://example.com", {waitUntil: "domcontentloaded"});
 
        const client = await page.createCDPSession();
 
        console.log('Waiting for example event...');
        const result = await client.send('Signal.wait', {
            event: 'example_event',
            timeout: 60000
        });
 
        console.log('Received example data:', result.data);
 
        await browser.close();
    } catch (error) {
        console.error('Error occurred:', error);
    }
})();

# Send data to example event using curl
# Replace {taskId} with the session ID from the first step
curl 'https://browser.scrapeless.com/browser/{taskId}/signal/send' \
  -H 'content-type: application/json' \
  -H 'token: API Key' \
  --data-raw $'{  "event":"example_event",  "data":{ "email":"example@host.com", "password": "mypassword"  }}'

The Scraping Browser Real-time Signaling system provides a robust, flexible framework for handling multi-factor authentication in automated workflows. By combining CDP and HTTP interfaces with an intelligent event queue system, it enables seamless integration of external verification systems while maintaining security and reliability.

Whether you’re building simple automation scripts or complex multi-system orchestrations, the signal system’s asynchronous, non-blocking architecture ensures your workflows complete reliably—even when MFA verification is required.