CDP API

Scrapeless 爬虫浏览器扩展了标准的 CDP (Chrome DevTools Protocol) 功能，增加了一系列强大的自定义函数来增强浏览器自动化能力。本文档主要介绍与 CAPTCHA 处理相关的 CDP 函数。

CAPTCHA 解决方案功能

功能概述

爬虫浏览器包含高级 CAPTCHA 解决方案功能，自动处理网页上常见的 CAPTCHA 类型。

支持的 CAPTCHA 类型

reCaptcha
Cloudflare Turnstile
Cloudflare 5s Challenge
AWS WAF

事件监控机制

核心事件

爬虫浏览器提供三个核心事件来监控 CAPTCHA 解决过程：

事件名称	描述
`Captcha.detected`	检测到 CAPTCHA
`Captcha.solveFinished`	CAPTCHA 解决完成
`Captcha.solveFailed`	CAPTCHA 解决失败

事件响应数据结构

字段	类型	描述
`type`	string	CAPTCHA 类型: `recaptcha` `turnstile`
`success`	boolean	解决结果
`message`	string	状态消息: `"NOT_DETECTED"` `"SOLVE_FINISHED"` `"SOLVE_FAILED"` `"INVALID"`
`token?`	string	成功返回的 Token（可选）

实现示例

// Listen for CAPTCHA solving events
const client = await page.createCDPSession();
 
client.on('Captcha.detected', (result) => {
  console.log('Captcha detected:', result);
});
 
await new Promise((resolve, reject) => {
  client.on('Captcha.solveFinished', (result) => {
    if (result.success) resolve();
  });
  client.on('Captcha.solveFailed', () =>
    reject(new Error('Captcha solve failed'))
  );
  setTimeout(() =>
      reject(new Error('Captcha solve timeout')),
    5 * 60 * 1000
  );
});

高级配置 API

Scrapeless 爬虫浏览器提供一系列高级 API，用于对 CAPTCHA 求解器的行为进行细粒度控制。支持以下 API：

API 名称	描述
`Captcha.setAutoSolve`	控制自动 CAPTCHA 求解行为
`Captcha.setToken`	设置 CAPTCHA 服务的认证令牌
`Captcha.setConfig`	配置所有 CAPTCHA 求解器参数
`Captcha.solve`	手动触发 CAPTCHA 求解过程
`Agent.click`	模拟鼠标点击
`Agent.liveURL`	获取当前会话页面实时 URL

详细 API 说明

1. `Captcha.setConfig`

配置 CAPTCHA 求解器的所有参数。

await client.send('Captcha.setConfig', {
    config: JSON.stringify(
        {
            apiKey: "your-token",
            autoSolve: true,
            enabledForRecaptcha: true,
            enabledForRecaptchaV3: true,
            enabledForTurnstile: true
        }
    )
});

2. `Captcha.solve`

手动触发 CAPTCHA 求解过程。

const { Puppeteer, createPuppeteerCDPSession } = require('@scrapeless-ai/sdk');
 
(async () => {
    const browser = await Puppeteer.connect({
      session_name: 'sdk_test',
      session_ttl: 180,
      proxy_country: 'US',
      session_recording: true,
      defaultViewport: null
    });
    const page = await browser.newPage();
    await page.goto('https://www.scrapeless.com');
    const cdpSession = await createPuppeteerCDPSession(page);
 
    await cdpSession.solveCaptcha({ timeout: 30000 });
})();

const result = await client.send('Captcha.solve', {
    detectTimeout: 10 * 1000,
    options: JSON.stringify([{
        type: 'rcaptcha',
        disabled: true,  // Disable rCaptcha solving
    }])
});
console.log(result);  // { type: 'recaptcha', success: true, message: 'solve_finished', token: 'xxx' }

import json
 
options = [{'type': 'rcaptcha', 'disabled': True}]
result = await client.send('Captcha.solve', {
'detectTimeout': 10_000,
'options': json.dumps(options)
})
print(result)  # { 'type': 'recaptcha', 'success': True, 'message': 'solve_finished', 'token': 'xxx' }

3. `Agent.click`

模拟鼠标点击。

const { Puppeteer, createPuppeteerCDPSession } = require('@scrapeless-ai/sdk');
 
(async () => {
    const browser = await Puppeteer.connect({
      session_name: 'sdk_test',
      session_ttl: 180,
      proxy_country: 'US',
      session_recording: true,
      defaultViewport: null
    });
    const page = await browser.newPage();
    await page.goto('https://www.scrapeless.com');
    const cdpSession = await createPuppeteerCDPSession(page);
 
    await cdpSession.realClick('button');
})();

await client.send('Agent.click', {
    selector: 'button',
});

await client.send('Agent.click', {
  'selector': 'button',
})

4. `Agent.type`

模拟键盘输入。

const { Puppeteer, createPuppeteerCDPSession } = require('@scrapeless-ai/sdk');
 
(async () => {
    const browser = await Puppeteer.connect({
      session_name: 'sdk_test',
      session_ttl: 180,
      proxy_country: 'US',
      session_recording: true,
      defaultViewport: null
    });
    const page = await browser.newPage();
    await page.goto('https://www.scrapeless.com');
    const cdpSession = await createPuppeteerCDPSession(page);
 
    await cdpSession.realFill('input', 'Hello, Scrapeless!');
})();

await client.send('Agent.type', {
  selector: 'input',
  content: 'Hello, Scrapeless!',
});

await client.send('Agent.type', {
  'selector': 'input',
  'content': 'Hello, Scrapeless!',
})

5. `Agent.liveURL`

获取当前会话页面实时 URL。

const { Puppeteer, log as Log, createPuppeteerCDPSession } = require('@scrapeless-ai/sdk');
const logger = Log.withPrefix('puppeteer-example');
 
(async () => {
    const browser = await Puppeteer.connect({
      session_name: 'sdk_test',
      session_ttl: 180,
      proxy_country: 'US',
      session_recording: true,
      defaultViewport: null
    });
    const page = await browser.newPage();
    await page.goto('https://www.scrapeless.com');
    const cdpSession = await createPuppeteerCDPSession(page);
 
    const { error, liveURL } = await cdpSession.liveURL();
    if (error) {
      logger.error('Failed to get current page URL:', error);
    } else {
      logger.info('Current page URL:', liveURL);
    }
    await browser.close();
})();

await page.goto("https://www.scrapeless.com", {
    waitUntil: "networkidle0",
    timeout: 30000,
});
const client = await page.createCDPSession();
const result = await client.send("Agent.liveURL");
console.log(result);

await page.goto("https://www.scrapeless.com", timeout=60000, wait_until="load")
client = await page.target.createCDPSession()
result = await client.send('Agent.liveURL')
print(result)

6. `Captcha.imageToText`

解决图片验证码

const { Puppeteer, createPuppeteerCDPSession } = require('@scrapeless-ai/sdk');
 
const browser = await Puppeteer.connect({
  session_name: 'sdk_test',
  session_ttl: 180,
  proxy_country: 'US',
  session_recording: true,
  defaultViewport: null
});
const page = await browser.newPage();
await page.goto('https://www.example.com');
const cdpSession = await createPuppeteerCDPSession(page);
 
await cdpSession.imageToText({
  imageSelector: '.captcha__image',
  inputSelector: 'input[name="captcha"]',
  timeout: 30000,
})

const client = await page.createCDPSession();
await client.send("Captcha.imageToText", {
  imageSelector: '.captcha__image',
  inputSelector: 'input[name="captcha"]',
  timeout: 30000,
})

await page.goto("https://www.scrapeless.com", timeout=60000, wait_until="load")
client = await page.target.createCDPSession()
await client.send('Captcha.imageToText', {
  'imageSelector': '.captcha__image',
  'inputSelector': 'input[name="captcha"]',
  'timeout': 30000,
})

自定义指纹验证码解决