JS Render

Universal Scraping API is a powerful web content retrieval service that supports complex web page rendering and interaction scenarios.

Basic Request Structure

{
  "actor": "unlocker.webunlocker",
  "input": {
    "url": "https://example.com",
    "js_render": false,
    "headless": false
  },
  "proxy": {
    "country": "US"
  }
}

Core Features

JavaScript Rendering

JavaScript rendering enables handling of dynamically loaded content and SPAs (Single Page Applications). Enables a complete browser environment, supporting more complex page interactions and rendering requirements.

js_render=true,we will use the browser to request.

{
  "actor": "unlocker.webunlocker",
  "input": {
    "url": "https://www.google.com/",
    "js_render": true
  },
  "proxy": {
    "country": "US"
  }
}

JavaScript Instructions

Provides an extensive set of JavaScript directives that allow you to dynamically interact with web pages.

These directives enable you to click elements, fill out forms, submit forms, or wait for specific elements to appear, providing flexibility for tasks such as clicking a “read more” button or submitting a form.

{
  "actor": "unlocker.webunlocker",
  "input": {
    "url": "https://example.com",
    "js_render": true,
    "js_instructions": [
      {
        "wait_for": [
          ".dynamic-content",
          30000
        ]
        // Wait for element
      },
      {
        "click": [
          "#load-more",
          1000
        ]
        // Click element
      },
      {
        "fill": [
          "#search-input",
          "search term"
        ]
        // Fill form
      },
      {
        "keyboard": [
          "press",
          "Enter"
        ]
        // Simulate key press
      },
      {
        "evaluate": "window.scrollTo(0, document.body.scrollHeight)"
        // Execute custom JS
      }
    ]
  }
}

Here are some common actions you can perform with JavaScript Instructions:

JavaScript Instructions Reference

Instruction	Syntax	Description	Example
`wait_for`	`[selector, timeout]`	Wait for element to appear	`{"wait_for": [".content", 30000]}`
`click`	`[selector, delay]`	Click element	`{"click": [".button", 1000]}`
`fill`	`[selector, value]`	Fill form	`{"fill": ["#input", "text"]}`
`wait`	`milliseconds`	Fixed wait time	`{"wait": 2000}`
`evaluate`	`javascript_code`	Execute JS code	`{"evaluate": "console.log('test')"}`
`keyboard`	`[action, value, delay?]`	Keyboard operation	See keyboard operations table below

Keyboard Operations

Operation	Syntax	Description	Example
Press key	`["press", keyInput]`	Press a specific keyInput	`{"keyboard": ["press", "Enter"]}`
Type text	`["type", text, delay?]`	Type text with optional delay	`{"keyboard": ["type", "Hello", 20]}`
Key down	`["down", key]`	Hold down a key	`{"keyboard": ["down", "Shift"]}`
Key up	`["up", key]`	Release a key	`{"keyboard": ["up", "Shift"]}`

Supported Special KeyInput types: https://pptr.dev/api/puppeteer.keyinput

Resource Control

Resource loading control system for optimizing performance and bandwidth usage.

{
  "actor": "unlocker.webunlocker",
  "input": {
    "url": "https://example.com",
    "js_render": true,
    "block": {
      "resources": [
        "Image",
        "Font",
        "Stylesheet",
        "Script"
      ],
      "urls": [
        // Optional, URL pattern-based blocking
        "*.analytics.com/*",
        "*/ads/*"
      ]
    }
  }
}

Complete Resource Types Reference:

Resource Type	Description	Impact
`Document`	Main document and iframes	Core page content
`Stylesheet`	CSS files	Page styling and layout
`Image`	Images and icons	Visual content
`Media`	Audio and video resources	Multimedia content
`Font`	Web fonts	Text rendering
`Script`	JavaScript files	Page functionality
`TextTrack`	Video subtitles and captions	Media accessibility
`XHR`	XMLHttpRequest calls	Legacy async requests
`Fetch`	Fetch API requests	Modern async requests
`Prefetch`	Prefetched resources	Performance optimization
`EventSource`	Server-sent events	Real-time updates
`WebSocket`	WebSocket connections	Bidirectional communication
`Manifest`	Web app manifests	PWA configuration
`SignedExchange`	Signed HTTP exchanges	Content authenticity
`Ping`	Ping requests	Analytics and tracking
`CSPViolationReport`	CSP violation reports	Security monitoring
`Preflight`	CORS preflight requests	Cross-origin security
`Other`	Unclassified resources	Miscellaneous

Usage Example:

{
  "actor": "unlocker.webunlocker",
  "input": {
    "url": "https://example.com",
    "js_render": true,
    "block": {
      "resources": [
        "Image",
        "Font",
        "Stylesheet",
        "Script",
        "Media",
        "Ping",
        "Prefetch"
      ]
    }
  }
}

Best Practices for Resource Blocking:

Performance Optimization
- Enable js_render only when necessary
- Use resource blocking wisely, Block non-essential resources for faster loading
- Consider blocking Prefetch and Ping for reduced network usage
- Keep Document and critical Script resources unblocked
Bandwidth Management
- Block Image and Media for bandwidth-intensive pages
- Consider blocking Font to use system fonts instead
Stability Enhancement
- Implement request retry mechanisms
- Add error handling logic
- Use wait_for instead of fixed wait
Resource Efficiency
- Load resources on demand
- Close unnecessary connections promptly

Note: Resource type strings are case-sensitive. Use exact matches as shown in the reference table.

Getting Started Captcha Solving