Introduction
Crawl is a cutting-edge feature designed specifically for large-scale data scraping and processing. It distinguishes itself through its core strengths: intelligent recursive scraping, robust bulk data processing capabilities, and flexible multi-format output. These features enable enterprises and developers to efficiently acquire and process vast amounts of web data, driving applications in AI training, market analysis, business decision-making, and more.
Key Features & Advantages
- Large-Scale Crawling Capabilities: Supports massive single-page crawling and intelligent recursive crawling.
- Flexible Multi-Format Delivery: Output data in multiple formats, including JSON, Markdown, Metadata, HTML, Links, and Screenshots, ensuring compatibility with diverse workflows and systems.
- Advanced Anti-Detection Strategy: Powered by our independently developed Chromium kernel, offering robust anti-detection tools to bypass website blocks, like fingerprint config, CAPTCHA solving, stealth mode, and proxy rotation (built-in 195 countries) to bypass website blocks.
- Self-developed Chromium-Driven Performance
- Auto CAPTCHA Solver: Handles complex CAPTCHAs automatically, such as reCAPTCHA v2, and Cloudflare Turnstile/Challenge for free.
- Concurrency Advantage: Unlike competitors constrained by rigid concurrency limits, Crawl offers 50 concurrent sessions as standard in its basic plan — and premium tiers unlock unlimited concurrency for ultra-fast, high-volume data acquisition.
- Cost Efficiency: Outperforms other tools on anti-crawl websites, offering free CAPTCHA resolution, with an expected 70% cost savings compared to alternative solutions.
Billing Information:
Charges are based on a hybrid pricing model that combines proxies volume and hourly rate, starting at $1.8 per GB and $0.09 per hour, the same as Browser.
Tips
- For pages involving extensive JS rendering and requiring automation operations, we recommend our Universal Scraping API. It offers a cost-effective per-page pricing model, starting at $0.20 per 1k URLs.
- For complex automation and data scraping workflows that require operating browsers through frameworks like Puppeteer or Playwright, please use Browser service.