AI Scrapers
What does AI Scraper collect?
AI Scraper collects complete AI responses, including answer text, sources, citations with URLs and labels, and shopping or product cards when enabled. Results are delivered as structured JSON with parsed Markdown content. HTML output and SSE streaming are also available for supported engines and must be enabled separately.
Which AI engines can I scrape?
We support most major AI chat platforms and LLMs, including ChatGPT, Perplexity, Gemini, Microsoft Copilot, Google AI Mode, Google AI Overview, Grok, and Alexa. We are actively expanding coverage. Let us know if you need support for another platform.
How long does a task take?
Our API typically returns structured responses within 30 seconds and generally within one minute, at a speed comparable to using the AI platform directly. Responses include Markdown content and citations. AI platform load or network delays may occasionally cause timeouts.
How long are completed results available?
The quickstart specifies a retention window of 4 minutes after completion. Retrieve and save them promptly using the task_id from the creation call, or capture them through your configured webhook receiver.
Do you support asynchronous requests?
Yes! We offer an async endpoint that allows you to submit tasks and receive the results via webhook. This is ideal for high-volume scraping and ensures reliability without keeping connections open.
Can one request return results from multiple AI engines?
No. Each request targets one AI engine. To collect results from multiple engines, send a separate request for each actor. Supported parameters vary by scraper.
How many concurrent requests do you support?
Concurrency limits depend on your subscription plan. Business plans typically offer higher limits, such as 100 concurrent requests. Exceeding your limit returns an HTTP 429 status code. Use client-side queuing or our async endpoint to manage requests. Share your expected peak concurrency with us in advance so we can recommend a suitable plan and quickly increase capacity as needed.
Can I get the raw HTML?
Yes. By default, we return parsed, structured JSON to keep responses lightweight. For supported engines, set should_fetch_html: true to retrieve the full raw HTML of the result page. If an engine does not support this parameter, you can use should_fetch_share_link: true to obtain a share link as an alternative where supported.
Can I extract data from specific geographic regions?
Yes. We support countries and regions worldwide wherever the selected AI platform is available, using standard international country codes. If you need a region that the AI platform supports but that is not yet available through Scrapeless, let us know so we can configure support as soon as possible.
Why is my task still running?
Read the task status before interpreting the payload. pending and running are not completed results; success and failed are terminal states. Follow the Task Lifecycle guide for polling and response handling.
How do I track a task and receive its result?
Keep the task_id returned when you create a task. A task can be pending or running before it finishes with success or failed. Check its status using the result endpoint, or set an HTTPS webhook.url to receive the completed task automatically.
Am I billed if I never retrieve the result?
A task that completes successfully and produces a result within five minutes is billed even if your application never retrieves it. Set up periodic retries for result retrieval or a webhook to capture completed results. See Subscription & Billing for billing help.