批量抓取
批量抓取与常规抓取的工作方式相同，区别在于你无需提供单个 URL，而是可以一次性提供一组要抓取的 URL 列表。
import { ScrapingCrawl } from "@scrapeless-ai/sdk";
// Initialize the client
const client = new ScrapingCrawl({
apiKey: "your-api-key", // Get your API key from https://scrapeless.com
});
(async () => {
const result = await client.batchScrapeUrls(
["https://example.com", "https://scrapeless.com"],
{
formats: ["markdown", "html", "links"],
onlyMainContent: false,
timeout: 15000,
browserOptions: {
proxyCountry: "ANY",
sessionName: "Crawl",
sessionRecording: true,
sessionTTL: 900,
},
}
);
console.log(result);
})();
有关抓取端点的完整参考，请查看 API 参考。