抓取Shopee网站的数据

参数字段说明

参数	类型	值	说明
actor	string	scraper.shopee	固定填写
input.action	string	shopee.product	支持三种类型

shopee.product 商品详情数据
shopee.search 商品搜索数据
shopee.live 直播相关数据 | | input.url | string | URL链接 | 支持四种格式的URL链接
商品详情页面的 URL 链接
商品详情页的 API 链接 (/api/v4/pdp/get_pc)
商品搜索的 API 链接 (/api/v4/search/search_items)
直播的API链接（/api/v1/session/{sessionId}/more_items） |

商品详情数据

import requests
import json
 
API_KEY = ""
host = "api.scrapeless.com"
url = f"https://{host}/api/v1/scraper/request"
 
payload = json.dumps({
   "actor": "scraper.shopee",
   "input": {
      "action": "shopee.product",
      "url": "https://shopee.tw/2312312.10228173.24803858474"
   }
})
headers = {
   'Content-Type': 'application/json',
   'x-api-token': f'{API_KEY}'
}
 
response = requests.request("POST", url, headers=headers, data=payload)
 
print(response.text)

商品搜索数据

import requests
import json
 
API_KEY = ""
host = "api.scrapeless.com"
url = f"https://{host}/api/v1/scraper/request"
 
payload = json.dumps({
   "actor": "scraper.shopee",
   "input": {
      "action": "shopee.search",
      "url": "https://shopee.co.th/api/v4/search/search_items?by=sales&keyword=baby%20pants&limit=30&newest=0&order=desc&page_type=search"
   }
})
headers = {
   'Content-Type': 'application/json',
   'x-api-token': f'{API_KEY}'
}
 
response = requests.request("POST", url, headers=headers, data=payload)
 
print(response.text)

直播相关数据

import requests
import json
 
API_KEY = ""
session_id = ""
host = "api.scrapeless.com"
url = f"https://{host}/api/v1/scraper/request"
 
payload = json.dumps({
   "actor": "scraper.shopee",
   "input": {
      "action": "shopee.live",
      "url": f"https://live.shopee.co.th/api/v1/session/{session_id}/more_items?offset=0&limit=10"
   }
})
headers = {
   'Content-Type': 'application/json',
   'x-api-token': f'{API_KEY}'
}
 
response = requests.request("POST", url, headers=headers, data=payload)
 
print(response.text)

如何构建API链接

商品详情页的 API

# API可分为三个部分
# 1. region
# 2. item_id
# 3. shop_id
 
# 支持的 region 列表为
# ["shopee.co.id", "shopee.vn", "shopee.co.th", "shopee.ph", "shopee.com.my", "shopee.sg", "shopee.com.co", "shopee.cl", "shopee.com.mx", "shopee.com.br", "shopee.tw"]
 
url = f"https://{region}/api/v4/pdp/get_pc?item_id={item_id}&shop_id={shop_id}"
print(url)

商品搜索的 API

limit = 20  # 10 20 30 40
order = "desc"
page_type = "search"
keyword = "keyword"   # edit it 
region = "shopee.co.id"
 
# 支持的 region 列表为
# ["shopee.co.id", "shopee.vn", "shopee.co.th", "shopee.ph", "shopee.com.my", "shopee.sg", "shopee.com.co", "shopee.cl", "shopee.com.mx", "shopee.com.br", "shopee.tw"] 
 
url = f"https://{region}/api/v4/search/search_items?limit={limit}&newest=0&by=sales&keyword={keyword}&order={order}&page_type={page_type}&scenario=PAGE_OTHERS&version=2"
print(url)

通过TaskId获取任务结果

import requests
 
API_KEY = ""
host = "api.scrapeless.com"
task_id = ""
 
url = f"https://{host}/api/v1/scraper/result/{task_id}"
 
headers = {
   'x-api-token': f'{API_KEY}'
}
 
response = requests.request("GET", url, headers=headers)
 
print(response.text)

入门