fork/MediaCrawler
1
0
Fork 0
mirror of https://github.com/NanmiCoder/MediaCrawler.git synced 2026-10-03 14:18:12 +08:00
Code Issues Packages Projects Releases Wiki Activity

fix issue #689，目前来看，应该是 httpx 库的问题，因为无论是使用同步还是异步版本，构不构造 httpx.***Client 对象来发起请求，返回的响应都是为空，response.content = b''，response.text = ’‘，但换成 requests 库就能正常获取数据了

Browse Source
This commit is contained in:
未来可欺
2025-07-31 22:01:48 +08:00
parent a1535289c1
commit 9d90e9fc6d
1 changed files with 2 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
media_platform/douyin/client.py
+2 -4
View File
@@ -14,7 +14,7 @@ import json
import urllib.parse import urllib.parse
from typing import Any, Callable, Dict, Union, Optional from typing import Any, Callable, Dict, Union, Optional
import httpx import requests
from playwright.async_api import BrowserContext from playwright.async_api import BrowserContext
from base.base_crawler import AbstractApiClient from base.base_crawler import AbstractApiClient
@@ -95,9 +95,7 @@ class DouYinClient(AbstractApiClient):
params["a_bogus"] = a_bogus params["a_bogus"] = a_bogus
async def request(self, method, url, **kwargs): async def request(self, method, url, **kwargs):
response = None response = requests.request(method, url, timeout=self.timeout, **kwargs)
async with httpx.AsyncClient(proxies=self.proxies) as client:
response = await client.request(method, url, timeout=self.timeout, **kwargs)
try: try:
if response.text == "" or response.text == "blocked": if response.text == "" or response.text == "blocked":
utils.logger.error(f"request params incrr, response.text: {response.text}") utils.logger.error(f"request params incrr, response.text: {response.text}")