 程序员阿江-Relakkes
|
31a092c653
|
Merge pull request #782 from NanmiCoder/fix/xhs-sign-20251127
feat: xhs sign playwright version
|
2025-11-27 11:05:24 +08:00 |
|
|
程序员阿江(Relakkes)
|
f989ce0788
|
feat: xhs sign playwright version
|
2025-11-27 10:53:08 +08:00 |
|
|
程序员阿江-Relakkes
|
15b98fa511
|
ip proxy expired logic switch
Fix/proxy 20251125
|
2025-11-26 16:05:01 +08:00 |
|
|
程序员阿江(Relakkes)
|
f1e7124654
|
fix: proxy extract error
|
2025-11-26 16:01:54 +08:00 |
|
|
程序员阿江(Relakkes)
|
6eef02d08c
|
feat: ip proxy expired check
|
2025-11-25 12:39:10 +08:00 |
|
|
程序员阿江(Relakkes)
|
1da347cbf8
|
docs: update index.md
|
2025-11-22 09:12:25 +08:00 |
|
|
程序员阿江(Relakkes)
|
422cc92dd1
|
docs: update README
|
2025-11-22 08:20:09 +08:00 |
|
|
程序员阿江(Relakkes)
|
13d2302c9c
|
docs: update README
|
2025-11-18 17:56:55 +08:00 |
|
|
程序员阿江(Relakkes)
|
ff8c92daad
|
chore: add copyright to every file
|
2025-11-18 12:24:02 +08:00 |
|
|
程序员阿江(Relakkes)
|
5288bddb42
|
refactor: weibo search #771
|
2025-11-17 17:24:47 +08:00 |
|
|
程序员阿江(Relakkes)
|
6dcfd7e0a5
|
refactor: weibo login
|
2025-11-17 17:11:35 +08:00 |
|
|
程序员阿江(Relakkes)
|
e89a6d5781
|
feat: cdp browser cleanup after crawler done
|
2025-11-17 12:21:53 +08:00 |
|
|
程序员阿江(Relakkes)
|
a1c5e07df8
|
fix: xhs sub comment bugfix #769
|
2025-11-17 11:47:33 +08:00 |
|
|
程序员阿江(Relakkes)
|
b6caa7a85e
|
refactor: add xhs creator params
|
2025-11-10 21:10:03 +08:00 |
|
|
程序员阿江(Relakkes)
|
1e3637f238
|
refactor: update xhs note detail
|
2025-11-10 18:13:51 +08:00 |
|
|
程序员阿江(Relakkes)andClaude
|
b5dab6d1e8
|
refactor: 使用 xhshow 替代 playwright 签名方案
感谢 @Cloxl/xhshow 开源项目
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
|
2025-11-10 18:12:45 +08:00 |
|
|
程序员阿江-Relakkes
|
54f23b8d1c
|
Merge pull request #768 from yangtao210/main
优化mongodb配置获取逻辑，移动存储基类位置。集成测试
|
2025-11-07 05:44:07 -05:00 |
|
|
程序员阿江(Relakkes)
|
60cbb3e37d
|
fix: weibo container error #568
|
2025-11-06 19:43:09 +08:00 |
|
|
程序员阿江-Relakkes
|
05a1782746
|
Merge pull request #764 from yangtao210/main
新增存储到mongoDB
|
2025-11-06 06:10:49 -05:00 |
|
|
程序员阿江(Relakkes)
|
45ec4b433a
|
docs: update
|
2025-11-06 00:08:03 +08:00 |
|
|
程序员阿江(Relakkes)
|
0074e975dd
|
fix: dy search
|
2025-11-04 00:14:16 +08:00 |
|
|
程序员阿江(Relakkes)
|
889fa01466
|
fix: bili词云图修复
|
2025-11-02 13:25:31 +08:00 |
|
|
程序员阿江(Relakkes)
|
3f5925e326
|
feat: update xhs sign
|
2025-10-27 19:06:07 +08:00 |
|
|
程序员阿江(Relakkes)
|
ed6e0bfb5f
|
refactor: tieba 改为浏览器获取数据
|
2025-10-19 17:09:55 +08:00 |
|
|
程序员阿江(Relakkes)
|
26a261bc09
|
Merge branch 'feature/config-refactor-20251018'
|
2025-10-19 15:32:42 +08:00 |
|
|
程序员阿江(Relakkes)
|
03e384bbe2
|
refactor: cdp模式下移除stealth注入
|
2025-10-19 15:32:03 +08:00 |
|
|
程序员阿江-Relakkes
|
56bf5d226f
|
The configuration file supports URL crawling
Feature/config refactor 20251018
|
2025-10-18 07:42:14 +08:00 |
|
|
程序员阿江(Relakkes)
|
ae7955787c
|
feat: kuaishou support url link
|
2025-10-18 07:40:10 +08:00 |
|
|
程序员阿江(Relakkes)
|
a9dd08680f
|
feat: xhs support creator url link
|
2025-10-18 07:20:09 +08:00 |
|
|
程序员阿江(Relakkes)
|
cae707cb2a
|
feat: douyin support url link
|
2025-10-18 07:00:21 +08:00 |
|
|
程序员阿江(Relakkes)
|
906c259cc7
|
feat: bilibili support url link
|
2025-10-18 06:30:20 +08:00 |
|
|
程序员阿江(Relakkes)
|
3b6fae8a62
|
docs: update README.md
|
2025-10-17 15:30:44 +08:00 |
|
|
程序员阿江-Relakkes
|
a72504a33d
|
Merge pull request #739 from callmeiks/add-tikhub-sponsor
docs: resize TikHub banner to smaller size
|
2025-10-16 16:54:18 +08:00 |
|
|
程序员阿江-Relakkes
|
1a5dcb6db7
|
Merge pull request #738 from callmeiks/add-tikhub-sponsor
docs: add TikHub as sponsor
|
2025-10-16 16:41:19 +08:00 |
|
|
程序员阿江(Relakkes)
|
d1f73e811c
|
docs: update README.md
|
2025-10-12 21:19:11 +08:00 |
|
|
程序员阿江(Relakkes)
|
2d3e7555c6
|
docs: update README.md
|
2025-10-11 16:16:11 +08:00 |
|
|
程序员阿江(Relakkes)
|
3c5b9e8035
|
docs: update wechat qrcode
|
2025-10-02 14:27:10 +08:00 |
|
|
程序员阿江(Relakkes)
|
e6f3182ed7
|
Merge branch 'codex/replace-argparse-with-typer-for-cli'
|
2025-09-26 18:11:02 +08:00 |
|
|
程序员阿江(Relakkes)
|
2cf143cc7c
|
fix: #730
|
2025-09-26 18:10:30 +08:00 |
|
|
程序员阿江-Relakkes
|
eb625b0b48
|
Merge pull request #729 from NanmiCoder/codex/replace-argparse-with-typer-for-cli
feat(cli): migrate CLI argument parsing to Typer
|
2025-09-26 18:08:21 +08:00 |
|
|
程序员阿江(Relakkes)
|
84f6f650f8
|
fix: typer args bugfix
|
2025-09-26 18:07:57 +08:00 |
|
|
程序员阿江-Relakkes
|
9d6cf065e9
|
fix(cli): support runtime without peps604
|
2025-09-26 17:38:50 +08:00 |
|
|
程序员阿江-Relakkes
|
95c740dee2
|
refine: harden typer cli defaults
|
2025-09-26 17:38:44 +08:00 |
|
|
程序员阿江-Relakkes
|
f97e0c18cd
|
feat(cli): migrate argument parsing to typer
|
2025-09-26 17:21:47 +08:00 |
|
|
程序员阿江-Relakkes
|
879a72ea30
|
fix: 修复cdp启动的浏览器无法关闭的bug
Improve BrowserLauncher shutdown reliability
|
2025-09-26 16:57:48 +08:00 |
|
|
程序员阿江-Relakkes
|
3237073a0e
|
Improve BrowserLauncher cleanup handling
|
2025-09-26 16:52:38 +08:00 |
|
|
程序员阿江-Relakkes
|
7b9db2f748
|
Merge pull request #726 from LePao1/main
feat(bilibili)：增加视频清晰度参数，可以通过`BILI_QN`更改下载的视频清晰度；
|
2025-09-25 01:58:24 +08:00 |
|
|
程序员阿江(Relakkes)
|
e2554288e0
|
docs: update README
|
2025-09-23 09:46:44 +08:00 |
|
|
程序员阿江-Relakkes
|
1342797486
|
Merge pull request #718 from persist-1/refactor
fix(store): 修复'crawler_type_var'的不当使用导致csv/json保存文件名异常的bug
|
2025-09-11 06:45:26 +08:00 |
|
|
程序员阿江-Relakkes
|
4b346cfb61
|
Merge pull request #716 from persist-1/refactor
Refactor: 使用 SQLAlchemy ORM 全面重构数据库层
|
2025-09-10 14:38:39 +08:00 |
|
|
程序员阿江-Relakkes
|
bf7a0098bd
|
Merge pull request #717 from wisty/patch-1
log client modify
|
2025-09-09 17:14:37 +08:00 |
|
|
程序员阿江(Relakkes)
|
fa5f07e9ee
|
docs: update README.md
|
2025-09-05 17:51:36 +08:00 |
|
|
程序员阿江(Relakkes)
|
6b6fedd031
|
fix: #711
|
2025-09-02 18:57:18 +08:00 |
|
|
程序员阿江(Relakkes)
|
2bce3593f7
|
feat: support time deplay for all platform
|
2025-09-02 16:43:09 +08:00 |
|
|
程序员阿江(Relakkes)
|
eb799e1fa7
|
refactor: xhs extractor
|
2025-09-02 14:50:32 +08:00 |
|
|
程序员阿江-Relakkes
|
ce52c58b98
|
Merge pull request #707 from CzsGit/fix-douyin-json-format
fix: 为抖音JSON存储添加格式化输出
|
2025-08-18 19:15:50 +08:00 |
|
|
程序员阿江(Relakkes)
|
9e8c979164
|
fix: note_download_url field length error
|
2025-08-14 14:57:24 +08:00 |
|
|
程序员阿江(Relakkes)
|
4a68e79ed0
|
docs: update README.md
|
2025-08-12 22:25:21 +08:00 |
|
|
程序员阿江-Relakkes
|
526c37822b
|
Merge pull request #700 from 2513502304/main
将捕获异常的类型由HTTPStatusError换成基类HTTPError，以便正确处理爬取媒体资源出现任何错误时，都不会导致爬取评论的中断，详情参见提交记录
|
2025-08-06 17:26:29 +08:00 |
|
|
程序员阿江-Relakkes
|
e4e0f659e0
|
Merge pull request #699 from 2513502304/main
添加了对媒体资源服务器的异常处理，参见 issue #691
|
2025-08-05 16:11:03 +08:00 |
|
|
程序员阿江(Relakkes)
|
b9d30bbabb
|
fix: #693
|
2025-08-01 15:55:21 +08:00 |
|
|
程序员阿江(Relakkes)
|
12450759d8
|
fix: httpx proxy format error
feat: add a ip proxy provider
|
2025-08-01 01:05:11 +08:00 |
|
|
程序员阿江-Relakkes
|
0024ce6ab4
|
feat: upgrade httpx version to 0.28.1
fix issue #689
|
2025-07-31 23:19:08 +08:00 |
|
|
程序员阿江-Relakkes
|
a1535289c1
|
Merge pull request #687 from 2513502304/main
添加抖音平台爬取短视频以及帖子图片的逻辑，并规范目前实现爬取媒体的抖音、b站、小红书、微博四个平台存储逻辑
|
2025-07-30 23:06:35 +08:00 |
|
|
程序员阿江-Relakkes
|
c61ed57a20
|
fix: 二维码在部分系统无法显示 #685
fix: #685
|
2025-07-30 21:26:41 +08:00 |
|
|
程序员阿江(Relakkes)
|
417c39de69
|
docs: add a sponsor
|
2025-07-30 16:44:10 +08:00 |
|
|
程序员阿江-Relakkes
|
b2d52918ae
|
Merge pull request #684 from 2513502304/main
添加抖音笔记图片的存储逻辑
|
2025-07-30 14:51:46 +08:00 |
|
|
程序员阿江(Relakkes)
|
8ab1b7ee4c
|
fix: fixed circular import issue
|
2025-07-30 14:47:11 +08:00 |
|
|
程序员阿江-Relakkes
|
72a4d194c0
|
Merge pull request #680 from korruz/main
refactor: move format_proxy_info to utils and update crawler classes
|
2025-07-29 14:21:48 +08:00 |
|
|
程序员阿江-Relakkes
|
2935841872
|
Merge pull request #676 from andy9468/main
add aiosqlite
|
2025-07-28 11:33:42 +08:00 |
|
|
程序员阿江-Relakkes
|
dbbc33a0df
|
Merge pull request #674 from persist-1/chore
chore: 增加--help参数中文显示支持；增加"douyin_aweme"表"music_download_url"字段与功能实现
|
2025-07-25 17:23:03 +08:00 |
|
|
程序员阿江(Relakkes)
|
fc06c783f5
|
fix: fixed xhs req headers
|
2025-07-23 13:28:58 +08:00 |
|
|
程序员阿江(Relakkes)
|
b41896f4f3
|
docs: add a sponsor
|
2025-07-23 13:17:19 +08:00 |
|
|
程序员阿江(Relakkes)
|
a4d9aaa34a
|
refactor: xhs update
|
2025-07-21 21:26:16 +08:00 |
|
|
程序员阿江(Relakkes)
|
26a43358cb
|
chore: update config
|
2025-07-20 14:34:56 +08:00 |
|
|
程序员阿江(Relakkes)
|
13b00f7a36
|
refactor: config update
|
2025-07-18 23:26:52 +08:00 |
|
|
程序员阿江(Relakkes)
|
122978b35c
|
Merge pull request #652 from gaoxiaobei/dev
feat(bilibili): Add flexible search modes and fix limit logic
|
2025-07-18 21:41:20 +08:00 |
|
|
程序员阿江(Relakkes)
|
2753e7631e
|
Merge pull request #664 from cfl-chenfangliang/feat-dyCommentBug
feat: 修复抖音二级评论地理位置缺失问题
|
2025-07-18 15:53:33 +08:00 |
|
|
程序员阿江(Relakkes)
|
08c28e6f7b
|
Merge pull request #658 from persist-1/feature/sqlite-support
增加对本地Sqlite数据库的支持（在不便于使用Mysql服务时也能使用数据库进行相关操作）
|
2025-07-16 20:35:58 +08:00 |
|
|
程序员阿江(Relakkes)
|
c5509ab91f
|
fix: update ip proxy valid url for #662
|
2025-07-16 11:06:58 +08:00 |
|
|
程序员阿江(Relakkes)
|
c795b1316a
|
fix: import error for #663
|
2025-07-16 10:58:11 +08:00 |
|
|
程序员阿江(Relakkes)
|
3184e0a8d9
|
fix: sign data too long error #659
|
2025-07-14 18:43:17 +08:00 |
|
|
程序员阿江(Relakkes)
|
dd8a3f5db8
|
docs: add a Sponsor
|
2025-07-12 23:26:30 +08:00 |
|
|
程序员阿江(Relakkes)
|
ec0d29cf0f
|
Merge pull request #642 from cllei12/weibo-search-type
增加选择微博搜索类型的配置
|
2025-07-07 15:05:29 +08:00 |
|
|
程序员阿江(Relakkes)
|
0d21a27b6e
|
Merge pull request #641 from cllei12/main
Update playwright version to support Ubuntu 24.04
|
2025-07-07 15:03:42 +08:00 |
|
|
程序员阿江(Relakkes)
|
3cb0e2f91f
|
Merge pull request #637 from Mirza-Samad-Ahmed-Baig/fix/bilibili-creator-videos
refactor(bilibili): process creator videos in batches
|
2025-07-05 00:13:49 +08:00 |
|
|
程序员阿江(Relakkes)
|
66a68fbb13
|
docs: update multi language badges size
|
2025-07-04 14:15:03 +08:00 |
|
|
程序员阿江(Relakkes)
|
8dcc540797
|
Merge pull request #635 from Root-FTW/main
🌐 Add multilingual documentation support (English & Spanish)
|
2025-07-04 13:54:31 +08:00 |
|
|
程序员阿江(Relakkes)
|
848df2b491
|
feat: other platfrom support the cdp mode
|
2025-07-03 17:13:32 +08:00 |
|
|
程序员阿江(Relakkes)
|
c892c3324c
|
chore: update uv lock file use aliyun pypi
|
2025-07-03 16:07:07 +08:00 |
|
|
程序员阿江(Relakkes)
|
452aafeec8
|
chore: update uv depends source
|
2025-07-03 15:27:39 +08:00 |
|
|
程序员阿江(Relakkes)
|
89359aa259
|
docs: update README.md
|
2025-07-02 01:20:44 +08:00 |
|
|
程序员阿江(Relakkes)
|
0514758cff
|
docs: 更新uv安装文档
|
2025-06-29 00:07:13 +08:00 |
|
|
程序员阿江(Relakkes)
|
e83b2422d9
|
feat: 支持playwright通过cdp协议连接本地chrome浏览器
Deploy VitePress site to Pages / build (push) Has been cancelled
Deploy VitePress site to Pages / Deploy (push) Has been cancelled
docs: 增加uv来管理python依赖的文档
|
2025-06-25 23:22:39 +08:00 |
|
|
程序员阿江(Relakkes)
|
fbc9788d54
|
docs: update README.md
Deploy VitePress site to Pages / build (push) Has been cancelled
Deploy VitePress site to Pages / Deploy (push) Has been cancelled
|
2025-06-24 17:44:43 +08:00 |
|
|
Relakkes
|
71210e3f38
|
chore: delete unnecessary file
Deploy VitePress site to Pages / build (push) Has been cancelled
Deploy VitePress site to Pages / Deploy (push) Has been cancelled
|
2025-06-20 16:26:02 +08:00 |
|
|
Relakkes
|
fc45b22963
|
fix: vitepress path problem
|
2025-06-20 16:10:44 +08:00 |
|
|
Relakkes
|
22ec34dca3
|
docs: add list of sponsors
|
2025-06-20 16:04:04 +08:00 |
|
|
Relakkes
|
fd33813f8f
|
feat: add like_count field to bilibi for issue #623
|
2025-06-20 15:50:38 +08:00 |
|
|
Relakkes
|
31bcdb191f
|
docs: update README.md
Deploy VitePress site to Pages / build (push) Has been cancelled
Deploy VitePress site to Pages / Deploy (push) Has been cancelled
|
2025-06-16 13:58:09 +08:00 |
|