fork/MediaCrawler
1
0
Fork 0
mirror of https://github.com/NanmiCoder/MediaCrawler.git synced 2026-10-09 00:38:12 +08:00
Code Issues Packages Projects Releases Wiki Activity
298 Commits 11 Branches 0 Tags
f45f5f75fd6a643bc0c4e2b0b03d524c1b69d028
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
程序员阿江-Relakkes f45f5f75fd Merge pull request #325 from 3DRX/fix/update_config_comment 
fix: update outdated comment in base_config.py
 2024-07-11 20:58:58 +08:00
Relakkes 1264b2997c docs: update README.md 2024-06-30 01:32:30 +08:00
Relakkes cf1b309c68 docs: 增加IP代理文档link 2024-06-30 01:18:32 +08:00
Relakkes b78f694aca docs: 更新代理IP文档 2024-06-30 01:10:52 +08:00
Relakkes 4f2dcc85ff docs: update README.md 2024-06-30 00:41:37 +08:00
程序员阿江-Relakkes 7156fc4385 Update README.md 2024-06-27 22:36:11 +08:00
程序员阿江-Relakkes 65fcd5ceb0 Merge pull request #319 from Rosyrain/main 
Update WordCloud Image related operation instructions document.
 2024-06-27 22:26:27 +08:00
程序员阿江-Relakkes 86a88f7260 Merge pull request #316 from AragornJIA/main 
修复无法采集小红书 creator IP 归属信息的问题
 2024-06-26 23:04:12 +08:00
Relakkes d3eeccbaac feat: logger record current search page 2024-06-24 22:24:51 +08:00
Relakkes Yang a0e5a29af8 fix: weibo bug 2024-06-17 00:25:48 +08:00
程序员阿江-Relakkes c3d49c1019 Merge pull request #306 from NanmiCoder/contributors-readme-action-_q_mUGF7yC 
docs(contributor): contributors readme action update
 2024-06-16 21:41:09 +08:00
Relakkes 8f9540b43c docs: update README.md 2024-06-16 21:21:15 +08:00
程序员阿江-Relakkes 7baa3b5e1b Merge pull request #303 from 522109452/dev_dy_0614 
feat: base_config 增加抖音发布时间配置
 2024-06-14 23:05:26 +08:00
程序员阿江-Relakkes bea9193405 Merge pull request #301 from Hiro-Lin/main 
添加快手指定创作者主页抓取视频、评论、二级评论的功能
 2024-06-13 21:14:19 +08:00
程序员阿江-Relakkes a84e96d473 Merge pull request #299 from 522109452/fix_dy_filter 
fix: 修复抖音筛选发布时间和排序失效问题
 2024-06-13 21:11:19 +08:00
程序员阿江-Relakkes d04fc7552f Merge pull request #298 from Rosyrain/main 
Add third-party libraries and their versions for word cloud generation to the requirements.txt file.
 2024-06-13 12:50:57 +08:00
程序员阿江-Relakkes 2e367a0414 Merge pull request #294 from Rosyrain/main 
Add the functionality to generate word cloud images.
 2024-06-12 21:53:54 +08:00
程序员阿江-Relakkes 131e68334d Merge branch 'main' into main 2024-06-12 21:53:41 +08:00
程序员阿江-Relakkes 551e00c093 Merge pull request #295 from nelzomal/bili_creator 
feat: support bilibili creator
 2024-06-12 21:44:59 +08:00
程序员阿江-Relakkes f2921b935d Merge pull request #293 from nelzomal/add_more_arg 
add few more cmd args
 2024-06-12 21:41:19 +08:00
Relakkes 645ec729f6 fix: command args need default value 2024-06-11 23:55:42 +08:00
程序员阿江-Relakkes 5a2bae4c2c Merge pull request #290 from nelzomal/base_config 
improve base config reading command line arg logic
 2024-06-11 21:27:56 +08:00
Relakkes 3c7c678d7a feat: 抖音db存储增加parent_comment_id 2024-06-07 13:32:58 +08:00
程序员阿江-Relakkes c8dbc0bf3d Merge pull request #278 from ZuWard/main 
抖音二级评论
 2024-06-07 13:04:18 +08:00
Relakkes 2082e16103 docs: 群二维码过期更新 2024-06-06 22:25:46 +08:00
Relakkes 4bba1447f8 feat: cache impl done 2024-06-02 19:57:13 +08:00
Relakkes 6c4116f240 feat: abstract cache class implementation 2024-06-02 11:16:18 +08:00
Relakkes 93dbc15f0f docs: update README.md 2024-06-01 00:55:09 +08:00
Relakkes a4ceaca4e8 docs: update README.md 2024-06-01 00:51:44 +08:00
程序员阿江-Relakkes fd51a15785 Merge pull request #281 from nelzomal/improve_db_config 
feat: improve db config
 2024-06-01 00:20:05 +08:00
程序员阿江-Relakkes 5e145c31b9 Merge pull request #276 from Zzendrix/main 
fix(weibo): weibo data regular
 2024-05-28 21:37:31 +08:00
Relakkes 478db4cc4b feat: 抖音指定创作者done 2024-05-28 01:07:19 +08:00
Relakkes df1e4a7b02 refactor: 抖音登录态检测不在抛出警告，可能会误导使用者 2024-05-27 22:44:35 +08:00
Relakkes 7635ffd9e5 docs: update README.md 2024-05-26 21:25:23 +08:00
程序员阿江-Relakkes 3725128a35 Merge pull request #273 from nelzomal/bili-level-two-comment 
feat: support bilibili level two comment
 2024-05-26 15:34:28 +08:00
Relakkes 764bafc626 feat: 抖音登录态检测逻辑更新支持 2024-05-23 22:15:14 +08:00
Relakkes 3c4695eed7 docs: update README.md 2024-05-18 17:09:51 +08:00
Relakkes e64df93edd feat: 由于xhs和dy现在检测playwright二维码登录了，大概率会出现滑块或者手机验证，增加登录态检测时间为5min，预留足够的时间手动过验证码。 2024-05-15 23:23:30 +08:00
Relakkes 46f8cc064a feat: 增加爬虫程序运行前的说明 2024-05-13 21:01:54 +08:00
Relakkes 704ee8cf9d docs: update README.md 2024-05-12 23:28:50 +08:00
Relakkes 60182213fa docs: update README.md 2024-05-10 22:42:41 +08:00
程序员阿江-Relakkes 2df6ebe0d4 Merge pull request #265 from leantli/fix/value_error&typo 
fix: 捕捉处理 ValueError 报错以及修改错字
 2024-05-10 22:28:37 +08:00
Relakkes 57ee8dd63b docs: update README.md 2024-05-03 18:31:18 +08:00
Relakkes 292cceb4a6 docs: update README.md 2024-05-01 16:56:45 +08:00
程序员阿江-Relakkes 411ed2ead9 Merge pull request #253 from kexinoh/main 
新增id，解决同一天内的不同查询写入同一个文件的问题
 2024-04-30 23:10:16 +08:00
程序员阿江-Relakkes 237d5be894 Merge pull request #255 from henryhyn/feat/log_config 
feat: 配置日志打印文件及行号
 2024-04-27 13:18:13 +08:00
程序员阿江-Relakkes 2d051cc670 Merge pull request #252 from henryhyn/fix/help-f-string 
fix: 修复 f-string 双引号问题
 2024-04-26 23:58:25 +08:00
程序员阿江-Relakkes abb82076eb Merge pull request #250 from bigfa/fixed-trace-id 
fix:兼容网页端上传的图片原图获取
 2024-04-25 22:18:40 +08:00
程序员阿江-Relakkes 829974c7df Merge pull request #251 from leantli/chore/add_frequency_qa 
chore: 增加常见问题[小红书扫码登录成功后如何手动验证]
 2024-04-25 22:18:16 +08:00
Relakkes Yang e41fc05eac docs: update README.md 2024-04-22 21:25:10 +08:00
Relakkes Yang 48943a25c5 fix: add uvicorn to requirements.txt 2024-04-21 21:41:27 +08:00
Relakkes Yang 59c683a057 refactor: remove tornado server add fastapi 2024-04-21 21:11:46 +08:00
Relakkes 5d97ffac98 feat: 支持初始化mysql表结构 2024-04-18 22:34:51 +08:00
Relakkes 5681dd6925 fix: #237 2024-04-17 23:32:17 +08:00
Relakkes 41fb7bb583 fix: remove kdl secret key 2024-04-17 23:19:44 +08:00
Relakkes 487afc8e0c refactor: 修改导报顺心 2024-04-17 23:13:40 +08:00
Relakkes d0efd4e4f0 feat: 快代理实现 2024-04-17 23:12:41 +08:00
程序员阿江-Relakkes 30ae3514c3 Merge pull request #238 from keeper-jie/patch-1 
Update 常见问题.md
 2024-04-17 21:40:06 +08:00
Relakkes 9443e9d28d docs: Update README.md 2024-04-14 20:45:49 +08:00
Relakkes 87eb8aa6a7 fix: #230 2024-04-13 20:18:04 +08:00
Relakkes c26270146f fix: 修复ip代理池设计bug 2024-04-13 18:04:33 +08:00
Relakkes e367129b5f fix: 修复pr#229的小bug 2024-04-13 13:48:29 +08:00
程序员阿江-Relakkes a341dc2aff Merge pull request #229 from Tianci-King/main 
feat(core): 新增控制爬虫参数起始页面的页数start_page;perf(argparse): 向命令行解析器添加程序参数…
 2024-04-13 13:37:35 +08:00
程序员阿江-Relakkes da949f0c7a Merge pull request #226 from leantli/feat/sub_comments 
feat: 支持爬取小红书二级评论
 2024-04-13 01:17:55 +08:00
程序员阿江-Relakkes bba9841c26 Merge pull request #223 from Ermeng98/main 
新增对微博博客内照片获取的支持 文件存放路径data/weibo/images
 2024-04-11 00:11:04 +08:00
程序员阿江-Relakkes cab91474ad Merge pull request #224 from Klu5ure/main 
docs: 恢复文档中误删的字
 2024-04-09 23:14:14 +08:00
Relakkes 488acd04df fix: #222 2024-04-09 23:10:51 +08:00
Relakkes 5c409c6f0c docs: 群二维码失效，更新 2024-04-08 23:41:53 +08:00
Relakkes 8f02da73ad fix: #219 
docs: update README.md
 2024-04-08 00:19:50 +08:00
程序员阿江-Relakkes b54f168c6b Merge pull request #217 from Klu5ure/main 
docs: 修改错别字
 2024-04-07 23:11:19 +08:00
Relakkes d392747fe7 fix: 移除orm的所有内容 2024-04-06 23:51:03 +08:00
程序员阿江-Relakkes 1627ae4344 Merge pull request #216 from NanmiCoder/feature/refactor_db 
数据存储DB重构，移除tortoise-orm依赖
 2024-04-06 22:21:30 +08:00
Relakkes f9cbeb2b14 chore: 移除tortise-orm依赖 2024-04-06 22:14:03 +08:00
Relakkes 0c8484c334 feat: db数据存储重构完成 2024-04-06 22:11:10 +08:00
Relakkes de4a437dd7 style: rename image name 2024-04-06 12:23:55 +08:00
Relakkes 27f3461e31 docs: update README.md 2024-04-06 10:58:35 +08:00
Relakkes dde3c0429e refactor: IP代理池重构 2024-04-06 10:58:35 +08:00
程序员阿江-Relakkes d0c578c2bf Merge pull request #213 from Styunlen/patch-1 
chore: fix wrong log output when weibo crawler finished
 2024-04-06 00:51:36 +08:00
程序员阿江-Relakkes 5495d76fdb Merge pull request #212 from chunpat/fixbug_qrcode_show 
Remove duplication Qrcode Show
 2024-04-05 22:02:06 +08:00
程序员阿江-Relakkes 87e3d0848a Merge pull request #209 from xbsheng/chore/text 
chore: 修改错别字
 2024-04-04 18:11:42 +08:00
程序员阿江-Relakkes 6160428909 Merge pull request #202 from zhihuiio/patch-1 
Update playwright version for compiling
 2024-04-04 13:12:50 +08:00
程序员阿江-Relakkes 208978b88f Merge pull request #206 from leantli/fix/max_notes_count 
fix: 修复爬取视频/帖子最大数设置值较低导致不爬取的问题
 2024-04-04 00:46:42 +08:00
Relakkes 3de57459c1 docs: 增加交流2群二维码 2024-04-03 13:40:38 +08:00
Relakkes 85382ac7ed docs:更新许可证协议，不允许商用。 2024-04-03 12:34:34 +08:00
Relakkes e4836847cd docs: update README.md 2024-04-01 22:08:26 +08:00
程序员阿江-Relakkes 285ab7abe6 Update README.md 2024-04-01 12:54:53 +08:00
Relakkes bb732ed2ff docs: 增加MediaCrawler视频课程链接 2024-03-31 23:51:37 +08:00
Relakkes 6d8c8fb22e feat: add mysql table 2024-03-31 15:05:34 +08:00
程序员阿江-Relakkes 807af1abed Merge pull request #201 from Schofi/main 
Update README.md
 2024-03-31 14:40:11 +08:00
Relakkes e950e0d6e3 feat: add abstract api client to all platform 2024-03-30 21:27:25 +08:00
Relakkes 67ec49498a refactor: rename xhs to xiaohongshu 2024-03-30 21:17:33 +08:00
Relakkes aa257aab51 docs: update README.md 2024-03-30 14:14:10 +08:00
relakkes aca1924bd7 Merge pull request #183 from BaoZhuhan/fix-markdown 
fix ： 修复”常见问题.md“
 2024-03-19 23:13:34 +08:00
relakkes 96caede849 Merge pull request #182 from BaoZhuhan/fix-markdown 
fix: 修复了“常见问题.md"的语法问题。
 2024-03-19 22:51:45 +08:00
Relakkes 4ea9700db0 docs: 增加赞助名单 2024-03-19 21:50:25 +08:00
relakkes 3a318faee1 Update README.md 2024-03-19 13:27:58 +08:00
Relakkes 5f5147a236 docs: 添加捐赠者名单 2024-03-18 22:55:40 +08:00
Relakkes 8e8f5d41e4 docs: 添加捐赠者名单 2024-03-18 22:03:45 +08:00
relakkes ea66839d72 Update README.md 2024-03-18 13:40:36 +08:00
Relakkes 67443737f5 docs: 添加捐赠者名单 2024-03-17 22:16:58 +08:00