fork/MediaCrawler
1
0
Fork 0
mirror of https://github.com/NanmiCoder/MediaCrawler.git synced 2026-10-07 01:48:11 +08:00
Code Issues Packages Projects Releases Wiki Activity
212 Commits 11 Branches 0 Tags
d0efd4e4f05d2f8ca83befdbe41399d0bc50a0d5
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Relakkes d0efd4e4f0 feat: 快代理实现 2024-04-17 23:12:41 +08:00
程序员阿江-Relakkes 30ae3514c3 Merge pull request #238 from keeper-jie/patch-1 
Update 常见问题.md
 2024-04-17 21:40:06 +08:00
Relakkes 9443e9d28d docs: Update README.md 2024-04-14 20:45:49 +08:00
Relakkes 87eb8aa6a7 fix: #230 2024-04-13 20:18:04 +08:00
Relakkes c26270146f fix: 修复ip代理池设计bug 2024-04-13 18:04:33 +08:00
Relakkes e367129b5f fix: 修复pr#229的小bug 2024-04-13 13:48:29 +08:00
程序员阿江-Relakkes a341dc2aff Merge pull request #229 from Tianci-King/main 
feat(core): 新增控制爬虫参数起始页面的页数start_page;perf(argparse): 向命令行解析器添加程序参数…
 2024-04-13 13:37:35 +08:00
程序员阿江-Relakkes da949f0c7a Merge pull request #226 from leantli/feat/sub_comments 
feat: 支持爬取小红书二级评论
 2024-04-13 01:17:55 +08:00
程序员阿江-Relakkes bba9841c26 Merge pull request #223 from Ermeng98/main 
新增对微博博客内照片获取的支持 文件存放路径data/weibo/images
 2024-04-11 00:11:04 +08:00
程序员阿江-Relakkes cab91474ad Merge pull request #224 from Klu5ure/main 
docs: 恢复文档中误删的字
 2024-04-09 23:14:14 +08:00
Relakkes 488acd04df fix: #222 2024-04-09 23:10:51 +08:00
Relakkes 5c409c6f0c docs: 群二维码失效，更新 2024-04-08 23:41:53 +08:00
Relakkes 8f02da73ad fix: #219 
docs: update README.md
 2024-04-08 00:19:50 +08:00
程序员阿江-Relakkes b54f168c6b Merge pull request #217 from Klu5ure/main 
docs: 修改错别字
 2024-04-07 23:11:19 +08:00
Relakkes d392747fe7 fix: 移除orm的所有内容 2024-04-06 23:51:03 +08:00
程序员阿江-Relakkes 1627ae4344 Merge pull request #216 from NanmiCoder/feature/refactor_db 
数据存储DB重构，移除tortoise-orm依赖
 2024-04-06 22:21:30 +08:00
Relakkes f9cbeb2b14 chore: 移除tortise-orm依赖 2024-04-06 22:14:03 +08:00
Relakkes 0c8484c334 feat: db数据存储重构完成 2024-04-06 22:11:10 +08:00
Relakkes de4a437dd7 style: rename image name 2024-04-06 12:23:55 +08:00
Relakkes 27f3461e31 docs: update README.md 2024-04-06 10:58:35 +08:00
Relakkes dde3c0429e refactor: IP代理池重构 2024-04-06 10:58:35 +08:00
程序员阿江-Relakkes d0c578c2bf Merge pull request #213 from Styunlen/patch-1 
chore: fix wrong log output when weibo crawler finished
 2024-04-06 00:51:36 +08:00
程序员阿江-Relakkes 5495d76fdb Merge pull request #212 from chunpat/fixbug_qrcode_show 
Remove duplication Qrcode Show
 2024-04-05 22:02:06 +08:00
程序员阿江-Relakkes 87e3d0848a Merge pull request #209 from xbsheng/chore/text 
chore: 修改错别字
 2024-04-04 18:11:42 +08:00
程序员阿江-Relakkes 6160428909 Merge pull request #202 from zhihuiio/patch-1 
Update playwright version for compiling
 2024-04-04 13:12:50 +08:00
程序员阿江-Relakkes 208978b88f Merge pull request #206 from leantli/fix/max_notes_count 
fix: 修复爬取视频/帖子最大数设置值较低导致不爬取的问题
 2024-04-04 00:46:42 +08:00
Relakkes 3de57459c1 docs: 增加交流2群二维码 2024-04-03 13:40:38 +08:00
Relakkes 85382ac7ed docs:更新许可证协议，不允许商用。 2024-04-03 12:34:34 +08:00
Relakkes e4836847cd docs: update README.md 2024-04-01 22:08:26 +08:00
程序员阿江-Relakkes 285ab7abe6 Update README.md 2024-04-01 12:54:53 +08:00
Relakkes bb732ed2ff docs: 增加MediaCrawler视频课程链接 2024-03-31 23:51:37 +08:00
Relakkes 6d8c8fb22e feat: add mysql table 2024-03-31 15:05:34 +08:00
程序员阿江-Relakkes 807af1abed Merge pull request #201 from Schofi/main 
Update README.md
 2024-03-31 14:40:11 +08:00
Relakkes e950e0d6e3 feat: add abstract api client to all platform 2024-03-30 21:27:25 +08:00
Relakkes 67ec49498a refactor: rename xhs to xiaohongshu 2024-03-30 21:17:33 +08:00
Relakkes aa257aab51 docs: update README.md 2024-03-30 14:14:10 +08:00
relakkes aca1924bd7 Merge pull request #183 from BaoZhuhan/fix-markdown 
fix ： 修复”常见问题.md“
 2024-03-19 23:13:34 +08:00
relakkes 96caede849 Merge pull request #182 from BaoZhuhan/fix-markdown 
fix: 修复了“常见问题.md"的语法问题。
 2024-03-19 22:51:45 +08:00
Relakkes 4ea9700db0 docs: 增加赞助名单 2024-03-19 21:50:25 +08:00
relakkes 3a318faee1 Update README.md 2024-03-19 13:27:58 +08:00
Relakkes 5f5147a236 docs: 添加捐赠者名单 2024-03-18 22:55:40 +08:00
Relakkes 8e8f5d41e4 docs: 添加捐赠者名单 2024-03-18 22:03:45 +08:00
relakkes ea66839d72 Update README.md 2024-03-18 13:40:36 +08:00
Relakkes 67443737f5 docs: 添加捐赠者名单 2024-03-17 22:16:58 +08:00
Relakkes 96309dcfee fix: 小红书创作者功能数据获取优化 2024-03-17 14:50:10 +08:00
Relakkes 78a9bf9f38 docs: 添加捐赠者名单 2024-03-17 01:05:10 +08:00
Relakkes 569202af78 feat: update user-agent list 2024-03-17 01:03:56 +08:00
Relakkes 59cd9f67a0 feat: 支持评论模式是否开启爬取选项 2024-03-16 11:52:42 +08:00
Relakkes 2d12ecb930 docs: 增加捐赠名单 2024-03-14 22:06:18 +08:00
Relakkes b48e4d297d docs: 将python3命令修改为python 2024-03-12 23:09:45 +08:00
Relakkes 9e2d1396b8 docs: 增加视频入门教程链接 2024-03-10 03:49:58 +08:00
Relakkes 9ca0a2b717 docs: update README.md 2024-03-09 15:37:51 +08:00
Relakkes 41fee4ff4f feat：小红书支持获取评论中的图片链接 #145 2024-03-07 22:30:44 +08:00
Relakkes 861019022a docs: 增加捐赠者名单 2024-03-05 22:45:32 +08:00
Relakkes 7606c1d993 docs: 增加捐赠者名单 2024-03-05 00:03:51 +08:00
Relakkes 149b6bcdc8 fix: 修复抖音关键词搜索为中文的情况下，有bug 2024-03-03 19:36:36 +08:00
relakkes 7002061d70 Merge pull request #142 from jayeeliu/main 
小红书支持通过博主ID采集笔记和评论
 2024-03-02 23:30:31 +08:00
Relakkes 384c8f9f7e fix: issue #140 2024-02-26 23:47:02 +08:00
relakkes 67d2b7cff8 Merge pull request #136 from jayeeliu/main 
增加sqlite配置示例，解决用sqlite保存数据时，抓取结束不退出的问题
 2024-02-22 21:32:19 +08:00
Relakkes 03ca445221 docs: update README.md 2024-02-18 13:06:12 +08:00
Relakkes 192e72588d fix: #123 2024-01-26 23:42:26 +08:00
relakkes f1d3bc6e28 Merge pull request #124 from aa65535/patch-1 
修复小红书评论重复插入
 2024-01-25 14:33:09 +08:00
relakkes eb89a6ada8 docs: 添加赞赏者信息 2024-01-21 16:42:37 +08:00
Relakkes e940a41033 refactor: 移除评论中指定数量和过滤特定关键词的逻辑 2024-01-17 23:02:05 +08:00
Relakkes e0f9a487e4 refactor: 代码优化 2024-01-16 00:40:07 +08:00
relakkes e490123fcd Merge pull request #112 from NanmiCoder/feature/stored_in_json 
refactor: 重构数据存储代码，新增jSON
 2024-01-14 22:45:00 +08:00
Relakkes 4dfa0d3fbf feat: 数据保存支持JSON格式 2024-01-14 22:40:01 +08:00
Relakkes 894dabcf63 refactor: 数据存储重构，分离不同类型的存储实现 2024-01-14 22:06:31 +08:00
Relakkes e31aebbdfb fix: 修复代理Bug 2024-01-13 15:50:02 +08:00
Relakkes 0240de7324 docs: 添加捐赠信息 2024-01-10 23:59:14 +08:00
Relakkes f85bd0c2b9 docs: 添加捐赠信息 2024-01-07 18:33:08 +08:00
Relakkes 4de14ad6a8 fix: 修复微博PC端登录后COOKIE在手机端无法使用的bug 2024-01-06 19:18:07 +08:00
Relakkes fe073801f8 feat: 小红书增加获取图片地址的方法 2024-01-04 00:11:05 +08:00
Relakkes 80ecc53cd3 fix: #107 2024-01-03 23:08:30 +08:00
Relakkes 38d6f10bf0 feat: 微博二维码登录done 2023-12-30 18:54:21 +08:00
Relakkes 27a2041929 doc: update README.md 2023-12-30 15:16:33 +08:00
Relakkes eee81622ac feat: 微博支持评论 & 指定帖子 2023-12-25 00:02:11 +08:00
Relakkes b1441ab4ae feat: 微博帖子支持保存到数据库中 2023-12-24 18:19:26 +08:00
Relakkes c5b64fdbf5 feat: 微博爬虫帖子搜索完成 2023-12-24 17:57:48 +08:00
Relakkes 9785abba5d docs: update README.md 2023-12-23 01:41:19 +08:00
Relakkes aba9f14f50 refactor: 规范日志打印 
feat: B站指定视频ID爬取（bvid）
 2023-12-23 01:04:08 +08:00
Relakkes 273c9a316b fix: 修复日志打印时参数格式错误 2023-12-22 23:10:44 +08:00
Relakkes bced7f6802 doc: update README.md 2023-12-22 23:04:08 +08:00
Relakkes 2e032d09dc refactor: xhs add log 2023-12-19 23:05:12 +08:00
Relakkes e2aaf0e5c1 doc: update README.md 2023-12-17 18:57:43 +08:00
Relakkes f85fd97d25 fix: #10 2023-12-16 00:12:20 +08:00
relakkes 40cbd9f4d7 Merge pull request #96 from PeanutSplash/main 
fix-bug:修复抖音评论筛选
 2023-12-14 00:57:17 +08:00
relakkes c4d68f868a Merge pull request #94 from PeanutSplash/main 
添加功能:(哔哩哔哩,快手,小红书)每个视频/帖子抓取评论最大条数限制,评论关键词筛选
 2023-12-14 00:01:44 +08:00
Relakkes 5c42076ff8 doc: update README.md 2023-12-09 23:50:33 +08:00
Relakkes 05016305e8 doc: update README.md 2023-12-09 23:49:19 +08:00
Relakkes 97d7a0c38b feat: Bilibili comment done 2023-12-09 21:10:01 +08:00
Relakkes a136ea2739 refactor: delete account_config.py 2023-12-09 13:56:18 +08:00
Relakkes 2d0f12f058 doc: fixed pydantic version 2023-12-09 13:31:53 +08:00
Relakkes fd640f2008 doc: update README.md 2023-12-09 00:30:31 +08:00
Relakkes d37d5ccd12 doc: add SPONSORED BY text 2023-12-09 00:28:29 +08:00
Relakkes 385662a7b6 doc: 新增代理IP使用文档 2023-12-09 00:06:57 +08:00
Relakkes a40a91d7f7 doc: update README.md 2023-12-08 00:27:59 +08:00
Relakkes 72a68fbf1e doc:将文档拆出去 2023-12-08 00:25:06 +08:00
Relakkes 1cec23f73d feat: 代理IP功能 Done 2023-12-08 00:10:04 +08:00
Relakkes c530bd4219 feat: 代理IP缓存到redis中 2023-12-06 23:49:56 +08:00