fork/MediaCrawler
1
0
Fork 0
mirror of https://github.com/NanmiCoder/MediaCrawler.git synced 2026-10-09 02:08:11 +08:00
Code Issues Packages Projects Releases Wiki Activity
355 Commits 11 Branches 0 Tags
ab7d8142af0bc1668f85cc8f4db12707df8d3613
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Relakkes ab7d8142af feat: weibo支持指定创作者主页 2024-08-24 05:52:11 +08:00
Relakkes 61f023edac docs: update README.md 2024-08-23 17:11:16 +08:00
Relakkes c70bd9e071 feat: 增加搜索词来源渠道 2024-08-23 08:29:24 +08:00
Relakkes d3c5111985 docs: 添加捐赠者名单 2024-08-21 19:59:35 +08:00
Relakkes 9107a8a091 docs: update README.md 2024-08-20 17:28:12 +08:00
Relakkes 04cbe549af fix: 修复抖音关键词搜索bug 2024-08-20 03:09:42 +08:00
Relakkes 2ab14920a3 docs: update README.md 2024-08-19 18:38:52 +08:00
Relakkes d39f862f85 docs: update README.md 2024-08-17 22:47:21 +08:00
Relakkes ab3e8dcc9c docs: update README.md 2024-08-13 06:44:42 +08:00
Relakkes 01ea4cd543 fix: 解决IpInfoModel循环导入依赖问题 2024-08-12 00:14:50 +08:00
Relakkes c8dd40d547 docs: update README.md 2024-08-11 23:54:36 +08:00
Relakkes e318b4c648 docs: update README.md 2024-08-11 23:48:19 +08:00
Relakkes f371675d47 fix: xhs指定笔记ID获取方式增加解析html方式，原来的由于xsec_token导致失效 2024-08-11 22:37:10 +08:00
Relakkes ec47c230a9 feat: jieba日志调整为warning级别 2024-08-09 18:12:15 +08:00
Relakkes a599916d31 fix: add parsel package 2024-08-08 18:21:33 +08:00
Relakkes e7c6c57aa6 fix: kuaishou store not implement store_creator 2024-08-08 16:36:33 +08:00
程序员阿江-Relakkes a10cdcf474 Merge pull request #374 from NanmiCoder/feature/baidu_tieba_20240805 
feat: MediaCrawler支持百度贴吧
 2024-08-08 14:24:58 +08:00
Relakkes 62ac454639 Merge branch 'main' into feature/baidu_tieba_20240805 2024-08-08 14:21:59 +08:00
Relakkes 3f42368c02 feat: 百度贴吧done 2024-08-08 14:19:32 +08:00
Relakkes df0f5c1113 feat: 百度贴吧子评论done 2024-08-07 04:13:15 +08:00
Relakkes 1208682a9a fix: 评论移除html标签内容 2024-08-07 02:39:50 +08:00
Relakkes 026d81e131 feat: 百度贴吧一级评论done 2024-08-07 02:34:56 +08:00
Relakkes 7e9a759d94 fix: 修复windows系统下初始化表结构编码问题 2024-08-07 01:02:35 +08:00
Relakkes 3c98808409 feat: 贴吧搜索重构 2024-08-07 01:01:21 +08:00
Relakkes 1b585cb215 temp commit 2024-08-06 19:21:34 +08:00
程序员阿江-Relakkes 85727da1d3 Merge pull request #370 from Jasonyang2014/wordcloud-filter-whitespace 
词云过滤空白字符
 2024-08-06 17:11:04 +08:00
Relakkes d347cf5a2c feat: 帖子搜索 & 移除登录代码使用IP代理 2024-08-06 03:37:55 +08:00
Relakkes a87094f2fd feat: 百度贴吧架子 & 登录done 2024-08-05 18:51:51 +08:00
Relakkes 1c2237a66f fix: 微博登录问题修复 
feat: 微博二级评论
 2024-08-05 00:48:42 +08:00
Relakkes 7229d29123 feat: xhs update 2024-08-04 14:54:03 +08:00
程序员阿江-Relakkes 49171a71b0 Merge pull request #361 from NanmiCoder/contributors-readme-action-KRFLV76rsz 
docs(contributor): contributors readme action update
 2024-07-29 19:32:11 +08:00
程序员阿江-Relakkes ccf71f0905 Merge pull request #360 from fruitswordman/xhs_note_id 
<bug fix> fetch_creator_notes_detail: id ->note_id
 2024-07-27 18:10:34 +08:00
Relakkes 573ca9a659 feat: xhs笔记详情更新 2024-07-25 00:44:46 +08:00
Relakkes 678b358e2e docs: update README.md 2024-07-21 03:11:18 +08:00
程序员阿江-Relakkes 6f5b5fe525 Merge pull request #348 from NanmiCoder/contributors-readme-action-pFW2NPT4l6 
docs(contributor): contributors readme action update
 2024-07-18 22:16:16 +08:00
程序员阿江-Relakkes 100b8e3496 Merge pull request #346 from Jasonyang2014/xhs-search-optimization 
小红书查询没有结果时跳出循环
 2024-07-18 22:14:47 +08:00
程序员阿江-Relakkes 3f2f03bb44 Merge pull request #345 from ZhouXsh/main 
新增B站创作者（UP主）信息爬取（合并为单个commit）
 2024-07-18 22:13:54 +08:00
Relakkes 548271e537 fix: 修复抖音中文搜索关键二次编码问题 2024-07-16 01:33:58 +08:00
程序员阿江-Relakkes 13ee7bdf95 Merge pull request #336 from helloteemo/feature/bilibli_video_download 
feat: 支持bilibili视频下载
 2024-07-15 23:05:58 +08:00
Relakkes f8096e3d58 feat: 抖音abogus参数更新 2024-07-14 03:20:05 +08:00
程序员阿江-Relakkes 0807862b63 Merge pull request #332 from NanmiCoder/contributors-readme-action-qLcNTeg9Bm 
docs(contributor): contributors readme action update
 2024-07-12 15:25:57 +08:00
程序员阿江-Relakkes e26c2aaf2a Merge pull request #331 from helloteemo/feature/xiaohognshu_get_image 
fix: 小红书下载新版本使用>3.10特性, 降低使用版本
 2024-07-12 10:16:44 +08:00
程序员阿江-Relakkes 4fde5699af Merge pull request #329 from NanmiCoder/contributors-readme-action-yUSOLe6cr3 
docs(contributor): contributors readme action update
 2024-07-11 23:47:46 +08:00
程序员阿江-Relakkes 8fe93dca23 Merge pull request #328 from helloteemo/feature/xiaohognshu_get_image 
feature: 支持小红书图片、视频下载
 2024-07-11 23:46:54 +08:00
程序员阿江-Relakkes 1a776ee968 Merge pull request #323 from helloteemo/fix/xiaohongshu_img 
feat: 解决小红书图片水印问题
 2024-07-11 21:00:10 +08:00
程序员阿江-Relakkes f45f5f75fd Merge pull request #325 from 3DRX/fix/update_config_comment 
fix: update outdated comment in base_config.py
 2024-07-11 20:58:58 +08:00
Relakkes 1264b2997c docs: update README.md 2024-06-30 01:32:30 +08:00
Relakkes cf1b309c68 docs: 增加IP代理文档link 2024-06-30 01:18:32 +08:00
Relakkes b78f694aca docs: 更新代理IP文档 2024-06-30 01:10:52 +08:00
Relakkes 4f2dcc85ff docs: update README.md 2024-06-30 00:41:37 +08:00
程序员阿江-Relakkes 7156fc4385 Update README.md 2024-06-27 22:36:11 +08:00
程序员阿江-Relakkes 65fcd5ceb0 Merge pull request #319 from Rosyrain/main 
Update WordCloud Image related operation instructions document.
 2024-06-27 22:26:27 +08:00
程序员阿江-Relakkes 86a88f7260 Merge pull request #316 from AragornJIA/main 
修复无法采集小红书 creator IP 归属信息的问题
 2024-06-26 23:04:12 +08:00
Relakkes d3eeccbaac feat: logger record current search page 2024-06-24 22:24:51 +08:00
Relakkes Yang a0e5a29af8 fix: weibo bug 2024-06-17 00:25:48 +08:00
程序员阿江-Relakkes c3d49c1019 Merge pull request #306 from NanmiCoder/contributors-readme-action-_q_mUGF7yC 
docs(contributor): contributors readme action update
 2024-06-16 21:41:09 +08:00
Relakkes 8f9540b43c docs: update README.md 2024-06-16 21:21:15 +08:00
程序员阿江-Relakkes 7baa3b5e1b Merge pull request #303 from 522109452/dev_dy_0614 
feat: base_config 增加抖音发布时间配置
 2024-06-14 23:05:26 +08:00
程序员阿江-Relakkes bea9193405 Merge pull request #301 from Hiro-Lin/main 
添加快手指定创作者主页抓取视频、评论、二级评论的功能
 2024-06-13 21:14:19 +08:00
程序员阿江-Relakkes a84e96d473 Merge pull request #299 from 522109452/fix_dy_filter 
fix: 修复抖音筛选发布时间和排序失效问题
 2024-06-13 21:11:19 +08:00
程序员阿江-Relakkes d04fc7552f Merge pull request #298 from Rosyrain/main 
Add third-party libraries and their versions for word cloud generation to the requirements.txt file.
 2024-06-13 12:50:57 +08:00
程序员阿江-Relakkes 2e367a0414 Merge pull request #294 from Rosyrain/main 
Add the functionality to generate word cloud images.
 2024-06-12 21:53:54 +08:00
程序员阿江-Relakkes 131e68334d Merge branch 'main' into main 2024-06-12 21:53:41 +08:00
程序员阿江-Relakkes 551e00c093 Merge pull request #295 from nelzomal/bili_creator 
feat: support bilibili creator
 2024-06-12 21:44:59 +08:00
程序员阿江-Relakkes f2921b935d Merge pull request #293 from nelzomal/add_more_arg 
add few more cmd args
 2024-06-12 21:41:19 +08:00
Relakkes 645ec729f6 fix: command args need default value 2024-06-11 23:55:42 +08:00
程序员阿江-Relakkes 5a2bae4c2c Merge pull request #290 from nelzomal/base_config 
improve base config reading command line arg logic
 2024-06-11 21:27:56 +08:00
Relakkes 3c7c678d7a feat: 抖音db存储增加parent_comment_id 2024-06-07 13:32:58 +08:00
程序员阿江-Relakkes c8dbc0bf3d Merge pull request #278 from ZuWard/main 
抖音二级评论
 2024-06-07 13:04:18 +08:00
Relakkes 2082e16103 docs: 群二维码过期更新 2024-06-06 22:25:46 +08:00
Relakkes 4bba1447f8 feat: cache impl done 2024-06-02 19:57:13 +08:00
Relakkes 6c4116f240 feat: abstract cache class implementation 2024-06-02 11:16:18 +08:00
Relakkes 93dbc15f0f docs: update README.md 2024-06-01 00:55:09 +08:00
Relakkes a4ceaca4e8 docs: update README.md 2024-06-01 00:51:44 +08:00
程序员阿江-Relakkes fd51a15785 Merge pull request #281 from nelzomal/improve_db_config 
feat: improve db config
 2024-06-01 00:20:05 +08:00
程序员阿江-Relakkes 5e145c31b9 Merge pull request #276 from Zzendrix/main 
fix(weibo): weibo data regular
 2024-05-28 21:37:31 +08:00
Relakkes 478db4cc4b feat: 抖音指定创作者done 2024-05-28 01:07:19 +08:00
Relakkes df1e4a7b02 refactor: 抖音登录态检测不在抛出警告，可能会误导使用者 2024-05-27 22:44:35 +08:00
Relakkes 7635ffd9e5 docs: update README.md 2024-05-26 21:25:23 +08:00
程序员阿江-Relakkes 3725128a35 Merge pull request #273 from nelzomal/bili-level-two-comment 
feat: support bilibili level two comment
 2024-05-26 15:34:28 +08:00
Relakkes 764bafc626 feat: 抖音登录态检测逻辑更新支持 2024-05-23 22:15:14 +08:00
Relakkes 3c4695eed7 docs: update README.md 2024-05-18 17:09:51 +08:00
Relakkes e64df93edd feat: 由于xhs和dy现在检测playwright二维码登录了，大概率会出现滑块或者手机验证，增加登录态检测时间为5min，预留足够的时间手动过验证码。 2024-05-15 23:23:30 +08:00
Relakkes 46f8cc064a feat: 增加爬虫程序运行前的说明 2024-05-13 21:01:54 +08:00
Relakkes 704ee8cf9d docs: update README.md 2024-05-12 23:28:50 +08:00
Relakkes 60182213fa docs: update README.md 2024-05-10 22:42:41 +08:00
程序员阿江-Relakkes 2df6ebe0d4 Merge pull request #265 from leantli/fix/value_error&typo 
fix: 捕捉处理 ValueError 报错以及修改错字
 2024-05-10 22:28:37 +08:00
Relakkes 57ee8dd63b docs: update README.md 2024-05-03 18:31:18 +08:00
Relakkes 292cceb4a6 docs: update README.md 2024-05-01 16:56:45 +08:00
程序员阿江-Relakkes 411ed2ead9 Merge pull request #253 from kexinoh/main 
新增id，解决同一天内的不同查询写入同一个文件的问题
 2024-04-30 23:10:16 +08:00
程序员阿江-Relakkes 237d5be894 Merge pull request #255 from henryhyn/feat/log_config 
feat: 配置日志打印文件及行号
 2024-04-27 13:18:13 +08:00
程序员阿江-Relakkes 2d051cc670 Merge pull request #252 from henryhyn/fix/help-f-string 
fix: 修复 f-string 双引号问题
 2024-04-26 23:58:25 +08:00
程序员阿江-Relakkes abb82076eb Merge pull request #250 from bigfa/fixed-trace-id 
fix:兼容网页端上传的图片原图获取
 2024-04-25 22:18:40 +08:00
程序员阿江-Relakkes 829974c7df Merge pull request #251 from leantli/chore/add_frequency_qa 
chore: 增加常见问题[小红书扫码登录成功后如何手动验证]
 2024-04-25 22:18:16 +08:00
Relakkes Yang e41fc05eac docs: update README.md 2024-04-22 21:25:10 +08:00
Relakkes Yang 48943a25c5 fix: add uvicorn to requirements.txt 2024-04-21 21:41:27 +08:00
Relakkes Yang 59c683a057 refactor: remove tornado server add fastapi 2024-04-21 21:11:46 +08:00
Relakkes 5d97ffac98 feat: 支持初始化mysql表结构 2024-04-18 22:34:51 +08:00
Relakkes 5681dd6925 fix: #237 2024-04-17 23:32:17 +08:00
Relakkes 41fb7bb583 fix: remove kdl secret key 2024-04-17 23:19:44 +08:00