feat(douyin): 重构抖音解析器使用API接口 - #584
Merged
Merged
Conversation
molanp
commented
Aug 21, 2026
Contributor
- 移除原有的HTML解析方式,改用抖音Web API进行数据获取
- 新增aweme.py文件定义API响应的数据结构模型
- 移除不再使用的video.py和slides.py文件
- 更新正则表达式匹配规则,统一处理不同类型的抖音链接
- 添加请求头配置支持OpenAPI访问
- 改进媒体内容处理逻辑,支持图片、视频和音频的正确提取
- 优化错误处理机制和异常情况的反馈信息
- resolve [Bug] 抖音分享页不再包含 videoInfoRes,导致所有抖音链接解析失败 #581
- 移除原有的HTML解析方式,改用抖音Web API进行数据获取 - 新增aweme.py文件定义API响应的数据结构模型 - 移除不再使用的video.py和slides.py文件 - 更新正则表达式匹配规则,统一处理不同类型的抖音链接 - 添加请求头配置支持OpenAPI访问 - 改进媒体内容处理逻辑,支持图片、视频和音频的正确提取 - 优化错误处理机制和异常情况的反馈信息
❌ 2 Tests Failed:
View the top 2 failed test(s) by shortest run time
To view more test analytics, go to the Test Analytics Dashboard |
Contributor
Author
|
本pr中,未设置抖音标题,因为部分抖音作品的desc可能会非常非常长,不适合设置为标题 |
Anyuluo996
added a commit
to Anyuluo996/nonebot-plugin-parser
that referenced
this pull request
Sep 7, 2026
线上 2026-09-07 实测: 签名 detail 请求被风控返回 403 时直接转 ParseException, 完全绕过 Bytespider 免签名兜底 (后者只在 200+空 body 时触发), 而 m/iesdouyin 分享页 fallback 2026-08 改版后已拿不到数据, 造成偶发整链失败 (重试 job 2 分钟后同一链接成功, 属瞬时风控)。 修复: - 签名请求 403/限流/超时不再直接抛错, 与空 body 一样进免签名兜底链 - 兜底链升级为 签名 -> open-api -> Bytespider: open-api 形态取自上游 fllesser fllesser#584 重构 ({aweme_id, aid} 极简参数 + Origin/Referer open.douyin.com, 该入口不校验 a_bogus 签名), Bytespider UA 保留次位 - 回归测试: 403→open-api 兜底成功 (mock), 三形态顺序与参数契约更新; 真实网络测试本机复现 403 并被 open-api 兜底救回 上游 fllesser#584 为整体重构 (删 slides/video 实况照片链路), 与 fork 17 个 自有提交冲突, 故取其免签名思路融入而非直接 merge。
Anyuluo996
added a commit
to Anyuluo996/nonebot-plugin-parser
that referenced
this pull request
Sep 7, 2026
清理 2026-08 改版后失效的死代码: 分享页 _ROUTER_DATA 不再含 videoInfoRes, 旧 parse_video 兜底近 7 天线上 0 成功(122 次全败), 删除 video.py 模块 + parse_video 方法 + m/iesdouyin 分享页兜底循环, video/note/slides 三种链接统一走 PC detail API。 请求链按 2026-09 实测可靠性重排: 1. open-api (主力): 极简参数 + open.douyin.com 入口, 免签名免登录, 上游 fllesser fllesser#584 同款形态; 2. 签名 (次选): a_bogus + 完整 PC web 参数, 配置了 ttwid/cookie 时附 凭据, 对无签名形态整体被封的场景兜底; 3. Bytespider (最后保险): 爬虫 UA 免签名。 slides.py 双格式 decoder 保留(抖音两种响应真实并存)。同步更新 4 个 链路契约 mock 测试到新顺序, 删除依赖分享页 mock 的失效用例。 本地: 抖音 18 用例全过(含真实网络)。
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.