-
Notifications
You must be signed in to change notification settings - Fork 2k
docs: 新增《在昇腾 NPU 上运行 FunASR》指南 #3708
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Liuzi0816
wants to merge
3
commits into
modelscope:main
Choose a base branch
from
Liuzi0816:Liuzi134-patch-1
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,95 @@ | ||
| # 在昇腾 NPU(Ascend)上运行 FunASR | ||
|
|
||
| 适用环境:Atlas 推理/训练卡 + CANN + torch_npu。 | ||
| 已在 Atlas 300I Pro(Ascend 310P3)、CANN 9.1.0-beta.1、torch 2.10.0+cpu、torch_npu 2.10.0、funasr 1.3.14 上完整验证。 | ||
|
|
||
| ## 1. 安装注意 | ||
|
|
||
| 昇腾环境要求 torch 为 CPU 版并与 torch_npu 严格配对(如 torch 2.10.0+cpu + torch_npu 2.10.0)。 | ||
| 建议使用 `--no-deps` 安装 funasr 以避免依赖解析替换 torch,随后手动补齐运行时依赖: | ||
|
|
||
| ```bash | ||
| pip install funasr --no-deps | ||
| pip install torch_complex kaldiio omegaconf librosa kaldi-native-fbank \ | ||
| editdistance jieba zhconv tgt umap-learn praat-parselmouth \ | ||
| tensorboardX onnxruntime sentencepiece | ||
| ``` | ||
|
|
||
| ## 2. 设备指定 | ||
|
|
||
| ```python | ||
| import torch_npu # 必须先导入,注册 npu 设备 | ||
| from funasr import AutoModel | ||
|
|
||
| model = AutoModel( | ||
| model=<模型路径>, | ||
| device="npu:0", | ||
| disable_update=True, | ||
| disable_log=True, | ||
| disable_pbar=True, | ||
| ) | ||
| ``` | ||
|
|
||
| ## 3. CAM++(说话人模型)已知问题与等价规避 | ||
|
|
||
| `funasr/models/campplus/components.py` 的 `seg_pooling` 使用 | ||
| `F.avg_pool1d` / `F.max_pool1d`(kernel_size=100, stride=100, ceil_mode=True)。昇腾上该调用被 | ||
| lower 为 `AvgPoolV2`,其融合算子仅支持 stride ∈ [1,63],图编译失败并崩溃。 | ||
|
|
||
| 运行时等价替换(仅用 NPU 支持的基础算子),需在首次前向之前执行。 | ||
| 实现要点:完整段正常池化;**不完整尾段仅对其真实帧归约(显式补零会污染尾段均值/最大值, | ||
| 与 ceil_mode 语义不一致)**;未知 stype 与原版一致抛出 ValueError: | ||
|
|
||
| ```python | ||
| import torch | ||
| import torch.nn.functional as F | ||
| from funasr.models.campplus import components as cp | ||
|
|
||
| def seg_pooling(self, x, seg_len=100, stype="avg"): | ||
| # numerically equivalent to avg_pool1d/max_pool1d(kernel_size=seg_len, | ||
| # stride=seg_len, ceil_mode=True): the incomplete tail segment is | ||
| # reduced over its REAL frames only (no zero padding). | ||
| if stype not in ("avg", "max"): | ||
| raise ValueError("Wrong segment pooling type.") | ||
| B, C, T = x.shape | ||
| n_full = T // seg_len | ||
| tail = T - n_full * seg_len | ||
| segs = [] | ||
| if n_full > 0: | ||
| full = x[:, :, : n_full * seg_len].reshape(B, C, n_full, seg_len) | ||
| segs.append(full.mean(dim=-1) if stype == "avg" else full.max(dim=-1).values) | ||
| if tail > 0: | ||
| tail_x = x[:, :, n_full * seg_len:] | ||
| segs.append(tail_x.mean(dim=-1, keepdim=True) if stype == "avg" | ||
| else tail_x.max(dim=-1, keepdim=True).values) | ||
| seg = torch.cat(segs, dim=-1) if len(segs) > 1 else segs[0] # [B, C, nseg] | ||
| shape = seg.shape | ||
| seg = seg.unsqueeze(-1).expand(*shape, seg_len).reshape(*shape[:-1], -1) | ||
| return seg[..., :T] | ||
|
|
||
| cp.CAMLayer.seg_pooling = seg_pooling | ||
| ``` | ||
|
|
||
| **等价性回归**(仅 CPU 语义检查,torch 2.10.0+cpu,对照 `avg_pool1d`/`max_pool1d` + `ceil_mode=True`): | ||
| 长度 {50, 99, 100, 101, 110, 150, 199, 200, 201, 250, 1000} × {avg, max} × | ||
| {全 1、全负、randn} 共 66 组用例,`torch.allclose(rtol=1e-4, atol=1e-6)` 全部通过; | ||
| 残差仅为 float32 累加顺序噪声(最大 ~1.2e-7)。代表例:T=150 全 1 输入 avg 模式尾帧 | ||
| = 1.0(与原版一致,错误补零实现会得到 0.5)。 | ||
|
|
||
| **范围说明**:上述回归验证的是 CPU 上的归约语义等价;NPU 侧的兼容性与精度 | ||
| 需在实际昇腾环境中另行验证(本仓库初测见上文运行环境)。 | ||
|
|
||
| ## 4. 调试技巧 | ||
|
|
||
| NPU 算子异步下发,报错堆栈可能指向下一个同步点的无关模型(容易误判)。 | ||
| 定位真实出错算子: | ||
|
|
||
| ```bash | ||
| export ASCEND_LAUNCH_BLOCKING=1 # 仅调试用,会显著降速,定位后取消 | ||
| ``` | ||
|
|
||
| ## 5. 预期行为 | ||
|
|
||
| - 首次推理包含 NPU 图编译开销(几十秒量级),属正常现象,非卡死; | ||
| - fsmn-vad / paraformer / ct-transformer 可直接运行; | ||
| - cam++ 需上述 seg_pooling 补丁后运行。 | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[P2] Import torch before using the revised snippet. The shown imports bind cp and F, not torch. When the input has both a full segment and an incomplete tail (for example T=150 with seg_len=100), this branch raises NameError instead of computing the pooling result. Add import torch to the code block and check it in a fresh namespace.