Skip to content

feat: Add generic InfiniOps operator performance adapter - #47

Merged
Chamberlain0w0 merged 2 commits into
masterfrom
feat/cambricon-op-performance-adapter
Aug 7, 2026
Merged

feat: Add generic InfiniOps operator performance adapter#47
Chamberlain0w0 merged 2 commits into
masterfrom
feat/cambricon-op-performance-adapter

Conversation

@baominghelly

@baominghelly baominghelly commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • add a generic InfiniOps operator performance adapter for ten common operators
  • describe operators declaratively and route them through binary, cast, concat, and matrix builder families
  • move repeated JSON fields and vendor runtime string mappings into common constants
  • route Cambricon and Ascend operator cases through lazy-loaded device plugins
  • propagate accuracy and runtime-availability failures through the executor
  • report latency, tensor accuracy, estimated TFLOPS, and estimated bandwidth

Supported Operators

  • Binary: add, sub, mul, div
  • Conversion and concat: cast, cat
  • Matrix: gemm, matmul, mm, linear

Model-specific and stateful operators such as flash attention, rotary embedding, and KV cache updates are intentionally deferred. Their optional inputs, mixed dtypes, multiple outputs, and aliasing rules need separate contracts and correctness references.

Why

The source branch implemented each operator with a separate setup method and duplicated platform mappings and JSON field strings throughout the adapter. It also assumed implementation slot 8 existed whenever no native implementation was found, which can terminate the InfiniOps C++ dispatcher.

The refactored adapter uses four reusable builder families and only invokes implementation indices returned by active_implementation_indices. Missing runtime implementations become actionable per-test failures instead of process-level aborts.

Input generation and result aggregation utilities are split into PR #48 so this PR remains focused on adapter integration.

Validation

Current software validation on ffb79d8f:

  • python -m pytest -q tests/test_infiniops_adapter.py tests/test_executor_adapter_response.py tests/test_operator_flops.py: 23 passed
  • python -m pytest -q --ignore=tests/test_hardware_adapter.py --ignore=tests/test_hardware_detection.py: 23 passed
  • python -m compileall -q infinibench tests main.py: passed
  • Black 23.9.1 check for all changed Python files: passed
  • Flake8 fatal-error check for all changed Python files: passed
  • git diff --check: passed

The unfiltered test suite currently stops during collection because two tests already on master still import the renamed infinimetrics package.

Earlier Cambricon hardware smoke test (performed before the current master rebase):

  • Hardware: one Cambricon MLU590-M9C
  • Runtime: Python 3.10.8, PyTorch 2.1.0, torch-mlu 1.25.3+torch2.1.0, InfiniOps 0.1.0
  • Method: float16 small shapes, 2 warmup iterations, 5 measured iterations, full main.py -> Dispatcher -> Executor -> InfiniOpsAdapter path
  • Runtime-supported add/sub/div/mm: 12/12 passed
  • All result files reported tensor accuracy PASS and populated latency, TFLOPS, and bandwidth metrics
  • Operators without an active MLU implementation returned clean per-test failures rather than invoking an unregistered slot

The hardware result is historical smoke-test evidence, not current-head validation or a performance baseline.

Supersedes #43, which GitHub automatically closed when its head branch was renamed.

@baominghelly
baominghelly marked this pull request as ready for review August 4, 2026 07:56
@baominghelly baominghelly changed the title Add generic InfiniOps operator performance adapter feat: Add generic InfiniOps operator performance adapter Aug 4, 2026
def operation(*args):
inps = list(args[:-1])
output = args[-1]
infini.ops.cat(inps[0], inps[1:], dim, output, stream=stream)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

其他算子都通过 _pick_slot() 获取已注册实现,cat 却直接调用默认槽。这与 PR 声称“只调用 active_implementation_indices 返回的实现”不一致,在缺少原生实现时仍可能调用未注册槽。建议对 cat 同样检查并显式传递 implementation_index。

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已加入implementation_index.

out = _empty_strided(shape, strides, dtype=dtype, device=device)
out.as_strided(
(out.untyped_storage().size() // out.element_size(),), (1,)
).normal_()

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DTYPE_MAP 包含 int8/int16/int32/int64,但所有输入均调用 normal();而 PyTorch 不支持对整数张量执行正态初始化。整数 cast、二元算子会在 benchmark 前失败。建议按 dtype 使用 random_/randint,并增加整数 cast 测试。

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已改为random.

@baominghelly
baominghelly force-pushed the feat/cambricon-op-performance-adapter branch from 27b89e4 to ffb79d8 Compare August 7, 2026 03:31
@Chamberlain0w0
Chamberlain0w0 merged commit a5dbd0e into master Aug 7, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants