Skip to content

Add generic InfiniOps operator performance adapter - #43

Closed
baominghelly wants to merge 1 commit into
masterfrom
agent/cambricon-op-performance-adapter
Closed

Add generic InfiniOps operator performance adapter#43
baominghelly wants to merge 1 commit into
masterfrom
agent/cambricon-op-performance-adapter

Conversation

@baominghelly

@baominghelly baominghelly commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • add a generic InfiniOps operator performance adapter for ten common operators
  • describe operators declaratively and route them through binary, cast, concat, and matrix builder families
  • move repeated JSON fields and vendor runtime string mappings into common constants
  • route Cambricon and Ascend operator cases through lazy-loaded device plugins
  • propagate accuracy and runtime-availability failures through the executor
  • report latency, tensor accuracy, estimated TFLOPS, and estimated bandwidth

Supported Operators

  • Binary: add, sub, mul, div
  • Conversion and concat: cast, cat
  • Matrix: gemm, matmul, mm, linear

Model-specific and stateful operators such as flash attention, rotary embedding, and KV cache updates are intentionally deferred. Their optional inputs, mixed dtypes, multiple outputs, and aliasing rules need separate contracts and correctness references.

Why

The source branch implemented each operator with a separate setup method and duplicated platform mappings and JSON field strings throughout the adapter. It also assumed implementation slot 8 existed whenever no native implementation was found, which can terminate the InfiniOps C++ dispatcher.

The refactored adapter uses four reusable builder families and only invokes implementation indices returned by active_implementation_indices. Missing runtime implementations become actionable per-test failures instead of process-level aborts.

Input generation and result aggregation utilities were split into draft PR #46 so this PR remains focused on adapter integration.

Validation

Software checks:

  • python3 -m pytest -q tests/test_infiniops_adapter.py tests/test_executor_adapter_response.py tests/test_operator_flops.py: 16 passed
  • python3 -m compileall -q infinimetrics tests main.py: passed
  • Black 23.9.1 check: passed
  • Flake8 7.3.0 check: passed

Cambricon hardware smoke test:

  • Hardware: one Cambricon MLU590-M9C
  • Runtime: Python 3.10.8, PyTorch 2.1.0, torch-mlu 1.25.3+torch2.1.0, InfiniOps 0.1.0
  • Method: float16 small shapes, 2 warmup iterations, 5 measured iterations, full main.py -> Dispatcher -> Executor -> InfiniOpsAdapter path
  • Runtime-supported add/sub/div/mm: 12/12 passed
  • All result files reported tensor accuracy PASS and populated latency, TFLOPS, and bandwidth metrics
  • Operators without an active MLU implementation return clean per-test failures rather than invoking an unregistered slot

The latency observations are smoke-test evidence only, not a performance baseline.

@baominghelly
baominghelly force-pushed the agent/cambricon-op-performance-adapter branch from 0d93f27 to d131873 Compare August 3, 2026 06:58
@baominghelly baominghelly changed the title Add hardened InfiniOps operator performance adapter Add generic InfiniOps operator performance adapter Aug 3, 2026
@baominghelly
baominghelly deleted the agent/cambricon-op-performance-adapter branch August 4, 2026 07:45

Copy link
Copy Markdown
Collaborator Author

GitHub automatically closed this PR when its head branch was renamed to feat/cambricon-op-performance-adapter. Review continues in draft PR #47 with the same commit SHA and diff.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant