Skip to content

Treat aten.slice.Tensor as a shared-observer op in XNNPACKQuantizer - #21568

Open
Hyungkeun-Park-Nota wants to merge 1 commit into
pytorch:mainfrom
Hyungkeun-Park-Nota:xnnpack-quantizer-slice-shared-obs
Open

Treat aten.slice.Tensor as a shared-observer op in XNNPACKQuantizer#21568
Hyungkeun-Park-Nota wants to merge 1 commit into
pytorch:mainfrom
Hyungkeun-Park-Nota:xnnpack-quantizer-slice-shared-obs

Conversation

@Hyungkeun-Park-Nota

@Hyungkeun-Park-Nota Hyungkeun-Park-Nota commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Summary

_is_share_obs_or_fq_op lists aten.slice_copy.Tensor but not aten.slice.Tensor.
The quantizer runs before to_edge(), so the op it actually sees is
aten.slice.Tensor, the _copy entry never matches, and propagate_annotation
does not give the slice its producer's SharedQuantizationSpec.

The other data-movement ops in that list are registered under both names
(permute.default/permute_copy.default, squeeze.dim/squeeze_copy.dim,
view.default/view_copy.default); slice only had the _copy one. slice has
no .default overload, which is why the pre-functionalization name is
slice.Tensor and not slice_copy.default.

Without the shared spec, the slice input and output end up with different quant
params. XNNPACK's static_slice requires them to be equal, so the model
partitions and lowers fine and then fails at runtime init:

[XNNCompiler.cpp:1421] Failed to create static slice node 377 with code: xnn_status_invalid_parameter
[XNNPACKBackend.cpp:119] XNNCompiler::compileModel failed: 0x1
[method.cpp:110] Init failed for backend XnnpackBackend: 0x1

I hit this quantizing a Gemma-family embedding model with XNNPACKQuantizer
(per-channel W8A8 static), where all 96 slices from the RoPE rotate_half
pattern were left without quant params.

_test_qs8_slice_copy was already disabled for this ("T156004676 - slice isn't
propagated"), and its comment names the same cause, so I re-enabled it here.

Test plan

pytest backends/xnnpack/test/ops/test_slice_copy.py -k qs8

The test runs end to end now: quantize, lower, serialize, run and compare
outputs. Reverting the one-line change makes it fail with 2
quantize_per_tensor nodes instead of 5.

Two assertions in it needed updating first. check_node_count takes op
overloads rather than the "aten::slice.Tensor" strings the test was written
with, so it reported Expected 3 aten::slice.Tensor nodes but found 0 on a
graph that did contain them. And the expected quantize_per_tensor count is 5,
not 3, since the three slices are now quantized along with the input and the add
output.

lintrunner -m upstream/main is clean.

cc @GregoryComer @digantdesai @cbilgin @JakeStevens

@pytorch-bot

pytorch-bot Bot commented Aug 4, 2026

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21568

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

❌ 9 Pending, 1 Unclassified Failure

As of commit 208a8e2 with merge base 4b4df96 (image):

UNCLASSIFIED FAILURE - DrCI could not classify the following job because the workflow did not run on the merge base. The failure may be pre-existing on trunk or introduced by this PR:

  • Check Labels (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 4, 2026
`_is_share_obs_or_fq_op` only listed `aten.slice_copy.Tensor`, but the quantizer
runs before `to_edge()`, so the op it sees is `aten.slice.Tensor` and the slice
never gets its producer's SharedQuantizationSpec. The neighbouring ops are all
registered under both names (permute, squeeze, view); slice only had the `_copy`
one.

Input and output then end up with different quant params, which XNNPACK's
static_slice rejects, so the model lowers fine and fails at runtime init with
xnn_status_invalid_parameter. Hit this on a Gemma-family embedding model where
the RoPE rotate_half slices were all left unquantized.

`_test_qs8_slice_copy` was disabled for this ("T156004676 - slice isn't
propagated") and is re-enabled here. Its check_node_count keys had to move to
op overloads, and the expected quantize_per_tensor count is 5 rather than 3 now
that the slices are quantized too.
@Hyungkeun-Park-Nota
Hyungkeun-Park-Nota force-pushed the xnnpack-quantizer-slice-shared-obs branch from f76d609 to 208a8e2 Compare August 4, 2026 05:21
@Hyungkeun-Park-Nota

Copy link
Copy Markdown
Contributor Author

@pytorchbot label "release notes: xnnpack" "module: xnnpack"

@pytorch-bot pytorch-bot Bot added module: xnnpack Issues related to xnnpack delegation and the code under backends/xnnpack/ release notes: xnnpack Changes to the XNNPack backend delegate labels Aug 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. module: xnnpack Issues related to xnnpack delegation and the code under backends/xnnpack/ release notes: xnnpack Changes to the XNNPack backend delegate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants