Skip to content

Numba: don't materialize index broadcast - #2353

Open
ricardoV94 wants to merge 1 commit into
pymc-devs:mainfrom
ricardoV94:numba_adv_index_do_not_materialize_idx_bcast
Open

Numba: don't materialize index broadcast#2353
ricardoV94 wants to merge 1 commit into
pymc-devs:mainfrom
ricardoV94:numba_adv_index_do_not_materialize_idx_bcast

Conversation

@ricardoV94

@ricardoV94 ricardoV94 commented Aug 14, 2026

Copy link
Copy Markdown
Member

Follow up to #2345 that I missed

import numpy as np
import pytensor
import pytensor.tensor as pt

rng = np.random.default_rng(0)
x_val = rng.normal(size=(1000, 8, 5))
i_val = rng.integers(0, 1000, (2500, 1))
j_val = rng.integers(0, 8, (1, 8))
y_val = rng.normal(size=(2500, 8, 5))

x, y = pt.tensor3("x"), pt.tensor3("y")
i, j = pt.lmatrix("i"), pt.lmatrix("j")

take = pytensor.function([x, i, j], x[i, j], mode="NUMBA", trust_input=True)
add_at = pytensor.function([x, y, i, j], x[i, j].inc(y), mode="NUMBA", trust_input=True)
take(x_val, i_val, j_val), add_at(x_val, y_val, i_val, j_val)  # warm up compilation

%timeit take(x_val, i_val, j_val)
%timeit add_at(x_val, y_val, i_val, j_val)

Before:

140 μs ± 8.96 μs per loop (mean ± std. dev. of 7 runs, 10,000 loops each)
230 μs ± 6.28 μs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

After:

112 μs ± 10.3 μs per loop (mean ± std. dev. of 7 runs, 10,000 loops each)
143 μs ± 3.54 μs per loop (mean ± std. dev. of 7 runs, 10,000 loops each)

@ricardoV94 ricardoV94 changed the title numba adv index do not materialize idx bcast Numba: don't materialize index broadcast Aug 14, 2026
@ricardoV94
ricardoV94 force-pushed the numba_adv_index_do_not_materialize_idx_bcast branch from f20e893 to 9ca5e4b Compare August 14, 2026 09:07
A multi-dimensional advanced group was walked as
`enumerate(zip(idx0.ravel(), idx1.ravel(), ...))`. The indices reaching that
loop are `np.broadcast_to` views, and `ravel` of a stride-0 view cannot alias:
it materialises a contiguous copy of the full broadcast size, per index, per
call. The update array paid the same cost, having to be raveled to be reshaped.

Nest over the broadcast shape explicitly and read the views by index, which is
a stride-0 load -- no copy and no reshape.
@ricardoV94
ricardoV94 force-pushed the numba_adv_index_do_not_materialize_idx_bcast branch from 9ca5e4b to 7f1ca91 Compare August 14, 2026 09:32
@ricardoV94

Copy link
Copy Markdown
Member Author

The string codegen for AdvancedSubtensor and AdvancedIncSubtensor is pretty grotesque by now. I suggest we follow up with a rewrite ala vectorize_codegen. Unreadable strings aren't better than unreadable numba intrinsic codegen. The only advantage of this one is you can always dprint the string codegen at the end of the day I guess...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant