feat(wheels): add configurable build tag hook for wheel filenames - #1273
feat(wheels): add configurable build tag hook for wheel filenames#1273jlarkin09 wants to merge 1 commit into
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds global Estimated code review effort: 3 (Moderate) | ~30 minutes 🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
src/fromager/packagesettings/_models.py (1)
48-58: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winDocstring omits an important behavioral gotcha.
wheels.get_build_tag()skips the hook entirely when the package's base build tag is empty (no changelog build-tag bump for that version). Worth documenting here so hook authors don't expect it to fire unconditionally. Also worth stating the determinism requirement mentioned in the PR description (hook must not depend on wheel contents/build env/ELF info) since nothing enforces it in code.📝 Suggested docstring addition
"""Callable that returns suffix segments for the wheel build tag. The callable receives keyword-only arguments ``ctx``, ``req``, ``version``, and ``wheel_tags`` and returns ``Sequence[str]`` of suffix segments. + Only invoked when the package already has a non-empty build tag + from its changelog entry for the given version; otherwise the hook + is skipped and no build tag is added. The callable must be + deterministic and independent of wheel contents, build environment, + or ELF metadata so fresh builds and cache lookups compute the same + tag. + .. versionadded:: 0.92.0 """🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/fromager/packagesettings/_models.py` around lines 48 - 58, Update the build_tag_hook docstring to document that wheels.get_build_tag() does not invoke the hook when the package’s base build tag is empty, and state that hook results must be deterministic and independent of wheel contents, build environment, and ELF information.tests/test_wheels.py (1)
374-440: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winMissing edge-case coverage: hook skip + call-argument verification.
No test verifies (1) the hook is not invoked when the package has no base build tag, and (2) the hook receives the correct
ctx/req/version/wheel_tagsvalues — both are explicit contract points for this feature.def test_hook_not_called_without_base_tag(self, tmp_path: pathlib.Path) -> None: """Hook is skipped entirely when the package has no changelog build tag.""" from packaging.tags import Tag calls = [] def hook(**kwargs: object) -> list[str]: calls.append(kwargs) return ["el9.6"] ctx = _ctx_with_hook(tmp_path, hook=hook) req = Requirement("mypkg") # no changelog entry -> base tag is () version = Version("1.0") tags = frozenset({Tag("cp312", "cp312", "linux_x86_64")}) result = wheels.get_build_tag(ctx=ctx, req=req, version=version, wheel_tags=tags) assert result == () assert calls == []As per path instructions, "Verify test actually tests the intended behavior. Check for missing edge cases."
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/test_wheels.py` around lines 374 - 440, Extend the get_build_tag tests to cover both contract edges: configure a hook through _ctx_with_hook for a package with no base build tag, assert the result is empty, and verify the hook is never called; also add call-argument assertions for a non-empty base-tag case, confirming the hook receives the exact ctx, req, version, and wheel_tags values used by get_build_tag.Source: Path instructions
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tests/test_wheels.py`:
- Around line 342-488: Move the repeated packaging.tags.Tag and
fromager.packagesettings Settings, SettingsFile, and WheelSettings imports to
the module-level import section of tests/test_wheels.py. Remove the
corresponding local imports from _ctx_with_hook and every TestGetBuildTag
method, preserving their existing usage.
---
Nitpick comments:
In `@src/fromager/packagesettings/_models.py`:
- Around line 48-58: Update the build_tag_hook docstring to document that
wheels.get_build_tag() does not invoke the hook when the package’s base build
tag is empty, and state that hook results must be deterministic and independent
of wheel contents, build environment, and ELF information.
In `@tests/test_wheels.py`:
- Around line 374-440: Extend the get_build_tag tests to cover both contract
edges: configure a hook through _ctx_with_hook for a package with no base build
tag, assert the result is empty, and verify the hook is never called; also add
call-argument assertions for a non-empty base-tag case, confirming the hook
receives the exact ctx, req, version, and wheel_tags values used by
get_build_tag.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: e8030f0f-912e-408f-b107-c1a722aaac6c
📒 Files selected for processing (10)
src/fromager/bootstrapper/_cache.pysrc/fromager/commands/build.pysrc/fromager/finders.pysrc/fromager/packagesettings/__init__.pysrc/fromager/packagesettings/_models.pysrc/fromager/packagesettings/_settings.pysrc/fromager/wheels.pytests/test_finders.pytests/test_packagesettings.pytests/test_wheels.py
6445684 to
5409eae
Compare
|
This pull request has merge conflicts that must be resolved before it can be merged. |
5409eae to
6f17876
Compare
|
@jlarkin09 The CI is failing here and this also needs rebase. Can you please rebase it? |
6f17876 to
c65d608
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/fromager/commands/build.py`:
- Around line 490-492: Move the wheels.get_build_tag call outside the broad
exception handler, or narrow that handler so hook exceptions and invalid hook
output propagate to the caller. Preserve cache-miss handling only for the
intended lookup failures, ensuring expected_tag computation cannot trigger an
unnecessary source build.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 23d637e9-8522-4f9e-aaa7-0ca9ea5e2542
📒 Files selected for processing (10)
src/fromager/bootstrapper/_cache.pysrc/fromager/commands/build.pysrc/fromager/finders.pysrc/fromager/packagesettings/__init__.pysrc/fromager/packagesettings/_models.pysrc/fromager/packagesettings/_settings.pysrc/fromager/wheels.pytests/test_finders.pytests/test_packagesettings.pytests/test_wheels.py
🚧 Files skipped from review as they are similar to previous changes (6)
- tests/test_finders.py
- src/fromager/finders.py
- src/fromager/packagesettings/_settings.py
- src/fromager/bootstrapper/_cache.py
- tests/test_wheels.py
- src/fromager/wheels.py
ff96cef to
fbc46b5
Compare
rd4398
left a comment
rd4398
left a comment
There was a problem hiding this comment.
I have left few review comments. I would prefer if @LalatenduMohanty takes a look as well.
Also a nit: The commit has 6 duplicate Signed-off-by / Co-authored-by trailer pairs. Can you please squash/clean up that?
| wheels: | ||
| build_tag_hook: "mypackage.hooks:build_tag_hook" | ||
|
|
||
| .. versionadded:: 0.92.0 |
There was a problem hiding this comment.
I think we tagged 0.92.0, so this needs to change to 0.93.0 everywhere
| return base_tag | ||
|
|
||
| raw = hook(ctx=ctx, req=req, version=version, wheel_tags=wheel_tags) | ||
| if isinstance(raw, str | bytes): |
There was a problem hiding this comment.
Can we add a test to verify bytes are also rejected? Currently, we only have test for str
| else: | ||
| with pytest.raises(ValueError): | ||
| wheels.validate_wheel_filename(req, version, wheel_file) | ||
|
|
There was a problem hiding this comment.
We are missing test for hook exception propagation. The whole point of moving get_build_tag() outside the broad exception handler in _is_wheel_built is so hook errors propagate. But there's no test verifying that a hook raising (e.g.) RuntimeError actually surfaces to the caller rather than being swallowed.
def test_hook_exception_propagates(self, testdata_context, ...):
def hook(**kwargs):
raise RuntimeError("hook failure")
...
with pytest.raises(RuntimeError, match="hook failure"):
wheels.get_build_tag(...)
| ) | ||
| logger.info(f"has expected build tag {expected_build_tag}") | ||
| changelogs = pbi.get_changelog(resolved_version) | ||
| logger.debug(f"has change logs {changelogs}") |
There was a problem hiding this comment.
Is there a reason why we are removing this log line?
There was a problem hiding this comment.
My mistake. Will restore the changelog debug logging.
| wheel_filename = wheels.download_wheel(req, url, wkctx.wheels_downloads) | ||
|
|
||
| return wheel_filename | ||
| _, _, build_tag_from_name, wheel_tags = parse_wheel_filename(wheel_basename) |
There was a problem hiding this comment.
url, wheel_basename, build_tag_from_name, and wheel_tags are assigned inside the try block and used after the except. The except block returns None, so the code is technically correct, but the flow is fragile. An else clause on the try would make the intent explicit and survive future refactoring more safely:
try:
...
_, _, build_tag_from_name, wheel_tags = parse_wheel_filename(wheel_basename)
except Exception:
...
return None
else:
# build tag comparison lives here
expected_tag = wheels.get_build_tag(...)
...
| req.name.replace("-", "."), | ||
| ] | ||
|
|
||
| candidate_bases: list[str] = [] |
There was a problem hiding this comment.
Can we continue using set() here? The original code used a set to deduplicate candidate bases. The new code uses a list built with nested loops. If, say,filename_prefix == canonical_name (common for simple package names), the same prefix is searched twice. This is a minor performance concern
| return None | ||
|
|
||
|
|
||
| def find_wheel( |
There was a problem hiding this comment.
The new logic adds a "{build_tag[0]}_" prefix when build_tag[1] is empty, so that a base tag (2, "") can match suffixed wheels like 2_el9.6-.... But find_wheel returns the first glob match and stops. If the downloads directory contains multiple suffixed wheels (e.g., 2_el9.6-... and 2_el9.6_rocm7.1-...), the function may return the wrong one. The caller in _look_for_existing_wheel then rejects it on exact-tag mismatch and returns (None, None) — the correct wheel is never tried.
This creates a chicken-and-egg: you need wheel_tags from a found wheel to compute the expected suffix (via the hook), but you need the suffix to find the right wheel. The current loose-find + exact-check approach only works reliably when there's at most one candidate. Can we make find_wheel() return multiple values? cc @LalatenduMohanty for his thoughts
Implement the accepted proposal from docs/proposals/wheel-build-tag-hook.md (issue python-wheel-build#1059, tracking issue python-wheel-build#1181). Add a wheels.build_tag_hook option in global settings that lets downstream projects append environment-specific suffixes (OS, accelerator, torch ABI) to wheel build tags via a user-defined callable. The hook receives ctx, req, version, and wheel_tags and returns suffix segments joined with _. - Add WheelSettings model with build_tag_hook: ImportString to settings - Add get_build_tag() and _validate_build_tag_segments() to wheels.py - Update add_extra_metadata_to_wheels(), bootstrapper cache checks, and _is_wheel_built() to use computed build tags - Minimal finder update to match suffixed build tag filenames - Validate hook output: reject single strings, bytes, invalid chars - No behavior change when hook is not configured Closes: python-wheel-build#1181 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Justin Larkin <jlarkin@redhat.com> Co-authored-by: Cursor <cursoragent@cursor.com>
fbc46b5 to
7c17832
Compare
2 participants
Implement the accepted proposal from docs/proposals/wheel-build-tag-hook.md (issue #1059, tracking issue #1181).
Add a
wheels.build_tag_hookoption in global settings that lets downstream projects append environment-specific suffixes (OS, accelerator, torch ABI) to wheel build tags via a user-defined callable. The hook receives ctx, req, version, and wheel_tags and returns suffix segments joined with_.WheelSettingsmodel withbuild_tag_hook: ImportStringto settingsget_build_tag()and_validate_build_tag_segments()to wheels.pyadd_extra_metadata_to_wheels(), bootstrapper cache checks, and_is_wheel_built()to use computed build tagsContinues on PR: #1217
Closes: #1181