feat: single apify-cli bundle + native Windows ARM64 (supersedes #1057) - #1169
feat: single apify-cli bundle + native Windows ARM64 (supersedes #1057)#1169vladfrangu wants to merge 14 commits into
Conversation
The CLI previously shipped three full bundles (apify, actor, apify-cli), dropping the same ~70MB binary three times into the install directory. Now we build one apify-cli bundle; apify and actor are tiny wrapper scripts that invoke it with APIFY_CLI_ENTRYPOINT set, which the entrypoint uses to pick the command set. - build-cli-bundles: build only apify-cli; publish apify-*/actor-* copies as backwards-compatible backups for the old upgrade flow - entrypoints: new apify-cli entrypoint + resolveEntrypoint() - install.sh / install.ps1 / dev-test-install.sh: drop the apify-cli binary and generate apify/actor (.cmd on Windows) wrapper scripts - upgrade: download the single apify-cli bundle; write it and the wrappers atomically (temp + rename) to avoid ETXTBSY on a running binary (Linux) - bundleMigration: on first run after upgrading from a legacy 3-bundle install, copy the running bundle to apify-cli and replace apify/actor with wrappers (rename-not-delete on Windows for the running exe) - useCLIVersionAssets: match only apify-cli-* assets, ignoring backups - add dev-test-install-legacy.sh to reproduce the old layout for testing
Bun now ships native Windows ARM64 builds, so drop the hack that compiled an x64 bundle and relabelled it as arm64. - build-cli-bundles: build a native bun-windows-arm64 target (full list and the win32 branch); remove the pwsh SystemType detection + APIFY_BUNDLE_ARCH override and the per-target metadata reset - install.ps1: detect the arch from PROCESSOR_ARCHITECTURE; only x64 has baseline (non-AVX2) builds, so ARM64 never downloads a -baseline bundle - install.sh: map MINGW64 ARM64 to the windows-arm64 target - useCLIVersionAssets: Windows ARM64 has a native (non-baseline) bundle, so stop requiring -baseline assets there (otherwise upgrade finds no asset) - CI (check/pre_release/release): build on windows-11-arm with native bun via setup-bun (drop the manual install workaround) and a bun-target-arch matrix Unlike #1057, asset names keep the arm64 suffix (matching the build output, useCLIMetadata's process.arch, and the asset matcher) rather than renaming to aarch64, which in #1057 mismatches the produced -arm64 artifacts.
Per Bun's supported-targets matrix (https://bun.com/docs/bundler/executables#supported-targets), baseline/modern (SIMD) variants only exist for x64, and @types/bun's CompileTarget union only accepts the SIMD-then-libc order (`bun-linux-x64-baseline-musl`). - reorder `bun-linux-x64-musl-baseline` -> `bun-linux-x64-baseline-musl` so it matches the type (no cast needed); bun accepts either order at build time - drop `bun-linux-arm64-musl-baseline`: arm64 has no baseline variant, so it was a meaningless duplicate of the arm64 musl build and nothing ever requested it - parse the target's trailing modifiers order-independently and still emit the asset suffix in the canonical `-musl-baseline` order the install/upgrade matchers expect, so every published asset name is unchanged
Baseline (non-AVX2) builds are an x64-only concept, so bun-darwin-arm64-baseline and bun-linux-arm64-baseline just duplicated their non-baseline arm64 builds. The install/upgrade flow never requests them (baseline is only ever appended for x64), and GitHub download counts confirm no real usage: across all releases the arm64 baseline assets sit at the scraper-noise floor (~205 total, max 3/release) versus darwin-arm64 at 1557 (max 260) and linux-arm64 at 262 (max 15).
There was a problem hiding this comment.
Copilot reviewed 14 out of 14 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Cuz it doesn't exist anyways Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Make the bundle the single source of truth for the apify/actor wrapper scripts, and broaden Windows shell coverage (inspired by npm's cmd-shim, which we don't need: it locates a node interpreter for script bins, whereas we exec a native binary with an env var). - install command gains a hidden --shims-only flag and now (re)writes the wrapper scripts via writeEntrypointShims(); install.sh/install.ps1/upgrade.ps1 invoke the bundle instead of hand-rolling the shim text, so it lives in one place (TS) rather than being duplicated across sh + ps1 + ts - writeEntrypointShims now writes, on Windows, a .cmd (with setlocal so APIFY_CLI_ENTRYPOINT no longer leaks into the caller's shell), a native .ps1, and an extensionless POSIX sh shim so the command also resolves from Git Bash / MSYS2 / Cygwin (previously only .cmd existed -> bare `apify` failed there) - migration reuses writeEntrypointShims and isolates the Windows legacy-.exe cleanup as its own step - install/upgrade scripts skip the automatic version check when only creating shims (local, offline operation; we just downloaded the requested version)
l2ysho
left a comment
There was a problem hiding this comment.
LGTM. Just question if we want to ship this as a part of 1.6.2. or we rather wait? cc @patrikbraborec
|
Thanks, looks cool. I would wait, and not ship it as part of 1.6.2. |
|
1.6.2 out so we can merge this cc @vladfrangu |
The install.sh, install.ps1 and upgrade.ps1 changes are moved to a stacked follow-up PR for isolated review. They depend on this PR's bundle/build, the `install --shims-only` flag, and the upgrade.ts<->upgrade.ps1 URL contract, so they are reviewed on top of this branch rather than independently.
|
Update: #1219 now targets |
The merge with master brought in the OS-keyring build logic (#1197), whose `keyringSubpackage(os, arch, Boolean(musl))` call referenced the positional `musl` variable that the single-bundle refactor had replaced with the order-independent `isMusl`. That left build-cli-bundles.ts referencing an undefined name, breaking `pnpm run build-bundles`. Pass `isMusl` instead. Verified by running build-bundles: all targets compile and the keyring native subpackage resolves per target.
The --no-git-checks workaround (from #1221) was only needed because the manual Windows ARM bun install left a stray install.ps1 in the checkout. This branch installs bun via native setup-bun, so the working tree stays clean and the check can guard the version bump again.
DaveHanns
left a comment
There was a problem hiding this comment.
Great job 🚀. Sorry it took so long to review; I have used it to learn the mechanics of CLI bundling in depth.
Mirror the build script's modifier parsing when filtering release assets: split the trailing modifiers and require both musl and baseline to match exactly. The previous positional check let a glibc machine without AVX2 match -musl-baseline too (both downloading to the same path, with the musl one winning alphabetically and failing to start on glibc), and let musl machines match -musl-baseline when only -musl was wanted. Addresses review feedback on #1169.
The entrypoint serves both the apify and actor CLIs (picked via APIFY_CLI_ENTRYPOINT), so naming it after just one of the published names was misleading next to the former apify.ts/actor.ts. The published asset prefix stays apify-cli - the build script now sets it explicitly instead of deriving it from the entrypoint file name. Addresses review feedback on #1169.
What & why
Reworks how the CLI bundles are built, installed, and upgraded, and adds native Windows ARM64 support.
Previously the bundle install shipped three full ~70 MB binaries (
apify,actor,apify-cli) — the same binary dropped three times into the install dir. This builds oneapify-clibundle;apifyandactorbecome tiny wrapper scripts that invoke it withAPIFY_CLI_ENTRYPOINTset, which the entrypoint uses to pick the command set.Note
This supersedes #1057 (native Windows ARM bundles) — those changes are included here, adapted on top of the single-bundle work. See the "Windows ARM64" section for one deliberate deviation.
Single-bundle layout
build-cli-bundles.ts): build only theapify-clientrypoint; additionally publishapify-*/actor-*copies as backwards-compatible backups so installs using the old two-bundle upgrade flow can still pull the new bundle during the transition.apify-cli.tsentrypoint +resolveEntrypoint()(readsAPIFY_CLI_ENTRYPOINT, falls back to the executable's basename).install.sh/install.ps1): drop theapify-clibinary and generateapify/actor(.cmdon Windows) wrapper scripts instead of three binaries.bundleMigration.ts): on the first run after upgrading from a legacy 3-bundle install, copy the running bundle toapify-cliand replaceapify/actorwith wrapper scripts. Writes are atomic (temp + rename) so we never truncate a running executable — an in-place overwrite fails withETXTBSYon Linux (verified in a container; this matters because theactorCLI runs in Linux Actor images). On Windows the running.exeis renamed (can't be deleted) and cleaned up on a later run.upgrade.ts): download the singleapify-clibundle, write it + the wrappers atomically.Windows ARM64 (supersedes #1057)
Bun now ships native Windows ARM64 builds, so the old hack (compile an x64 bundle on the ARM runner and relabel it
arm64) is gone.bun-windows-arm64target; drop thepwshSystemType detection +APIFY_BUNDLE_ARCHoverride.install.ps1: detect arch fromPROCESSOR_ARCHITECTURE; ARM64 never downloads a-baselinebundle (baseline is x64/AVX2-only).useCLIVersionAssets: Windows ARM64 now matches the native (non-baseline) asset — a fix chore: native windows arm bundles #1057 lacks; without it the upgrade command finds zero assets on Windows ARM64.check/pre_release/release): build onwindows-11-armwith nativesetup-bun(dropped the manual install workaround) via abun-target-archmatrix var.Deviation from #1057: asset names keep the
arm64suffix (matching the produced artifacts,useCLIMetadata'sprocess.arch, and the asset matcher) rather than renaming toaarch64, which in #1057 mismatches the-arm64artifacts the build actually emits.Bundle target cleanup
as nevercasts on the bun compile targets. Per Bun's supported-targets matrix,@types/bun'sCompileTargetonly accepts the SIMD-then-libc order, sobun-linux-x64-musl-baseline→bun-linux-x64-baseline-musl(bun accepts either order at build time). The target parser now reads the trailing modifiers order-independently and still emits the canonical-musl-baselineasset suffix, so every published asset name is unchanged.bun-{darwin,linux}-arm64-baseline(and the arm64 musl-baseline): baseline is x64-only, so these just duplicated the non-baseline arm64 builds. They're unreachable via the install/upgrade path, and download counts confirm no real usage (~205 total / max 3 per release — scraper-noise floor — vsdarwin-arm64at 1557 / max 260).Testing
pnpm run build,oxlint,oxfmt, and the local test suite all pass.apify-cli+ wrapper scripts, idempotent,actorruns in actor mode.ETXTBSYon in-place overwrite vs success with atomic rename on Linux.bun-windows-arm64and the reorderedbun-linux-x64-baseline-musltargets and appends.exefor Windows.Added
dev-test-install-legacy.shto reproduce the old 3-binary layout locally for testing the migration.closes #1221