ci(vcpkg): Repair the vcpkg CI path and move the binary cache to GitHub Packages

[bobtista]

The vcpkg builds were failing on a dead download link. ffmpeg needs an MSYS2 runtime package, and MSYS2 deletes old versions once a newer one supersedes them. We were still asking for the deleted one because builtin-baseline had not moved since March 2025, and the baseline pins the vcpkg tree that lists those package versions.

Now the binary cache is a NuGet feed on GitHub Packages keyed by vcpkg’s own ABI hashes, so there is no hand-written cache key to invalidate and no save race. Compiler tracking is back on, the baseline is the 2026.07.29 release, vcpkg.json is included in the CI path filter, the redundant custom triplet overlay is removed, and vcpkg-lock.json is removed since nothing reads it and it is not vcpkg’s lockfile format. Fork pull requests are set to read, since their token cannot upload.

The cache design was tested with a trusted workflow_dispatch: after the feed was populated, fork checks restored 10 packages in under 6 seconds and the vcpkg jobs completed in 6 to 9 minutes, against roughly 26 minutes cold.

Removing the custom triplet changed the target package ABI hashes. The latest fork run therefore rebuilt ffmpeg, stb, and zlib in read-only mode; all six vcpkg jobs passed in 21 to 28 minutes, but the fork could not upload those new packages. A trusted workflow_dispatch on bobtista/temp/vcpkg-cache-warm then populated the new entries; all 22 jobs passed. Subsequent matching builds should return to 6 to 9 minutes.

Follow-up: #3184 closes #3175 by making ffmpeg an opt-in vcpkg feature, so default cold builds no longer spend most of their time building a dependency that is not linked.

Testing:

• Cold and warm trusted workflow_dispatch runs
• Confirm fork pull requests restore from a populated feed
• Confirm the built-in-triplet cold path succeeds on all six vcpkg jobs
• Populate the feed for the built-in-triplet ABI with a trusted run

[greptile-apps]

Greptile Summary

This PR replaces the fragile file-based vcpkg binary cache (actions/cache + files backend) with a NuGet feed hosted on GitHub Packages, eliminating the GHA cache-key/ABI-hash synchronization problem that caused full rebuilds whenever the two systems drifted. It also re-enables the three win32-vcpkg* build matrix entries that had been commented out.

• build-toolchain.yml: Removes cache-key computation, actions/cache/restore, actions/cache/save, and the local cache-directory setup; replaces them with a single NuGet source authentication step (nuget sources add + setapikey using GITHUB_TOKEN) that runs only for vcpkg presets. VCPKG_BINARY_SOURCES is now set to the GitHub Packages NuGet feed at job level.
• ci.yml: Adds packages: write to the top-level permissions block and uncomments the win32-vcpkg, win32-vcpkg-profile, and win32-vcpkg-debug matrix entries in both the build-generals and build-generalsmd-win32 jobs.

Confidence Score: 5/5

Safe to merge — a well-scoped CI infrastructure change with no application code affected.

The change is a clean swap of one caching backend for another, following the canonical vcpkg + GitHub Packages NuGet pattern. The NuGet authentication step correctly fetches the nuget CLI bundled with vcpkg, gates both the setup and credential steps on contains(inputs.preset, 'vcpkg'), and uses the ephemeral GITHUB_TOKEN (masked in logs). The packages: write permission is correctly added to both the caller (ci.yml) and callee (build-toolchain.yml) workflows, and the previously-commented-out vcpkg matrix entries are re-enabled in lockstep with the new cache infrastructure.

No files require special attention; both workflow files are straightforward and internally consistent.

Important Files Changed

Filename	Overview
.github/workflows/build-toolchain.yml	Replaces file-based vcpkg binary cache with NuGet/GitHub Packages; removes cache-key computation, restore, and save steps; adds conditional NuGet source authentication; no logical issues found.
.github/workflows/ci.yml	Adds packages:write permission and re-enables the three previously-commented-out win32-vcpkg matrix entries in both build-generals and build-generalsmd-win32 jobs.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant CI as ci.yml (matrix job)
    participant BT as build-toolchain.yml
    participant vcpkg as vcpkg.exe
    participant NuGet as GitHub Packages NuGet
    participant CMake as CMake Build

    CI->>BT: workflow_call (preset, game, tools, extras)
    BT->>BT: "Set env VCPKG_BINARY_SOURCES=nuget feed"

    alt contains(preset, 'vcpkg')
        BT->>vcpkg: lukka/run-vcpkg (runVcpkgInstall: false)
        vcpkg-->>BT: sets VCPKG_ROOT
        BT->>vcpkg: fetch nuget → returns nuget.exe path
        BT->>NuGet: nuget sources add (GitHubPackages, GITHUB_TOKEN)
        BT->>NuGet: nuget setapikey (GITHUB_TOKEN)
        BT->>BT: Set VCPKG_OVERLAY_TRIPLETS, VCPKG_INSTALL_OPTIONS
    end

    BT->>CMake: cmake --preset (configure)
    CMake->>vcpkg: install dependencies
    vcpkg->>NuGet: check binary cache (ABI hash as package id)
    alt Cache hit
        NuGet-->>vcpkg: restore package (fast)
    else Cache miss
        vcpkg->>vcpkg: build from source
        vcpkg->>NuGet: upload package (push)
    end
    CMake->>CMake: cmake --build
    BT-->>CI: artifacts

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant CI as ci.yml (matrix job)
    participant BT as build-toolchain.yml
    participant vcpkg as vcpkg.exe
    participant NuGet as GitHub Packages NuGet
    participant CMake as CMake Build

    CI->>BT: workflow_call (preset, game, tools, extras)
    BT->>BT: "Set env VCPKG_BINARY_SOURCES=nuget feed"

    alt contains(preset, 'vcpkg')
        BT->>vcpkg: lukka/run-vcpkg (runVcpkgInstall: false)
        vcpkg-->>BT: sets VCPKG_ROOT
        BT->>vcpkg: fetch nuget → returns nuget.exe path
        BT->>NuGet: nuget sources add (GitHubPackages, GITHUB_TOKEN)
        BT->>NuGet: nuget setapikey (GITHUB_TOKEN)
        BT->>BT: Set VCPKG_OVERLAY_TRIPLETS, VCPKG_INSTALL_OPTIONS
    end

    BT->>CMake: cmake --preset (configure)
    CMake->>vcpkg: install dependencies
    vcpkg->>NuGet: check binary cache (ABI hash as package id)
    alt Cache hit
        NuGet-->>vcpkg: restore package (fast)
    else Cache miss
        vcpkg->>vcpkg: build from source
        vcpkg->>NuGet: upload package (push)
    end
    CMake->>CMake: cmake --build
    BT-->>CI: artifacts

Loading

_{Reviews (4): Last reviewed commit: "ci: Re-enable win32 vcpkg presets" | Re-trigger Greptile}

[bobtista]

Greptile Summary

This PR replaces the fragile actions/cache + files vcpkg binary cache with a GitHub Packages NuGet feed, eliminating the GHA cache-key / vcpkg ABI-hash synchronization doom loop described in the PR description. The structural change is sound and fork-tested.

• build-toolchain.yml: VCPKG_BINARY_SOURCES now points at the NuGet feed; a new step fetches vcpkg's bundled NuGet binary and registers the GitHub Packages source with GITHUB_TOKEN credentials; the old cache-key computation, actions/cache/restore, directory setup, and actions/cache/save steps are all removed.
• ci.yml: packages: write added so the GITHUB_TOKEN issued to calling jobs has sufficient scope to push packages to GitHub Packages (required because GitHub Actions intersects caller and callee permissions for reusable workflows).
• Minor: nuget sources add is not idempotent — a step retry or self-hosted runner reuse would cause a "Source already exists" failure. Replacing with an existence check + update fallback would make the step more resilient.

Confidence Score: 4/5

Safe to merge; the NuGet-based caching approach is sound and fork-tested, with only a minor idempotency edge case on step retries.

The structural approach is correct and well-understood (vcpkg + GitHub Packages NuGet is a documented pattern). packages:write is correctly added to both caller and callee workflows. The only concern is nuget sources add failing on retry or self-hosted runners, which is low-probability on ephemeral GitHub-hosted runners. The first post-merge run will be a cold build but subsequent runs should be fast.

.github/workflows/build-toolchain.yml — specifically the NuGet auth step around lines 119-128 for the idempotency concern.

Important Files Changed

Filename Overview
.github/workflows/build-toolchain.yml Replaces files-based vcpkg binary cache with GitHub Packages NuGet feed; adds packages:write permission, NuGet auth step via vcpkg-bundled nuget binary, and removes old cache key/restore/save steps — minor idempotency concern on nuget sources add
.github/workflows/ci.yml Adds packages:write permission to the caller workflow so the GITHUB_TOKEN can write to GitHub Packages when delegated through the reusable build-toolchain.yml

Sequence Diagram

sequenceDiagram
    participant GHA as GitHub Actions
    participant vRun as lukka/run-vcpkg
    participant vcpkg as vcpkg.exe
    participant NuGet as NuGet CLI
    participant GPkg as GitHub Packages

    GHA->>vRun: Setup vcpkg (doNotCache:true)
    vRun-->>GHA: VCPKG_ROOT set
    GHA->>vcpkg: fetch nuget
    vcpkg-->>GHA: path/to/nuget.exe
    GHA->>NuGet: sources add GitHubPackages (GITHUB_TOKEN)
    NuGet-->>GHA: source registered
    GHA->>NuGet: setapikey (GITHUB_TOKEN)
    NuGet-->>GHA: API key stored
    GHA->>vcpkg: cmake --preset (triggers vcpkg install)
    vcpkg->>GPkg: GET package@{abi-hash}
    alt cache hit
        GPkg-->>vcpkg: package restored (e.g. 28.8ms for ffmpeg)
    else cache miss
        GPkg-->>vcpkg: 404 Not Found
        vcpkg->>vcpkg: build from source
        vcpkg->>GPkg: PUT package@{abi-hash}
    end

Loading

Loading
Prompt To Fix All With AI

This is a comment left during a code review.
Path: .github/workflows/build-toolchain.yml
Line: 120-127

Comment:
**`nuget sources add` is not idempotent**

If this step is retried (e.g., via GitHub Actions "Re-run failed jobs") or runs on a self-hosted runner where the `GitHubPackages` source was previously registered, `nuget sources add` will exit with "Source already exists" and fail the step. For ephemeral GitHub-hosted runners this is fine, but wrapping it in an existence check would make the step resilient to retries:

$src = "https://nuget.pkg.github.com/${{ github.repository_owner }}/index.json"
$exists = (& $nuget sources list) -match "GitHubPackages"
if ($exists) {
& $nuget sources update -Name GitHubPackages -Source $src -StorePasswordInClearText
-UserName "${{ github.repository_owner }}" -Password "${{ secrets.GITHUB_TOKEN }}" } else { & $nuget sources add -Name GitHubPackages -Source $src
-StorePasswordInClearText -UserName "${{ github.repository_owner }}"
-Password "${{ secrets.GITHUB_TOKEN }}"
}
& $nuget setapikey "${{ secrets.GITHUB_TOKEN }}" -Source $src

How can I resolve this? If you propose a fix, please make it concise.

Reviews (1): Last reviewed commit: "ci(vcpkg): Switch binary cache from loca..." | Re-trigger Greptile

We use ephemeral GitHub-hosted runners, do we need to worry about this? Every job starts with a clean NuGet config

[xezon]

What is now the strategy with vcpkg? We still have it disabled because it caused long CI build times every now and then.

[bobtista]

What is now the strategy with vcpkg? We still have it disabled because it caused long CI build times every now and then.

The strategy is to move the dependency build output into the GitHub Packages NuGet binary cache instead of continuing with handmade caching tweaks trying to use actions/cache as a big shared folder cache. We should have used nuget from the beginning - Microsoft’s vcpkg docs specifically recommend the NuGet provider for this use case.

I tested this on my fork. The first GenCI run still took about 24 minutes for the vcpkg jobs, which I think is expected because it had to build/populate the cache. Then I ran the same workflow again on the same commit, and the vcpkg jobs dropped to about 5.5-8 minutes. The CMake configure step was only around 30-45 seconds on the second run, so ffmpeg was restored from the cache instead of rebuilt.

The workflow is still red on my fork because the replay checks need R2 secrets I don’t have set up, but all six win32-vcpkg build variants passed on both runs.

[bobtista]

Rebased onto main and updated the PR, updated the description here too - this is ready for review and merge IMHO

[bobtista]

Found the reason the downloads were failing - a retired link 😆 This can be merged now to turn vcpkg back on, the caching works, and I added a follow up PR that gates ffmpeg so it only gets built when it's being used. That's the majority of the build time on cold builds

[xezon]

Can we remove the "triplets" thing again?

[bobtista]

Can we remove the "triplets" thing again?

Yeah we can remove the custom triplet setup. vcpkg will still use its built-in x86-windows triplet, but ours only existed to disable compiler tracking, which this PR removes anyway. Cleaning that up and pushing now - it'll probably change the ABI hashes, so we’ll need another run to warm the feed again.

[bobtista]

Done - and yeah, when we merge this the cache will update

[xezon]

This branch build takes 26 minutes... will this carry over as is to every next build?

[bobtista]

This branch build takes 26 minutes... will this carry over as is to every next build?

It's set up so thesuperhackers ci can update the cache - forks are read only. So you can push a temp branch to TSH and run the ci job it'll update the cache and get back down to 6 min

[xezon]

It's set up so thesuperhackers ci can update the cache - forks are read only. So you can push a temp branch to TSH and run the ci job it'll update the cache and get back down to 6 min

So this branch is expected to be 26 minutes but main branch will make it 6 minutes later on yes?

[bobtista]

It's set up so thesuperhackers ci can update the cache - forks are read only. So you can push a temp branch to TSH and run the ci job it'll update the cache and get back down to 6 min

So this branch is expected to be 26 minutes but main branch will make it 6 minutes later on yes?

I went ahead and warmed the cache - then verified it works here with https://github.com/TheSuperHackers/GeneralsGameCode/actions/runs/32889287522/job/98324498344?pr=2542
It restored all 10 packages in 6 seconds, FFmpeg took 18ms, and the full job finished in 6 minutes

[xezon]

Ok we take your word for it. I am looking forward to the 26 minute builds in 4 weeks time.

[bobtista]

Ok we take your word for it. I am looking forward to the 26 minute builds in 4 weeks time.

I appreciate the confidence 🤣 I bet you a beer it will stay at 6 minutes 🍻