Skip to content

ci: skip the test suite on release-please Release PRs - #89

Merged
mogita merged 3 commits into
masterfrom
fix/cha-5511-skip-tests-on-release-prs
Sep 17, 2026
Merged

mogita merged 3 commits into
masterfrom
fix/cha-5511-skip-tests-on-release-prs

Conversation

@mogita

@mogita mogita commented Sep 17, 2026

Copy link
Copy Markdown
Collaborator

Ticket

CHA-5511

Problem

release-please opens a Release PR that bumps the version and rewrites the changelog. Every source commit in it already passed this suite on the PR it came from, so the full run on the Release PR only delays the release.

Solution

if: >-
  ${{ !(github.actor == 'github-actions[bot]'
  && github.event.pull_request.user.login == 'github-actions[bot]'
  && github.event.pull_request.head.repo.full_name == github.repository
  && startsWith(github.head_ref, 'release-please--')) }}

Guarded here: unit, integration-chat, integration-feed, integration-video. integration-feed already carried if: '!cancelled()', so the conditions are combined rather than replaced, or it would run alone after the chat suite skips. integration-gcp-lb is deliberately not guarded: it is a keep-alive probe running beside the other integration jobs, so skipping it shortens nothing and costs one exercise of the load balancer path it exists to keep warm

All four clauses must hold, and each one fails open, so anything short of a Release PR that release-please both opened and last pushed runs the suite. The branch name alone would not do: any PR, a fork's included, could call its branch release-please--x and skip every required check, which branch protection counts as satisfied. github.actor is the pusher rather than the PR author, so a human commit pushed onto the Release PR is tested like any other.

The pre-tag gate is unaffected: release.yml triggers on push, where there is no pull request, so the suite still runs before a tag lands.

Known trade

release-please rewrites the version file, and those bytes appear on no other PR. After this change nothing checks them until release.yml's pre-tag gate, which runs after the merge, so a broken substitution surfaces as a stuck release needing the manual recovery in the release README rather than as a red check on a PR you close. Accepted: the substitution is the only thing in a Release PR that has not already been tested, and splitting a cheap lint leg out of the combined job in five repos costs more than the failure does.

How to verify

  1. This PR is not a Release PR, so everything below should run as usual.
  2. On the next Release PR, the guarded jobs should report Skipped with no required check left pending.

A Release PR bumps the version and rewrites the changelog. Every source
commit in it already passed the suite on the PR it came from, so running
it again only delays the release.

The guard sits on each job rather than on the calling job in ci.yml: a job
skipped by `if:` reports success and satisfies a required status check,
while a reusable workflow that is never called produces no check at all and
would leave a required one pending forever.

The pre-tag gate is unaffected. release.yml triggers on push, where
github.head_ref is empty, so the suite still runs before a tag lands.
Comment thread .github/workflows/run_tests.yml Outdated
# so a skipped job still reports success to a required status check.
jobs:
unit:
if: ${{ !startsWith(github.head_ref, 'release-please--') }}

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This converts a cheap failure into the expensive one release.yml documents. Nothing broken ships: release.yml still runs the suite before tagging (on push, github.head_ref is empty, so the guard passes). But if the version-file edit breaks the build, the failure now lands after the Release PR is merged: tests goes red, release is skipped (its if: carries no status function, so a failed needs skips it), the autorelease: pending label stays set, and release-pr then stands down on needs.detect.outputs.pending == 'false', so no further Release PR is opened for that branch. Recovery is the manual path in release.yml: re-run the Release run for that SHA, or remove the label by hand. Before this change the same break was a red check on a PR you close and let release-please regenerate.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Accurate, and accepted rather than solved. The failure mode does move from a red check on a PR you close to the stuck release in release.yml: tests red, release skipped, autorelease: pending still set, and release-pr standing down on pending == 'false' so no replacement Release PR is opened.

What can still break that way is narrow: the version-file substitution itself, since every other commit in a Release PR already went through this suite. Recovery is the manual path the README documents. The alternative is splitting a cheap lint-and-build leg out of the combined job here and in four other repos, which is more config and more drift than a failure that has not yet happened once.

Worth a note in the release README if you want the recovery path closer to hand.

Comment thread .github/workflows/run_tests.yml Outdated
run: make test-integration-video

integration-gcp-lb:
if: ${{ !startsWith(github.head_ref, 'release-please--') }}

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

integration-gcp-lb is a keep-alive probe, not a gate. It runs in parallel with the other integration jobs, so skipping it shortens nothing, while each skipped Release PR is one fewer exercise of the GCP LB path, which is the job's entire purpose. Suggest leaving this one unguarded.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, unguarded. Comment in place saying it is a keep-alive probe rather than a gate, so skipping it shortens nothing and costs one exercise of the path it exists to keep warm.

The branch-name guard was spoofable: any PR, a fork's included, could name
its head branch release-please--x and skip every required check, which
branch protection then counts as satisfied. The guard now also requires the
PR to be opened by github-actions[bot] from a branch in this repository.

integration-gcp-lb stays unguarded: it is a keep-alive probe running beside
the other integration jobs, so skipping it shortens nothing and costs one
exercise of the load balancer path it exists to keep warm.
The guard keyed on who opened the PR, so a commit pushed by hand onto the
release-please branch, to fix a conflict or a changelog entry, inherited the
skip and reached the default branch having run nothing. github.actor is the
pusher rather than the PR author, so that commit is now tested like any other
and only release-please's own pushes skip.
@mogita
mogita merged commit dd3ed9f into master Sep 17, 2026
10 checks passed
@mogita
mogita deleted the fix/cha-5511-skip-tests-on-release-prs branch September 17, 2026 14:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant