Skip to content

Add stage override variable to shiftstack role - #4072

Merged
openshift-merge-bot[bot] merged 1 commit into
openstack-k8s-operators:mainfrom
tusharjadhav3302:OSPRH-33128-shiftstack-stages-override
Aug 11, 2026
Merged

Add stage override variable to shiftstack role#4072
openshift-merge-bot[bot] merged 1 commit into
openstack-k8s-operators:mainfrom
tusharjadhav3302:OSPRH-33128-shiftstack-stages-override

Conversation

@tusharjadhav3302

@tusharjadhav3302 tusharjadhav3302 commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

The shiftstack role invokes ocp_testing.yaml with a testconfig that bundles all stages (install + test) in a single ansible-navigator call. The upcoming phased pipeline needs to run install stages in one Zuul job and test stages in another, using the same testconfig file.

Add cifmw_shiftstack_stages_override (default: []) that appends --extra-vars to the ansible-navigator command when non-empty, overriding the stages list from the testconfig. This avoids creating per-phase testconfig files in shiftstack-qa and keeps the monolithic job unchanged (empty list = no override = all stages run as before).

Related-Issue: OSPRH-33128

@openshift-ci

openshift-ci Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Hi @tusharjadhav3302. Thanks for your PR.

I'm waiting for a openstack-k8s-operators member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/88db85dc71f04addbfafd2d0db930519

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 42m 01s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 39m 34s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 49m 32s
✔️ cifmw-crc-podified-edpm-baremetal-minor-update SUCCESS in 2h 21m 55s
✔️ cifmw-pod-zuul-files SUCCESS in 5m 43s
✔️ openstack-k8s-operators-content-provider-bootc SUCCESS in 42m 46s
cifmw-crc-podified-edpm-baremetal-bootc NODE_FAILURE Node(set) request 099-0000151063 failed in 0s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 9m 06s
✔️ cifmw-pod-pre-commit SUCCESS in 9m 54s
✔️ cifmw-molecule-shiftstack SUCCESS in 20m 32s

@imatza-rh

Copy link
Copy Markdown
Contributor

/ok-to-test

@tusharjadhav3302

Copy link
Copy Markdown
Contributor Author

/retest

@tusharjadhav3302

Copy link
Copy Markdown
Contributor Author

/retest

@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/3d9c55c6673640f993a849534f54d918

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 31m 43s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 33m 21s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 41m 30s
✔️ cifmw-crc-podified-edpm-baremetal-minor-update SUCCESS in 2h 19m 10s
✔️ cifmw-pod-zuul-files SUCCESS in 4m 32s
✔️ openstack-k8s-operators-content-provider-bootc SUCCESS in 2h 20m 57s
✔️ cifmw-crc-podified-edpm-baremetal-bootc SUCCESS in 1h 50m 43s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 55s
✔️ cifmw-pod-pre-commit SUCCESS in 8m 32s
cifmw-molecule-shiftstack FAILURE in 24m 31s

@tusharjadhav3302

Copy link
Copy Markdown
Contributor Author

Job: cifmw-molecule-shiftstack
Molecule converge got through most of the shiftstack role, then died here:

TASK [shiftstack : Install the ansible collections]
cd shiftstack-qa && ansible-galaxy collection install -f -r requirements.yaml
ERROR! Failed to download collection tar from 'default' ...
<urlopen error [Errno 101] Network is unreachable>

It was downloading openstack-cloud-2.2.0 from galaxy.ansible.com inside the shiftstackclient pod and the node lost outbound network.

@tusharjadhav3302
tusharjadhav3302 force-pushed the OSPRH-33128-shiftstack-stages-override branch from 5ec6e8b to e45958c Compare August 5, 2026 09:02
@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/2478e4bb849b4da9adadda9f15845e1c

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 13m 00s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 44m 42s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 50m 40s
cifmw-crc-podified-edpm-baremetal-minor-update FAILURE in 36m 45s
✔️ cifmw-pod-zuul-files SUCCESS in 5m 52s
✔️ openstack-k8s-operators-content-provider-bootc SUCCESS in 2h 24m 14s
✔️ cifmw-crc-podified-edpm-baremetal-bootc SUCCESS in 1h 48m 23s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 9m 41s
✔️ cifmw-pod-pre-commit SUCCESS in 10m 05s
✔️ cifmw-molecule-shiftstack SUCCESS in 20m 12s

@tusharjadhav3302

Copy link
Copy Markdown
Contributor Author

recheck

evallesp
evallesp previously approved these changes Aug 7, 2026

@evallesp evallesp left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@evallesp

evallesp commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

/ok-to-test

@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/938fde90952b443a9ea7f0f675f780de

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 33m 32s
podified-multinode-edpm-deployment-crc FAILURE in 1h 49m 05s
cifmw-crc-podified-edpm-baremetal FAILURE in 1h 55m 11s
✔️ cifmw-crc-podified-edpm-baremetal-minor-update SUCCESS in 2h 20m 11s
✔️ cifmw-pod-zuul-files SUCCESS in 6m 15s
✔️ openstack-k8s-operators-content-provider-bootc SUCCESS in 2h 46m 12s
cifmw-crc-podified-edpm-baremetal-bootc FAILURE in 2h 13m 07s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 10m 02s
✔️ cifmw-pod-pre-commit SUCCESS in 8m 41s
✔️ cifmw-molecule-shiftstack SUCCESS in 27m 13s

@tusharjadhav3302

Copy link
Copy Markdown
Contributor Author

recheck

@imatza-rh imatza-rh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see the Commit message prefix: the verify-prefix check requires [shiftstack] at the start (e.g. [shiftstack] Add stages override for phased CI)

{% if cifmw_shiftstack_stages_override | default([]) | length > 0 %}-e "{\"stages\":{{ cifmw_shiftstack_stages_override | to_json | replace(' ', '') }}}}"{% endif %}
ansible.builtin.include_tasks: exec_command_in_pod.yml

rescue:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

}}}} renders invalid JSON when cifmw_shiftstack_stages_override is non-empty - produces {"stages":[...]}} (extra }). Fix: change }}}} to }}}.

Verified:

from jinja2 import Environment
t = Environment().from_string(r'{"stages":{{ v | tojson }}}}')
print(t.render(v=["install"]))  # extra }

The molecule test doesn't catch this because the default [] skips the {% if %} block.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — dropped the extra } so a non-empty override renders valid JSON ({"stages":[...]}). Also rebased onto latest main, squashed to a single commit, and updated the subject to [shiftstack] Add stages override for phased CI for the verify-prefix check.

See df54b4df.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified fixed in df54b4d.

@tusharjadhav3302

Copy link
Copy Markdown
Contributor Author

Addressed both review notes from @imatza-rh:

  1. Removed the extra } in test_config.yml so non-empty cifmw_shiftstack_stages_override produces valid JSON.
  2. Rewrote the commit subject with the required [shiftstack] prefix.

Branch is rebased on latest main and kept as a single commit (df54b4df).

@imatza-rh

Copy link
Copy Markdown
Contributor

Both review points addressed — commit prefix fixed and Jinja2 brace count corrected. CI green. /lgtm

Two minor suggestions (neither blocking):

  1. Consider adding a cifmw_shiftstack_stages_override entry to the README Parameters section when convenient (4 other vars are also undocumented, so not urgent).

  2. The replace(' ', '') after to_json is cosmetic — to_json already produces valid JSON. Could be dropped to simplify the template.

@tusharjadhav3302

Copy link
Copy Markdown
Contributor Author

Thanks @imatza-rh — applied both suggestions:

  1. Documented cifmw_shiftstack_stages_override in the README Parameters section.
  2. Dropped the cosmetic | replace(' ', '') after to_json.

Kept as a single commit on the branch.

@tusharjadhav3302
tusharjadhav3302 force-pushed the OSPRH-33128-shiftstack-stages-override branch from df54b4d to 5a455f3 Compare August 10, 2026 09:17
@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/c6dc19c7f3af453ba7b82a0f3e32cf83

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 27m 44s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 30m 20s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 36m 57s
✔️ cifmw-crc-podified-edpm-baremetal-minor-update SUCCESS in 2h 15m 00s
✔️ cifmw-pod-zuul-files SUCCESS in 4m 54s
✔️ openstack-k8s-operators-content-provider-bootc SUCCESS in 2h 22m 24s
✔️ cifmw-crc-podified-edpm-baremetal-bootc SUCCESS in 1h 38m 51s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 11m 02s
✔️ cifmw-pod-pre-commit SUCCESS in 10m 57s
cifmw-molecule-shiftstack FAILURE in 20m 11s

@tusharjadhav3302

Copy link
Copy Markdown
Contributor Author

cifmw-molecule-shiftstack failed again, but this is a different flake from the earlier Galaxy network issue.

Galaxy install succeeded this run (~18s). Converge then failed mid-test on a trivial oc rsh check (test -f shiftstack-qa/jobs_definitions/cifmw-gate.yaml) with:

The connection to the server api.crc.testing:6443 was refused

CRC had already started successfully and the shiftstackclient pod was usable (clone + galaxy install worked), so the API becoming unreachable afterward looks like CRC/infra instability on the molecule node — not related to this PR (cifmw_shiftstack_stages_override is unused unless explicitly set).

Please recheck cifmw-molecule-shiftstack.

@imatza-rh

Copy link
Copy Markdown
Contributor

/lgtm

@evallesp evallesp left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve

@openshift-ci

openshift-ci Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: evallesp

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@evallesp

Copy link
Copy Markdown
Contributor

Inconsistent results of: cifmw-molecule-shiftstack. It previously succeded and it failed in last run. Overwritting.

/override rdoproject.org/github-check

@openshift-ci

openshift-ci Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

@evallesp: Overrode contexts on behalf of evallesp: rdoproject.org/github-check

Details

In response to this:

Inconsistent results of: cifmw-molecule-shiftstack. It previously succeded and it failed in last run. Overwritting.

/override rdoproject.org/github-check

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-ci

openshift-ci Bot commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

New changes are detected. LGTM label has been removed.

The shiftstack role runs ocp_testing.yaml with a testconfig that
bundles all stages (install and verification) in one
ansible-navigator call. The phased shiftstack pipeline needs to
run install stages in one Zuul job and test stages in another
while reusing the same testconfig file.

Add cifmw_shiftstack_stages_override (default: []). When set, the
role appends extra-vars that override the stages list from the
testconfig. An empty list preserves current monolithic behavior.

Per-phase testconfig files in shiftstack-qa were not added because
CI jobs can pass the stage list directly and the single testconfig
remains the source of truth for all other variables.

Related-Issue: OSPRH-33128
Signed-off-by: Tushar Jadhav <tjadhav@redhat.com>
Co-authored-by: Cursor AI <noreply@cursor.com>
@evallesp evallesp added the lgtm label Aug 11, 2026
@openshift-merge-bot
openshift-merge-bot Bot merged commit e4aa351 into openstack-k8s-operators:main Aug 11, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants