Skip to content

ci: add LVH-based integration tests for external contributors - #1795

Draft
Molter73 wants to merge 8 commits into
mainfrom
mauro/ci/external-contributions
Draft

Molter73 wants to merge 8 commits into
mainfrom
mauro/ci/external-contributions

Conversation

@Molter73

Copy link
Copy Markdown
Member

Description

Add a standalone GitHub Actions workflow that runs integration tests inside a nested QEMU/KVM VM using cilium/little-vm-helper (LVH). This enables external contributors (fork PRs) to run integration tests without requiring repository secrets or GCP credentials.

The workflow:

  • Builds the fact container image locally (no registry push).
  • Boots an LVH VM with a 6.6 LTS kernel and BPF LSM enabled via kernel cmdline.
  • Loads the image into the VM's Docker and runs pytest against test_file_open.py as an initial spike validation.
  • Caches the VM image and kernel for faster subsequent runs.
  • Uses pull_request trigger (works for both fork and internal PRs).

This is Phase 1 (spike) of issue #1794. Once validated, Phase 2 will expand to the full test suite and Phase 3 will reduce duplication with the existing GCP VM-based integration tests.

Refs: #1794

Assisted-by: claude-opus-4-6 noreply@opencode.ai

Checklist

  • Patch has a change log entry OR does not need one.
  • Investigated and inspected CI test results
  • Updated documentation accordingly

Automated testing

  • Added unit tests
  • Added integration tests
  • Added regression tests

If any of these don't apply, please comment below.

Testing Performed

Validation needs to happen in CI.

@coderabbitai

coderabbitai Bot commented Sep 16, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Comment @coderabbitai help to get the list of available commands.

@codecov-commenter

codecov-commenter commented Sep 16, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 33.40%. Comparing base (05da053) to head (dc5d1fa).
⚠️ Report is 8 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1795   +/-   ##
=======================================
  Coverage   33.40%   33.40%           
=======================================
  Files          22       22           
  Lines        3628     3628           
  Branches     3628     3628           
=======================================
  Hits         1212     1212           
  Misses       2407     2407           
  Partials        9        9           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Molter73
Molter73 force-pushed the mauro/ci/external-contributions branch 6 times, most recently from 8e84774 to 41e28cf Compare September 17, 2026 14:35
Add a standalone GitHub Actions workflow that runs integration tests
inside a nested QEMU/KVM VM using cilium/little-vm-helper (LVH). This
enables external contributors (fork PRs) to run integration tests
without requiring repository secrets or GCP credentials.

The workflow:
- Builds the fact container image locally (no registry push).
- Boots an LVH VM with a 6.6 LTS kernel and BPF LSM enabled via
  kernel cmdline.
- Loads the image into the VM's Docker and runs pytest against
  test_file_open.py as an initial spike validation.
- Caches the VM image and kernel for faster subsequent runs.
- Uses pull_request trigger (works for both fork and internal PRs).

This is Phase 1 (spike) of issue #1794. Once validated, Phase 2 will
expand to the full test suite and Phase 3 will reduce duplication with
the existing GCP VM-based integration tests.

Refs: #1794

Assisted-by: claude-opus-4-6 <noreply@opencode.ai>
Expand the LVH-based integration tests from the Phase 1 spike
(test_file_open.py only) to the full pytest suite:

- Drop -x and the single test-file argument so all tests under
  tests/ run and results are collected in the JUnit report.
- Add --tb=short for more readable failure output in CI logs.
- Bump the job timeout from 60 to 90 minutes to accommodate the
  full suite plus on-the-fly container builds (editors, fedora,
  self-deleter).
- Document why --no-local-builds is intentionally omitted:
  quay.io/rhacs-eng/qa-multi-arch is a private registry that fork
  PR runs have no credentials for, so tests/containers.py's
  pull_or_build() needs to fall back to building those test
  containers locally from their Containerfiles on a 401/404.

Phase 1's test_file_open.py run already exercised the test_container
fixture (which pulls quay.io/fedora/fedora:43), confirming the VM has
outbound network access needed for the additional container builds.

Refs: #1794

Assisted-by: claude-opus-4-6 <noreply@opencode.ai>
@Molter73
Molter73 force-pushed the mauro/ci/external-contributions branch from 41e28cf to eb50f46 Compare September 17, 2026 14:47
@Molter73
Molter73 force-pushed the mauro/ci/external-contributions branch from 6cd82f0 to 1c29c84 Compare September 18, 2026 10:47
Replace the LVH (little-vm-helper) based CI with a standalone
hack/qemu-vm.sh script that boots cloud images directly in QEMU/KVM.
This removes the Go toolchain, LVH build, and LVH OCI image/kernel
dependencies in favor of standard qcow2 cloud images.

hack/qemu-vm.sh:
  Reusable script (start/ssh/stop) for booting cloud images in
  QEMU/KVM.  Handles qcow2 overlay creation, ephemeral SSH key
  generation, cloud-init seed ISO with user-provided templates
  (__SSH_PUBKEY__ placeholder), virtiofsd for host directory sharing,
  two-boot BPF LSM activation via grubby, and Docker verification.
  Works both in CI and locally for reproducing test failures.

hack/cloud-init/{centos,fedora,ubuntu}.yml:
  Per-distro cloud-init templates configuring root SSH access, Docker
  CE installation, and BPF LSM kernel cmdline.

Workflow changes:
  - Use CentOS Stream 10 GenericCloud qcow2 image with SHA256
    verification.
  - Matrix uses named distro entries (extensible to other distros by
    adding image URLs and cloud-init templates).
  - Host directory shared via virtiofs instead of 9p.  virtiofs
    correctly reports FMODE_CREATED (unlike 9p), but does not support
    xattrs, so --tmp-dir is still needed for test directories.
  - Fact image loaded via virtiofs mount instead of scp.
  - No more LVH clone, Go setup, kernel/image pull, or libvirt deps.
  - VM lifecycle (start/stop) managed via hack/qemu-vm.sh.

Refs: #1794

Assisted-by: claude-opus-4-6 <noreply@opencode.ai>
@Molter73
Molter73 force-pushed the mauro/ci/external-contributions branch 4 times, most recently from f16c0e8 to 37e1cf8 Compare September 18, 2026 13:54
Expand the QEMU integration test matrix from a single CentOS Stream
10 entry to five distros:

- centos-9-stream
- centos-10-stream
- ubuntu-24.04
- ubuntu-26.04
- fedora-44

Each entry now carries its own cloud_init template so the matrix can
mix distro families instead of hardcoding hack/cloud-init/centos.yml.

Fedora 45 is not yet released (only unstable, date-stamped nightly
composes exist); Fedora 44 is used as the current stable release.

Checksum verification switched from manually extracting and
comparing hashes to sha256sum -c directly: the checksum file is
filtered down to the line(s) matching our image's filename (via
grep -F), then piped to sha256sum -c, which auto-detects both GNU
("hash  filename") and BSD ("SHA256 (filename) = hash") tagged
formats and correctly ignores '#'-prefixed comment lines. This also
made the extraction robust against Ubuntu's SHA256SUMS listing many
unrelated images and Fedora's PGP-clearsigned CHECKSUM files.

Moved test-runner dependency installation (python3, pip, venv, gcc)
from a post-boot SSH step into each cloud-init template's runcmd, so
it runs in parallel with the Docker setup during the boot we already
wait for, instead of adding a second SSH round-trip.

Refs: #1794

Assisted-by: claude-opus-4-6 <noreply@opencode.ai>
@Molter73
Molter73 force-pushed the mauro/ci/external-contributions branch from 37e1cf8 to dc5d1fa Compare September 18, 2026 14:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants