Skip to content

test: force headless Matplotlib backend for local + CI test runs#197

Open
petercorke wants to merge 1 commit into
rai-opensource:masterfrom
petercorke:fix/headless-tests-locally
Open

test: force headless Matplotlib backend for local + CI test runs#197
petercorke wants to merge 1 commit into
rai-opensource:masterfrom
petercorke:fix/headless-tests-locally

Conversation

@petercorke

Copy link
Copy Markdown
Collaborator

Problem

Running pytest locally pops up real Matplotlib windows, and a couple of
animation tests hang outright. CI already sets MPLBACKEND=Agg in the
workflow env, but that never applied to local runs — several modules
(geom2d, geom3d, spline, animate) import matplotlib.pyplot at
module load time, so whichever backend is active at that point sticks for
the whole session, and locally that's the platform's interactive backend.

Fix

  1. tests/conftest.py (new): os.environ.setdefault("MPLBACKEND", "Agg")
    before anything else runs. conftest.py is guaranteed to load before
    pytest imports any test module, so this closes the gap for local runs
    without touching the CI workflow (redundant there now, not conflicting).
    setdefault(), not a hard override, so MPLBACKEND=MacOSX pytest ...
    still shows a real plot when you actually want to eyeball one.

  2. Forcing Agg everywhere surfaced a real, previously-masked hang:
    tranimate(..., wait=True) busy-loops on plt.pause() waiting for
    FuncAnimation's timer callback to deregister itself — driven by the
    GUI event loop, which Agg doesn't have. Confirmed with a hard-alarm
    timeout: the affected test ate a full 12s budget with zero output vs.
    completing normally under a real backend. These tests were already
    skipped in CI via os.environ.get("CI") == "true", which happened to
    also mask this locally-only-when-Agg-is-forced hang — but only in CI;
    locally (pre-Agg) they ran fine because a real backend genuinely pumps
    the event loop.

    Broadened the three affected skip conditions (test_animate in
    test_transforms3d_plot.py; the two animate=True test_visualize
    tests and the inline guard in test_spline.py) to additionally skip
    when plt.get_backend() == "agg" — the actual root cause, rather than
    the CI-env-var proxy for it. Purely additive, no existing condition
    removed, so CI behaviour is unchanged. Everything else (plain,
    non-animated plotting tests) now safely runs headless both locally and
    in CI — strictly more coverage than before.

Testing

  • Full suite, CI unset (simulating a local run): 339 passed, 3 skipped,
    1.6s, no windows, no hangs.
  • Full suite, CI=true (simulating CI): 338 passed, 4 skipped — matches
    prior CI behaviour.
  • black --check clean at the pinned 23.10.0.

Local `pytest` runs were popping up real windows and, for a few tests,
hanging outright. Two separate problems, one root cause:

1. CI sets MPLBACKEND=Agg via the workflow env, but nothing did that
   for local runs. Several modules (geom2d, geom3d, spline, animate)
   import matplotlib.pyplot at module load time, so whatever backend
   is active when they're first imported sticks for the whole
   session. Locally that's the platform's interactive backend, hence
   the popups.

   Fix: tests/conftest.py sets os.environ.setdefault("MPLBACKEND",
   "Agg") before anything else runs. conftest.py is guaranteed to
   load before pytest imports any test module, so this closes the gap
   for local runs without touching the CI workflow (now redundant
   there, not conflicting). setdefault(), not a hard override, keeps
   an escape hatch: `MPLBACKEND=MacOSX pytest ...` still shows you a
   real plot when you actually want one.

2. Forcing Agg everywhere surfaced a latent bug: Twist/pose animation
   tests use tranimate(..., wait=True), which busy-loops on
   `plt.pause()` waiting for FuncAnimation's timer callback to
   deregister itself. That deregistration is driven by the GUI event
   loop, which Agg doesn't have — so under Agg the loop never exits.
   Confirmed with a hard-alarm timeout: the affected test consumed a
   full 12s budget with zero output, versus completing normally under
   a real backend.

   These tests were already skipped in CI (via an `os.environ.get(
   "CI") == "true"` check) for a related but distinct reason — no
   display in the CI runner. That check happened to also mask this
   hang, but only in CI; locally, before this change, it ran fine
   because a real backend really does pump the event loop. Once Agg
   is forced locally too, the same hang would trigger there.

   Fix: broadened the three affected skip conditions (test_animate in
   test_transforms3d_plot.py; the two animate=True visualize tests
   and the inline guard in test_spline.py) to additionally skip
   whenever plt.get_backend() == "agg", which is the actual root
   cause rather than the CI-env-var proxy for it. Purely additive —
   no existing skip condition was removed, so CI behaviour is
   unchanged. Everything else (plain, non-animated plotting tests)
   now safely runs headless both locally and in CI, which is strictly
   more coverage than before.

Verified: full suite passes locally with CI unset (339 passed, 3
skipped, 1.6s, no windows, no hangs) and with CI=true set (338 passed,
4 skipped, matching prior CI behaviour).
@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants