Skip to content

Show each rerun attempt outcome in summary - #370

Merged
icemac merged 3 commits into
pytest-dev:masterfrom
LouisDeconinck:fix/rerun-summary-191
Sep 16, 2026
Merged

icemac merged 3 commits into
pytest-dev:masterfrom
LouisDeconinck:fix/rerun-summary-191

Conversation

@LouisDeconinck

Copy link
Copy Markdown
Contributor

Fixes #191. The rerun summary now preserves one report per test attempt and displays each rerun and final outcome with pytest-style status colors. Tests: .venv/bin/pytest -q; pre-commit run --all-files --show-diff-on-failure.

@icemac icemac left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review of the rerun-summary change. Findings 1–3 (xdist crash reruns, teardown tracebacks, xdist ordering) were reproduced by running the plugin on master vs. this branch; the full suite passes on the branch because the existing summary tests only assert outcomes.

Comment created by Claude

Comment thread src/pytest_rerunfailures.py Outdated
Comment thread src/pytest_rerunfailures.py Outdated
Comment thread src/pytest_rerunfailures.py Outdated
Comment thread src/pytest_rerunfailures.py Outdated
Comment thread src/pytest_rerunfailures.py Outdated
Comment thread src/pytest_rerunfailures.py Outdated
Comment thread src/pytest_rerunfailures.py Outdated
Comment thread tests/test_pytest_rerunfailures.py Outdated
- Read terminalreporter.stats directly instead of a dedicated
  RerunSummary plugin, which duplicated the same reports.
- Stamp report.rerun on xdist crash reports in pytest_handlecrashitem
  so crash reruns appear in the summary instead of vanishing.
- Emit one line per report (grouped by nodeid, ordered by attempt and
  phase) so call+teardown failures in one attempt keep both tracebacks
  and the final teardown ERROR is not hidden behind the call FAILED.
- Take status words and colours from pytest_report_teststatus instead
  of a second hand-maintained markup table; traceback lines are no
  longer re-parsed as status words.
- Cover ERROR and SKIPPED lines plus crash-summary and xdist grouping
  in tests; scope colour assertions to the rerun section.
@LouisDeconinck

Copy link
Copy Markdown
Contributor Author

Thanks for the thorough review — all findings reproduced and addressed in 97fb513:

  • xdist crash reruns vanish: pytest_handlecrashitem now stamps report.rerun with the attempt index (from get_test_failures, before add_test_failure), so crash reports are collected and ordered alongside the rescheduled attempt's own reports. pytest -n 2 --reruns 2 -r R on an os._exit(1) test now prints RERUN, RERUN, FAILED — matching the 2 rerun footer.
  • Dropped teardown traceback / hidden final ERROR: the (nodeid, rerun) dedup is gone. Each emitted report gets its own line (sorted by rerun, then setup/call/teardown), so a call+teardown failure in one attempt shows both RERUN lines with both tracebacks under --rerun-show-tracebacks, and the final attempt shows FAILED + ERROR.
  • xdist ordering: reports are grouped by nodeid (first-seen order) then sorted by attempt number, so interleaved worker reports render as contiguous per-test groups.
  • Traceback lines re-colourised: show_rerun now returns (line, markup) pairs; traceback lines carry None markup and are written plainly.
  • Re-implemented pytest_report_teststatus: status words and markup now come from config.hook.pytest_report_teststatus (with the same default outcome colouring _pytest/terminal.py applies for plain-string words), so ERROR, SKIPPED, XFAIL/XPASS, and third-party categories are all handled consistently.
  • RerunSummary duplicates terminalreporter.stats: the plugin is removed entirely; show_rerun iterates chain.from_iterable(terminalreporter.stats.values()) filtered on hasattr(report, "rerun"), which also fixes the ordering issue per your suggestion.
  • Unreachable get_plugin fallback: gone with the plugin removal — -p no:pytest-rerunfailures-summary no longer exists as a silent revert path.
  • Coverage: added tests for the ERROR (failed setup) and SKIPPED (skipped call + failing teardown) paths, call+teardown traceback preservation, the crash summary line, and xdist per-test grouping; the colour assertions are now scoped to the rerun section.

On XFAIL/XPASS specifically: xfail-marked tests never reach the summary in practice — _should_not_rerun excludes wasxfail reports, and pytest converts teardown failures on xfail-marked tests into xfail too. Delegating to the hook still honours those (and any plugin-specific) statuses if they ever appear.

Verified locally: 248 tests pass on pytest 9.1.1 + xdist 3.8.0, ruff check/format clean.

@icemac icemac left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@icemac
icemac merged commit ed62cfc into pytest-dev:master Sep 16, 2026
45 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Enhancement: present final rerun test summary info section in the same way as the test session starts section

2 participants