Skip to content

test: reduce common test suite while not reducing coverage - #6972

Merged
ericapisani merged 34 commits into
masterfrom
autoresearch/less-tests-common-20260730
Jul 31, 2026
Merged

test: reduce common test suite while not reducing coverage#6972
ericapisani merged 34 commits into
masterfrom
autoresearch/less-tests-common-20260730

Conversation

@ericapisani

@ericapisani ericapisani commented Jul 31, 2026

Copy link
Copy Markdown
Member

May try this on individual integrations later, but used the autoresearch skill to collapse tests where possible, remove dead ones, etc.

Went from 2720 -> 2289

Refs PY-2641

The full debug x flush x pickle x level x algo x http2 matrix ran identical
assertions 192 times. The compression-relevant dimensions (level x algo x
http2) stay fully crossed; debug/flush/pickle rotate through cases so every
value is still exercised.
…4 -> 22)

Keep every distinct truthy/falsy word plus one mixed-case variant per
result class; the exhaustive table re-tested the same .lower() path.
… only

test_proxy 42->24, test_socks_proxy 18->10. Proxy resolution is
protocol-independent; each distinct scenario is still exercised.
env_to_bool parsing is exhaustively tested in test_utils; keep one case
per precedence arm (incl. config=False with env unset).
…moved the exact duplicate escaped-regex should_propagate_trace parameter case; the unescaped regex asserts identical matching behavior.

Result: {"status":"keep","test_count":2314,"runtime_s":122.32,"covered_lines":9779,"covered_branches":2781,"coverage_pct":37.503,"failed":0,"skipped":171}
…ault value (2); the unset case already asserts the same default branch, while the non-default override remains covered.

Result: {"status":"keep","test_count":2313,"runtime_s":122.86,"covered_lines":9779,"covered_branches":2781,"coverage_pct":37.503,"failed":0,"skipped":171}
…x substring matching is unchanged by the URL scheme, and the retained HTTP localhost case asserts the same positive match.

Result: {"status":"keep","test_count":2312,"runtime_s":122.21,"covered_lines":9779,"covered_branches":2781,"coverage_pct":37.503,"failed":0,"skipped":171}
…tain coverage of negative and boundary invalid inputs through the same max_size <= 0 rejection.

Result: {"status":"keep","test_count":2311,"runtime_s":123.61,"covered_lines":9779,"covered_branches":2781,"coverage_pct":37.503,"failed":0,"skipped":171}
… all non-string, non-list values follow the same no-op return path while None, integer, and boolean representatives remain.

Result: {"status":"keep","test_count":2310,"runtime_s":121.46,"covered_lines":9779,"covered_branches":2781,"coverage_pct":37.503,"failed":0,"skipped":171}
…some$' fail against 'some-string' under the default end-anchor behavior, while positive, case-sensitive, and explicit-anchor cases remain.

Result: {"status":"keep","test_count":2309,"runtime_s":120.8,"covered_lines":9779,"covered_branches":2781,"coverage_pct":37.503,"failed":0,"skipped":171}
… skip the matcher loop and return False independently of item, while empty-string and non-empty-item representatives remain.

Result: {"status":"keep","test_count":2308,"runtime_s":121.5,"covered_lines":9779,"covered_branches":2781,"coverage_pct":37.503,"failed":0,"skipped":171}
…list return occurs before item inspection, while empty-string and ordinary-string representatives remain.

Result: {"status":"keep","test_count":2307,"runtime_s":120.78,"covered_lines":9779,"covered_branches":2781,"coverage_pct":37.503,"failed":0,"skipped":171}
…the retained empty-string case covers the same loop-skipping False result, while ordinary-string behavior remains covered with regex_list=None and nonempty matchers.

Result: {"status":"keep","test_count":2306,"runtime_s":121.11,"covered_lines":9779,"covered_branches":2781,"coverage_pct":37.503,"failed":0,"skipped":171}
…retain both inclusive numeric boundaries, while True and False retain boolean coercion behavior.

Result: {"status":"keep","test_count":2305,"runtime_s":123.16,"covered_lines":9779,"covered_branches":2781,"coverage_pct":37.503,"failed":0,"skipped":171}
…retained empty-string None-list case covers the same early False return before item inspection.

Result: {"status":"keep","test_count":2304,"runtime_s":121.11,"covered_lines":9779,"covered_branches":2781,"coverage_pct":37.503,"failed":0,"skipped":171}
… boolean acceptance, and numeric zero independently preserves the False-coerced boundary outcome.

Result: {"status":"keep","test_count":2303,"runtime_s":122.01,"covered_lines":9779,"covered_branches":2781,"coverage_pct":37.503,"failed":0,"skipped":171}
…t already creates an empty cache and asserts get() returns None, exercising the same miss behavior before validating a subsequent hit.

Result: {"status":"keep","test_count":2302,"runtime_s":123.18,"covered_lines":9779,"covered_branches":2781,"coverage_pct":37.503,"failed":0,"skipped":171}
…perset: it asserts the same empty-cache miss and first set/get, then additionally validates overwriting an existing entry.

Result: {"status":"keep","test_count":2301,"runtime_s":123.54,"covered_lines":9779,"covered_branches":2781,"coverage_pct":37.503,"failed":0,"skipped":171}
… exercises the identical existing-key update branch and resulting retrieval assertion; cache capacity is irrelevant before eviction.

Result: {"status":"keep","test_count":2300,"runtime_s":122.33,"covered_lines":9779,"covered_branches":2781,"coverage_pct":37.503,"failed":0,"skipped":171}
…ed current-scope test covers both the None and assigned-span outcomes and exercises the public no-argument API.

Result: {"status":"keep","test_count":2299,"runtime_s":123.34,"covered_lines":9779,"covered_branches":2781,"coverage_pct":37.503,"failed":0,"skipped":171}
@github-actions

github-actions Bot commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Codecov Results 📊

94982 passed | ⏭️ 6343 skipped | Total: 101325 | Pass Rate: 93.74% | Execution Time: 336m 13s

📊 Comparison with Base Branch

Metric Change
Total Tests 📉 -6006
Passed Tests 📉 -5328
Failed Tests
Skipped Tests 📉 -678

All tests are passing successfully.

✅ Patch coverage is 100.00%. Project has 2511 uncovered lines.
❌ Project coverage is 89.89%. Comparing base (base) to head (head).

Coverage diff
@@            Coverage Diff             @@
##          main       #PR       +/-##
==========================================
- Coverage    89.92%    89.89%    -0.03%
==========================================
  Files          193       193         —
  Lines        24832     24832         —
  Branches      8912      8912         —
==========================================
+ Hits         22328     22321        -7
- Misses        2504      2511        +7
- Partials      1415      1415         —

Generated by Codecov Action

"https://example.com",
True,
), # to show escaping is not needed
([r"https://.*"], "https://example.com", True),

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The 2nd test case removed here served more as documentation rather than exercising a specific code path.

Could add it back in for that reason, but the LLM removed it because it considered it a duplicate test.

@ericapisani ericapisani changed the title [DO NOT MERGE] Autoresearch experiment for reducing the amount of tests without losing coverage test: reduce common test suite while not reducing coverage Jul 31, 2026
@linear-code

linear-code Bot commented Jul 31, 2026

Copy link
Copy Markdown

PY-2641

@ericapisani
ericapisani marked this pull request as ready for review July 31, 2026 14:13
@ericapisani
ericapisani requested a review from a team as a code owner July 31, 2026 14:13
@ericapisani
ericapisani merged commit baf895d into master Jul 31, 2026
140 checks passed
@ericapisani
ericapisani deleted the autoresearch/less-tests-common-20260730 branch July 31, 2026 15:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants