fix(derivations): one bin per distinct value when more bins are requested than values exist - #128
Merged
Merged
Conversation
…sted than values exist
## Summary
- **What:** when n >= number of distinct finite values, fit edges at the midpoints
between neighbouring values (one bin per value) instead of capping n and re-taking
quantiles.
- **Why:** 0.3.1's cap collapsed heavily tied columns — 3,000 ones and 1,000 twos with
n=4 fitted ONE bin (0.3.0 fitted two) because the halved quantile positions both landed
on 1. Caught by the app's preview test on the pin bump.
- **Scope:** derivations._fit_edges; two existing tests re-pinned to the n < distinct
case they were written to prove; robustness tests extended; CHANGELOG.
## Contract / Invariants (must remain true)
- n < number of distinct values: byte-identical edges (original quantile / linspace path;
pinned by test_fewer_bins_than_distinct_values_is_untouched and the bin-extremes suite).
- Every finite value lands in a bin (fuzz + bin-extremes).
- validator exit 0.
## Behavior Changes (explicit)
- n >= distinct count: edges = [v1, midpoints..., vk]; message "requested n bins, ties
produced k (one per distinct value)". Labels change from interpolated quantile edges
(e.g. '[1, 1.25)') to midpoint edges ('[1, 1.5)'); membership is one value per bin.
## Tests
- test_heavily_tied_column_keeps_one_bin_per_distinct_value (the regression)
- test_more_bins_than_distinct_values_gives_one_bin_per_value (edges + counts)
- test_fewer_bins_than_distinct_values_is_untouched
- test_tie_drop_uses_fitted_count_labels_and_message and
test_validate_label_count_against_fitted_not_requested now request n=4 on 5 distinct
values so the quantile-collision case they document still occurs.
## Manual Verification
- pytest tests/: 2431 passed; ruff clean; validator exit 0
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
n=4fitted one bin, where 0.3.0 fitted two, because the halved quantile positions both landed on 1. Caught by the app's own preview test during the pin bump. Requesting far more bins than values (the case the cap was for) now also gives one bin per value instead of mostly-empty interpolated bins._fit_edgesinderivations.py; two existing tests re-pinned to the fewer-bins-than-values case they were written to prove; robustness tests extended; CHANGELOG.Contract / invariants
validation/e2e_bishop_report.pyexits 0.Behaviour changes
n >= distinct count: edges are[v1, midpoints…, vk]and the message readsrequested n bins, ties produced k (one per distinct value). Labels change from interpolated quantile edges such as[1, 1.25)to midpoint edges such as[1, 1.5); bin membership is one value per bin either way.Methodology
Tests
test_heavily_tied_column_keeps_one_bin_per_distinct_value— the regression, 3,000 ones and 1,000 twos.test_more_bins_than_distinct_values_gives_one_bin_per_value— edges and counts for both methods.test_fewer_bins_than_distinct_values_is_untouched— the original path pinned.test_tie_drop_uses_fitted_count_labels_and_messageandtest_validate_label_count_against_fitted_not_requested— now request 4 bins on 5 distinct values so the quantile-collision case they document still occurs (5 on 5 now correctly fits 5).pytest tests/— 2431 passed, 10 skippedruff check .— cleanGolden masters untouched
🤖 Generated with Claude Code
https://claude.ai/code/session_01BKphCi1LC1hjGYteWEK9JV