Skip to content

Fix 1.1 release audit findings and complete documentation follow-ups - #286

Open
jeremymanning wants to merge 178 commits into
masterfrom
fix/1.1-release-review
Open

Fix 1.1 release audit findings and complete documentation follow-ups#286
jeremymanning wants to merge 178 commits into
masterfrom
fix/1.1-release-review

Conversation

@jeremymanning

@jeremymanning jeremymanning commented Sep 6, 2026

Copy link
Copy Markdown
Member

This PR fixes release-review regressions in forecasting, caching, feature preservation, fonts and plotting, and updates the tutorials and review tooling. Timestamped forecasts now use the training clock; Plotly 3D colors retain their hue under transparency; full-curve companion panels retain a moving current-time marker. All 25 tutorials have been re-executed.

Closes #284.
Closes #285.

The candidate is fa3e60e5670cabfc1cbbd282e122b9098cc6d1a3. Release publication, merging, tagging and public artifact updates remain subject to Jeremy's manual sign-off.

Findings fixed in the PR

Priority Finding Fix and evidence
High Forecast backtests reused an instance fitted on dataset 1 for dataset 2. On a sine series followed by a positive quadratic series, AutoRegressor() produced negative quadratic forecasts and MAE 2551.68, versus 0.00246 when passing the class. Deep-copy the model for each dataset; preserve caller state. Regression tests compare actual returned forecasts for class, instance, and dictionary forms.
High Scoring accepted previously fitted forecasters/imputers even when their learned state could contain the held-out values. Imputation scoring also fitted caller-owned instances. Require unfitted instances for scoring, copy them before fitting, and document the distinction from ordinary fitted-model replay. Real fitted/unfitted model tests cover both forms.
Medium Delay silently lost features when distinct pandas column labels had identical string representations (1 and '1'); a 2-column, 2-lag input produced only 2 output columns. Reject colliding labels with a renaming instruction; tests include mixed-type and duplicate labels.
Medium URL-cache temporary names used only the process ID. Threads caching the same URL collided: 82 of 100 concurrent writes failed with FileNotFoundError. Use a unique temporary file per write, clean it on error, and atomically replace the destination. Test 100 real writes across 12 threads and verify payload/metadata and cleanup.
Medium System-installed Noto Sans took precedence over the bundled Regular face, contradicting deterministic font selection and failing the existing font regression test on this machine. Register bundled faces ahead of equal-scoring system faces. A fresh interpreter with another real same-family font proves the bundled file wins.
Medium Optional dependency minimums allowed gensim 4.3 and scikit-image 0.22, predating NumPy 2 support despite the library requiring NumPy>=2. Raise floors to gensim>=4.4.0 and scikit-image>=0.23.2 in extras/dev/docs. Real minimum-version feature tests pass under NumPy 2.3.5.
Documentation Public plotting/predict/impute docstrings described shipped features as 1.2; dependency prose implied ARIMA imputation. Correct version labels and separate forecasting from imputation support.
Documentation The “convert now” forecast example still hand-wrote URL download/cache logic after the native cache landed. Use hyp.load(ARCHIVE, cache=True), regenerate and execute the tutorial. The tutorial and video are regenerated with the current code.
Tooling The browser verifier expected docs-notebooks/master, searched highlighted HTML for contiguous pip install, and demanded an autoplay call in deliberately paused Plotly animations. Validate versioned notebook links, rendered code text, loaded frames/play controls, and execute a real transition in Chromium. Allow evidence/build paths outside the checkout.

Source and regression-test map

  • Forecast/imputation ownership: hypertools/predict/backtest.py, hypertools/impute/backtest.py; tests/test_predict_backtest.py, tests/test_impute_backtest.py.
  • Cache atomicity: hypertools/io/sources.py; tests/test_load_url_cache.py.
  • Delay collisions: hypertools/manip/delay.py; tests/test_manip_delay.py.
  • Font precedence: hypertools/plot/fonts.py; tests/test_fonts_bold.py.
  • Dependency compatibility: pyproject.toml, docs/doc_requirements.txt; real minimum-version runs of tests/test_gensim_text.py and tests/test_density.py, plus packaging/optional-import checks.
  • Documentation: dispatcher docstrings, docs/optional_dependencies.rst, readme.md, CHANGELOG.md, examples/animate_forecast.py, and its executed tutorial notebook.
  • Browser verification: scripts/verify_docs_playwright.py.

Timestamp-aware forecast follow-ups

Forecast fitting accounts for time-like indexes, including irregular/out-of-order observations and different clocks across datasets. The default future interval is the median positive gap between sorted timestamps, with an explicit step override. GaussianProcess uses actual observation times; regular-grid models use a documented interpolation policy. Real-model regression checks cover unsigned and large integer timestamp arithmetic as well as forecast scoring and fitted-model reuse.

Notebook and visual review follow-ups

Finding Implemented resolution
Plotly legend/trajectory colors differ Uniform 3D RGBA alpha is transferred to native trace opacity with RGB colors, including frame updates. Opaque frames explicitly reset earlier transparency. Real Chrome pixel tests supplement color/alpha parity tests; nonuniform alpha retains its original semantics.
Colab plots disappear during Run all Each Plotly example retains a PNG preview and a standalone HTML artifact. A single final viewer opens one live iframe at a time; switching/closing disposes the previous iframe. Download payloads are prepared only on click, avoiding both cumulative live WebGL contexts and repeated multi-megabyte JavaScript payloads in saved cell outputs. Fresh Colab frontend verification remains a separate check.
Companion right marker freezes Full-curve visibility is separated from current-time marker position and hue. The example visits all 30 input rows over five seconds and labels its trailing smooth. Tests check real artists, dates, full trends, hues and backward seeking, including unequal frame/sample counts. Existing serial-clock tests remain in force.
Visual failures disappear from reports Attention tables and cross-run comparisons include visual verdicts and notes alongside automatic outcomes. Each rerun resets its visual verdict and receives a fresh execution identity.
Stale approvals/provenance Reports carry notebook, inventory and case-source hashes, execution IDs, actual package source, optional dependency versions, settings, binaries and structured warnings. Changed checks are distinguished from changed outcomes.
Animation exports falsely pass Every run uses fresh destinations. Image/movie decoders require multiple distinct frames and meaningful duration; single-frame GIF/APNG outputs fail. Export examples now have enough time and frames to demonstrate motion.
Nonportable downloads/playback Colab uses its file-download API; local Jupyter prepares a downloadable data link only on request. Tutorial MP4-producing cells embed movie bytes on Colab while retaining relative media for local Jupyter/Sphinx.
Incomplete feature/behavior coverage Tour expanded from 227 to 244 cases. Added multi-column series/date bounds, returned data/colors/metadata/panel bundles, fitted sklearn pipeline reuse, callback clocks and callable titles, CJK fonts, luminance controls, compressed/extensionless and legacy XLS input, smoothing warm-up, hosted fitted topic models and stronger hosted-data contracts. Every dispatcher family participates in registry coverage guards.
Optional-install claim lacked a real missing dependency scripts/verify_optional_install.py creates an isolated base-only environment, verifies the friendly disabled-autoinstall error, then genuinely installs the missing XLSX dependency and verifies a round trip. The working environment is never stripped of dependencies.
Tutorial installer inconsistencies All 25 tutorial installers require 1.1 or newer and preserve current local checkouts. Independent convokit/py7zr prerequisites still execute. Text tutorials demonstrate their stated on-demand policy; Kalman no longer requests the unrelated prediction extra. Successful prerequisite-only pip chatter is omitted from published outputs.
Executor skips configuration Configuration and the tagged candidate installer are separate. The executor skips only HyperTools installers, preserving other setup and work.
Installation failure evidence Colab candidate setup streams and retains its installer log; failure offers that log for download. Successful logs are included in the report ZIP. Local execution explicitly retains the selected checkout.
Native GUI case tests inline rendering Desktop interaction runs in a separate Qt process and leaves the window available. It is an explicit optional manual case, not an inline-rendering PASS.
Remote pickle trust Historical Drive/Dropbox pickle demonstrations require conspicuous, disabled-by-default trust opt-in. Built-in hosted fitted models retain their independent digest validation. No post-deserialization hash is presented as protection.
Documentation/maintenance Corrected 1D inspection text, synchronized inventory/markdown, removed generated literal branches, strengthened expected-error assertions, documented XLSX writing, corrected candidate-versus-wheel release checks, and documented behavior evidence routes. Setup/playback additions are checked against exact templates without increasing the existing example-logic size budgets.

Windows and lint follow-up

The previous head passed all eight Linux/macOS matrix jobs but failed two notebook tests on every Windows version because UTF-8 files were decoded as CP1252. Notebook generation, the embedded viewer/export helpers and tests now explicitly use UTF-8. A real Unicode Plotly HTML regression opens/closes the actual ipywidgets viewer; no mocks are used. Equivalent Colab availability imports in 14 tutorials and their template now satisfy Ruff. Repository-wide lint passes and CI provisions Ruff 0.15.20 explicitly.

Verification

Current candidate: fa3e60e5670cabfc1cbbd282e122b9098cc6d1a3.

  • Current local tour: 241 PASS, 3 SKIP, 0 FAIL/BLOCKED, all 244 cases. Reports record this exact commit and a clean tracked checkout. Source SHA-256: 10ff17a28db615994497a177dac75a58c2cb788f0a7b67fbdcbf41fb6535958b.
  • Current focused notebook/install/support/visual/tutorial checks: 364 passed, 8 skipped; full repository Ruff and git diff --check pass.
  • Opt-in end-to-end example smoke from an isolated checkout at this SHA: 6 passed, 338 deselected.
  • Actual macOS QtAgg interaction: PASS. A real window opened; mouse press/move/release rotated the plot; it closed successfully. Screenshots and event results are retained. Qt was installed only in an isolated verification environment.
  • Candidate wheel/sdist build and Twine metadata checks: PASS. Fresh wheel installation and pip check pass; public API/pipeline, three irregular-time forecasters, inferred future spacing, Matplotlib PNG and Plotly standalone HTML checks pass. Imports are from isolated site-packages, not the source checkout. Artifact digests, package versions, bundled fonts/config/licenses and exact candidate provenance are retained.
  • Prior implementation evidence: all 25 tutorials executed, full default suite 5,936 passed, 20 skipped, 2 deselected, both opt-in large-data tests passed separately, and a real missing-extra installation/round trip passed. The current equivalent tutorial import edits are covered by the focused checks above.
  • Prior browser/visual evidence: real Chrome color-pixel regressions and standalone animated HTML transitions passed. The companion movie has 30 H.264 frames over five seconds; its inspected middle frame shows both companion markers at sample 15 and the matching January 16 title. Local/standalone browser checks do not substitute for fresh Colab frontend acceptance.
  • The current executed tour is about 6 MB, retaining PNG previews rather than live Plotly MIME outputs. Automated reports preserve warnings and leave unreviewed visual cases unapproved.

Current hosted CI: run 34558379385 completed successfully: 16 successful jobs, zero failures, one expected skipped release-only gate. All 12 matrix jobs passed, including all four Windows versions. Ubuntu Python 3.12 additionally passed repository lint, pandas 3, coverage (91% total line coverage) and screenshot generation. Clean docs executed all 51 gallery examples and 323 doctests with zero failures. Dataset gate: 33 passed; live-source gate: 43 passed, 1 deselected. Fresh wheel and sdist install gates passed. PR CI does not replace master/tag release gates.

The final review packet is ready locally in notes/notebook_critical_review_2026-09-10/START_HERE_FINAL_REVIEW.md, with detailed CI evidence in ci_final_status.md. The cleared Colab notebook and executed local reference have identical cell sources. All 104 tracked package Python files match the built candidate wheel byte-for-byte. No package or test changes were needed during this final verification pass.

Local evidence: notes/colab/feature_tour_preflight_fa3e60e5/; detailed follow-up report: notes/notebook_critical_review_2026-09-10/windows_preflight_report.md. These are local review files, not published documentation. The cleared Colab upload copy is notes/colab/hypertools_1.1_candidate_fa3e60e5.ipynb.

Remaining release checks

  1. Run the pinned tour in a fresh Colab runtime. After Run all, inspect early previews, viewer switching/closing, legend/trajectory colors, companion playback/scrubbing/dates, movie playback and report/artifact downloads. Retain notebook/reports and record visual decisions.
  2. Review remaining manual cases; macOS Qt interaction is verified, while native Windows GUI and explicitly trusted remote-pickle demonstrations remain separate acceptance decisions.
  3. Obtain Jeremy's manual sign-off before merging, tagging or changing public release artifacts. From the approved final release commit, rebuild gallery/manifest and wheel/sdist, record digests, and require master and tag release-gate CI to pass. The current tag/draft/assets describe the older 96ac8b7f draft.
  4. Publish only with explicit approval, then verify the actual published wheel, public documentation and Colab links, and conda-forge follow-up.

@jeremymanning
jeremymanning marked this pull request as ready for review September 6, 2026 03:36
jeremymanning and others added 14 commits September 5, 2026 23:54
…coring loops

The ultrareview of PR #286 flagged both 'from .common import ...' lines
as loop-body imports with no circular-import reason (common.py does not
import backtest.py). Module-scope imports match every sibling module.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…e, attribute the unscored warning, score ragged return_score input

Release-review findings on the 1.1 draft:
- resolve_metrics() now raises ValueError naming a metric listed twice
  (case-insensitively) instead of a TypeError deep in build_scores(),
  for both predict(holdout=) and impute(truth=).
- holdout=True with t=0 reports t=0 as the problem.
- the 'left N scored value(s) missing' warning uses external_stacklevel()
  like every other user-facing warning in these modules.
- align(return_score=True) works on ragged input that align() trims; the
  'before' score is computed on the row-trimmed input (documented).
Seven regression tests, each failing on the unfixed library.

Also: RELEASE_CHECKLIST.md rewritten for the 1.1.0 re-cut (tag exists as a
draft and must be moved; gallery namespace is republished wholesale; the
example smoke gate is a manual step), and the session note for this review.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…al observables; export HypertoolsOfflineError

- tests/predict/test_common.py: the all-identical-timestamps case is proven
  to come from the live _infer_step via the exception's own traceback and
  message, not a monkeypatched spy.
- tests/predict/test_predict_multiindex.py: grouping helpers are checked
  through their real outputs and the returned per-group models, not by
  wrapping them with observers.
- tests/test_names_display.py: notebook display timing is observed on a
  real in-process IPython.InteractiveShell with the json renderer captured
  through IPython's own capture_output; go.Figure.show is no longer patched.
- HypertoolsOfflineError is importable from hypertools and hypertools.io
  like the other three exceptions (public-API pin test updated).
- scripts/generate_baseline_screenshots.py points at the roadmap note that
  exists; trailing whitespace stripped from six tracked files so
  'git diff --check v1.0.0..HEAD' is clean.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…defect markers, allowlist the deliberate ax= demos, gate the rebuilt tutorials' output cells

Issue #284 claimed the DEFECT_MARKERS scan was tightened with a per-file
allowlist for deliberate ax= demos and that every tutorial's measured
output cells were recorded; the release review found the scan covered
only the six launch scripts and six launch notebooks.

- scan_for_defects(root) walks all 51 examples and every non-install
  code cell of all 25 tutorials (install cells hold the legitimate
  find_spec('hypertools') guard); a planted-marker test proves it.
- new ax= marker (hand-built axes, raw ax.plot/scatter, plt.subplots)
  with a counted DEFECT_ALLOWLIST so a new use in an allowlisted file
  still fails and a stale entry is reported.
- EXPECTED_VISIBLE_OUTPUTS gains the eight rebuilt tutorials; the
  ran-every-cell / right-cells / no-error-output tests cover all 14.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…le, colour, loader and text paths

Plot (22 findings, 53 regression tests):
- palette= lists shorter than the dataset count cycle again with no hue
  (1.0 behaviour); empty palettes raise ValueError, not StopIteration;
  per-dataset {category: color} dict entries merge by name; NaN in a
  continuous hue no longer poisons the colour range; legend_kwargs
  fontsize is honoured with font=; blend categories are RGB.
- dataset_fade= and on_frame artist mutations reach the LineCollections
  drawn under a continuous hue; loop=True accepts the documented
  2(n+1)-1 rotations list; companion= panels and {index} titles advance
  monotonically under order='serial'; window_bounds.start reflects the
  comet-head window; bad companion=/dataset_fade= values name the kwarg;
  a raising on_frame during .save() surfaces its own exception; a raising
  title= leaves no orphaned animation.
- title_wrap= applies to dynamic titles and keeps explicit newlines;
  plotly draws newline titles as line breaks and reserves top margin per
  title line and size; nested tuple labels= annotate; bare-string labels,
  bad label_anchor=, non-string title entries/callables, title_color vs
  title_kwargs colour conflicts and bad {index} formats raise clearly.

IO / tools (10 findings):
- load(offline=True) opens no connection: URLs skip the seaborn listing,
  the listing fetch has a timeout and a remembered failure, uncacheable
  sources raise HypertoolsOfflineError.
- yahoo: bars carry the exchange-local trading day (gmtoffset applied).
- synthetic datasets accept RandomState, Generator, SeedSequence and
  np.integer seeds everywhere; a reused SeedSequence is reproducible;
  n_datasets rejects non-integral values; streaming=True on a
  non-Hugging-Face source raises instead of returning everything.
- text2mat: a flat list of strings is one dataset (was [(N,d),(0,d),...]
  since 1.0), ragged nested lists work, mixed inputs raise; a dict
  semantic= spec with a gensim vectorizer warns and skips.
- text_windows accepts numpy integers.
- format_data warnings are attributed to the caller's line
  (external_stacklevel), so notebooks stop printing the library path.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Manipulator.fit() and Aligner.fit() returned None, so
Smooth().fit(x).transform(y) and HyperAlign().fit(xs).transform(ys)
raised AttributeError; Imputer.fit() already returned self. Both now
return the fitted instance on every path (documented), with a chaining
test over Normalize/ZScore/Smooth/Resample/Delay, HyperAlign/Procrustes/
NullAlign and PPCA on real data.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- CHANGELOG: 'Fixed during the release review' subsection (every fix
  since the draft), predict(metrics=/per_column=/return_forecasts=),
  impute(return_imputed=), load(**source_kwargs) documented; the
  unsupported '138 examples executed' claim reworded to what the tests do.
- readme: dependency floors match pyproject exactly (pillow added);
  '1.0 API' wording updated for 1.x.
- api.rst: HypertoolsOfflineError and HypertoolsTrustError listed with
  their autosummary stubs.
- conf.py: gallery pages' furo view/edit links point at the source
  example under examples/ (auto_examples/ is gitignored, so they 404'd);
  stale chemtrails/precog comment fixed.
- tutorials.rst + market_sectors prose use model='HyperAlign' like the code.
- notebooks (sources only; re-executed separately): 'hyper' marked as a
  deprecated alias (analyze, plot); Normalize(mode='isotropic') section
  (manip); alignment illustration restored (align); legend= form for the
  hue demo (text); hyp.load('wikipedia:...') replaces the wikipedia-api
  cells (wikipedia_embeddings).
- scripts/execute_tutorial.py scrubs the executing user's home directory
  from stored outputs so notebooks stop leaking developer paths.
- tests/AGENTS.md no longer claims the hierarchy guide's doctests run in
  the suite; hypertools/io/lsl.py numpydoc underline fixed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… into the summary so numpydoc stops warning

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…gs from the 1.1 release review

- forecasters carry min_history (ARIMA from its order); fit raises a
  clear ValueError on a shorter history; the animated forecast schedule
  waits for enough revealed rows, so predict='ARIMA' (and model lists)
  no longer crash under animate= with a statsmodels IndexError.
- a datetime-like t= works inside hyp.plot; predict= collections work on
  MultiIndex frames; dated column-MultiIndex frames draw dates for every
  leaf under ndims=1; forecast_hue= is one value per dataset with a model
  collection; series-mode bundles match hyp.predict's shape.
- panels=: predict+truth in both panel_fit modes; shared mode keeps
  DataFrame index/column names and accepts 3-column frames; nested hue=/
  labels= narrow per panel; ndims>3 draws 3-D panels; save_path is
  normalised and validated up front; plotly panels use the one-shot
  display wrapper.
- ndims=1: per-column fmt lists; date-aware xlim on both backends; no
  'dataset 1' y label; a 3-D ax= with ndims<=2 raises; TimedeltaIndex is
  drawn in a readable unit.
- a trailing Smooth(center=False) that introduces NaN rows is reported
  as such with the min_periods=1 hint.
- docstrings: font= weights, HyperAnimation.drawn_extent/.save.
- CHANGELOG entries for all of the above.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…d library; gate and test updates

analyze, plot, manip, align, text, wikipedia_embeddings, projectile_kalman
and conversation_trajectories re-executed with scripts/execute_tutorial.py
(home directory scrubbed from stored outputs; no /Users/ path remains in
any tutorial). manip's output-cell gate entry gains the new isotropic
Normalize cell; the nested-hue length test asserts the new message that
names the offending sub-list; tests/test_load_offline.py loses a trailing
blank line. Full suite before these two test updates: 5171 passed, 2 failed
(these two), 19 skipped.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…xecuted notebook

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ation)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ates Windows' transient access-denied

Every Windows job on PR #286 (run 34014888620) failed on the same ten
tests: the widened native-usage scanner compared 'docs\tutorials\x'
against its forward-slash allowlist and roster, and the URL cache's
os.replace raised PermissionError (WinError 5) when twelve threads
replaced one entry at once. The scanner now reports POSIX-separated
relative paths on every host; the cache retries the rename briefly and
accepts a concurrent writer's identical file. macOS and Linux jobs were
green on the same commit.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… (Windows paths, Windows os.replace, detached verification pipeline)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning

Copy link
Copy Markdown
Member Author

Second review pass on this branch (2026-09-06): the cloud ultrareview of the PR diff, then nine local reviewers over the full v1.0.0..HEAD diff (plot, io/tools/manip, predict/impute/align/core, packaging/CI, documentation) and item-by-item verification of #284 and #285. Every confirmed finding is fixed on the branch with a regression test; the fixes are listed in the CHANGELOG's "Fixed during the release review" subsection. The commits since the original report:

  • f116dd7 module-scope imports in the backtest loops (the ultrareview nit)
  • f1a1e09 duplicate metrics=, holdout=True, t=0 message, warning attribution, ragged return_score
  • e47968f three spy/fake-object tests rewritten against real observables; HypertoolsOfflineError exported
  • a59f2e2 native-usage scanner covers every example and tutorial, with the ax= allowlist and output-cell gates the issue described
  • ef887ca 32 plot/io/text fixes: offline=True was still opening connections, yahoo: dates east of UTC, synthetic seed types, short palette= lists (a 1.0 regression), dataset_fade= under a continuous hue, loop=True rotations, companion= under serial order, title_wrap on dynamic titles, plotly newline titles, text2mat on a flat list of strings
  • 1be634d fit() returns self on the manipulator and aligner bases
  • f0e56c9 documentation: changelog completeness and accuracy, readme floors, API exceptions, gallery edit links, tutorial prose and code
  • 384fe99 ARIMA under animate= (per-model min_history), datetime t= inside plot, predict= collections on MultiIndex frames, panels= with truth=/labels/nested hue, series-mode fmt=/xlim=/y label, TimedeltaIndex
  • 00e5a00, f775b37 eight tutorials re-executed on the reviewed library, home paths scrubbed from stored outputs
  • 9340300 Windows: POSIX paths in the scanner, retrying os.replace in the URL cache
  • 63aa9cf, d20fdde session and project notes

Local verification on the final library: full suite 5171 passed / 0 failed (after the two stale expectations were updated), ruff clean, sphinx -W over the full gallery with zero warnings, the example smoke gate 344 passed. PR CI on the branch head d20fdde: every job green (release-gate skipped by design on a branch).

After merge the release is re-cut from the merge commit; RELEASE_CHECKLIST.md now describes moving the draft tag, republishing the gallery namespace and replacing the draft release assets.

jeremymanning and others added 10 commits September 6, 2026 03:39
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The matplotlib backend draws 's--' as a smoothed line artist (which
carries the legend label) plus a markers-only artist at the raw sample
points labelled _nolegend_, so the legend handle showed only the dashes
(reported from the 1.1 feature tour, section 9.2). The line artist now
carries the marker with markevery=[]: its legend handle shows marker and
line while it still draws no markers along the interpolated vertices.
Pixel-level regression test in tests/test_plot_fmt_split_legend.py.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…data)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…it under return_model, quiet third-party reducer warnings

Reported from the 1.1 feature tour (section 9.8): a three-panel grid came
out 2x2 with a hole and, in a 9x3.2 in figure, each square 3-D axes
shrank to the short cell height; the cell also printed fifteen warnings.

- panels=True picks the grid from the figure's aspect ratio and prefers
  a grid with no spare cell (three panels form a row; four 2x2; six 2x3;
  five still 2x3 with one hidden cell). Explicit grids are unchanged.
- return_model=True reuses the pipeline analyze() fitted for the figure
  (the cluster stage, which runs on the reduced scores, is appended as a
  fitted step) instead of refitting every stage, so a UMAP/Isomap plot,
  and every panel grid built with it, fits and warns once.
- a seeded UMAP passes the n_jobs=1 umap forces anyway, so umap stops
  warning about a seed hypertools injected; a caller's own n_jobs= still
  reaches umap (and its warning).
- Isomap fits silence scipy's SparseEfficiencyWarning burst from
  sklearn's internal graph completion; sklearn's own connected-components
  data warning still reaches the user.
Seventeen new tests; two older panel tests updated to the new grid rule
(the hidden-spare property now covered with five panels).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…pan the whole scene

- The truth curve keeps marker='o' with markevery=[] so its 'truth'
  legend entry is line + marker, no longer identical to the observed
  trace's entry (feature tour 9.11).
- New density.scene_bounds_2d: every 2-D KDE grid covers all datasets'
  padded bounds plus the unit frame square, on both backends, so a wide
  flat cloud's glow fades out instead of stopping in a hard band inside
  the frame (feature tour 9.14).
- Tests for both; projectile_kalman and stock_forecasting tutorials
  re-executed (truth= legends), which also refreshed the stock snapshot
  CSVs by one trading day as the notebook is designed to.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…+ ax=<cell>, per-panel colorbars on matplotlib

- plotly_backend.transplant_panel moves a drawn single-axes figure into a
  make_subplots cell whole: traces, the 2-D axis layout (unit frame,
  hidden ticks, DataFrame-column labels, or axis_scale='data' axes), the
  frame square and labels= annotations re-referenced to the cell, its own
  legend (plotly multiple legends) and its own colorbar beside the cell;
  3-D cells back the camera off so the cube stays inside a narrow cell.
- make_panel_grid reserves a gutter beside every cell for those legends
  and colorbars (default-sized grids widen by it; explicit size= verbatim).
- hyp.subplots(backend='plotly') returns the grid figure plus PlotlyCell
  handles that hyp.plot(..., ax=cell) draws into (title= becomes the cell
  title); several cell calls display the grid once per notebook cell.
- matplotlib: a colorbar drawn into a caller-supplied ax= (every panels=
  cell, every hyp.subplots axes) uses fig.colorbar(ax=...) instead of the
  figure-widening placement, so panels no longer stack their colorbars
  over the last panel or trip tight_layout warnings.
- Tests: tests/test_subplots_plotly.py (new), 11 more in
  tests/test_plot_panels.py. CHANGELOG entries; RELEASE_CHECKLIST.md
  brought up to date (suite size, Colab tour smoke, announce step).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…id resolution, sparse ARIMA orders, plotly panel titles/fonts/left colorbars

- panels= on matplotlib under an active plotly preference built its grid
  with subplots()' new backend='auto' default; the grid is now explicitly
  matplotlib.
- 2-D density grids pad by four kernel widths past the data instead of
  spanning the whole scene: the glow still fades out inside the grid, and
  a small cloud beside a 10,000x larger one keeps its resolution (the
  scene-wide grid sampled it to all zeros).
- ARIMA.min_history_for accepts statsmodels' sparse lag orders
  (order=([1, 3], 0, 0)), counting the highest lag, as the fitter does.
- plotly panels: titles go through the single-axes title path (newlines,
  title_wrap=, title_kwargs=) and become cell annotations; the panel's
  font= travels with its legend and becomes the grid default; a
  location='left' colorbar stays on the cell's left; drawing into a 3-D
  cell twice keeps the earlier labels= annotations.
- Tests for each; a rendered-grid test now checks ink in both cells.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ly cell title placement/replacement, per-cell fonts

- predict_new only holds the refit path to the fit-time minimum history;
  a model with an applier (ARIMA) reuses its learned parameters on any
  context the new data offers.
- plotly cells: the title annotation follows the title's own x/y/anchors
  mapped into the cell, is replaced (not stacked) when the cell is drawn
  into again, and reserves the multi-line top margin the single-axes
  path computed; the panel's font= is materialized on the cell's legend,
  title, axis titles/ticks and colorbar under explicit overrides.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
jeremymanning and others added 30 commits September 11, 2026 13:51
…e0cd04

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ed grid's observation vertices

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ensation

tests/plot/test_multiindex_plotly.py::test_plotly_widths_match_the_documented_formula
pinned the UN-boosted Scatter3d request, i.e. 3-D lines rendering at half
the documented 1/1/2 pt (L1, kaleido-measured 0.50x). It now expects the
request times _GL_LINE_WIDTH_BOOST for a 3-D plot, which renders at the
documented width.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…be1c9a

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…tions

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…essages

axis_scale='unit' pins 2-D axes to +/-1.2375 (frame square half-width
1.125) and 3-D to the [-1, 1] cube, not (-1.1, 1.1) (measured on both
backends). The slow-schedule notice fires once fits at two history
lengths are timed. Title TypeErrors name callable titles, which 1.1
accepts. Found by the 2026-09-11 release-document review.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Forty-three entries under 'Fixed during the release review' for the
fixes merged today (plot colours/markers/legends/hover, forecasts and
time axes, calendar-aware forecasting, flat model-spec keys, aligners,
io/offline errors), and two corrected claims: forecast stepping on a
regular calendar, and hue markers carrying alpha.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…be1c9a

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…controls placement

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…phs, callback titles, companion colour, stream clamp)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…lot call

Re-executed tutorials printed '~/hypertools/hypertools/predict/common.py:
435: UserWarning' (a fixed stacklevel=3 landing on library frames), a
calendar step read 'step=<BusinessDay>', and a shuffled index under
hyp.plot warned 'not sorted' twice (two internal paths re-check the
data). Warnings now use external_stacklevel(), steps print as their
alias ('B'), and plot() collapses identical time-policy warnings within
one call via a context-variable scope.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
step=0.04000000000000001 in the projectile_kalman warnings; numeric
steps now use six significant digits. CHANGELOG entry for the warning
attribution fixes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every tutorial notebook re-executed against the merged review fixes
(scripts/execute_tutorial.py; stored mp4s rewritten), including the six
launch notebooks W5 regenerated. stock_forecasting's calendar-time prose
now describes business-day stepping (step='B') with only market holidays
interpolated; its cached price/volume CSVs were refreshed by the run.
Gallery thumbnails regenerated from a clean sphinx build (0 warnings,
51 examples; 323 doctests passed).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The summary cell ended with a bare interactive_viewer() call; the helper
already displays the widget and returns it, so Jupyter showed a second
view and every opened plot rendered twice. The viewer's HTML now starts
with a div, so IPython stops printing 'Consider using
IPython.display.IFrame' above each plot. Tests pin both.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An animation is paced on `_interp_anim_line`'s refined grid, so the drawn
head usually sits between two raw observations. `ForecastSchedule.anchor`
floors to the last observation at or before it and `polyline()` started
there, so the forecast hung back from the line's tip and stood still until
the head reached the next observation, then jumped. Measured on a 2-D
spiral in the [-1, 1] display box: 8 rows over 160 frames lagged by up to
0.90 (45% of the box) and stalled for 23 consecutive frames; 20 rows over
40 frames lagged 0.21 on 24 of 37 frames (maintainer report, 2026-09-11).

The schedule now carries the drawn head. `for_parallel`/`for_serial` read
the animation grid `plot.py` already holds -- not a second interpolation --
at the same `end`/`shown` they already compute, and `polyline()` replaces
its first vertex with it. The predicted points keep their absolute
positions, so `t=` still counts raw steps on from the last observation,
`pin_ramp`'s exact x ramp is untouched, and the vertex count is unchanged.
`to_display` maps heads through the full affine; head POSITIONS are raw row
indices, so they carry through unchanged. A `hue=`/`cluster=` regrouped
reveal passes no grid and keeps the raw-row anchor.

Covered for 1-D/series, 2-D and 3-D, on both backends, for 'parallel',
'serial' and 'window', and for the `forecast_trail=` fan (each retained
forecast starts at the head of the frame it was fit at). The docs, the
gallery example, the tutorial notebook and its generator all described the
old behaviour as the contract; each is corrected.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`post_build.GIF_REPLACEMENTS` registers
`sphx_glr_animate_forecast_thumb.gif` and the file ships, but the stem was
missing from this script's `MPL_ANIMS`, so nothing regenerated it -- the
thumb could only go stale. Found while regenerating the animated-forecast
clips after the anchor fix (the example animates 60 observations over 180
frames, so its old forecast trailed the drawn head by up to 10.7% of the
data diagonal on 134 of its 180 frames).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ast anchor

The first pass only exercised `animate=True`, which builds the head table
through `for_parallel`; `for_serial` (used by `animate='serial'` and
`order='serial'`) and `MultiModelSchedule`'s head forwarders were new code
with no coverage. Adds the three time-progressing styles on both backends,
and `predict=['Kalman', 'ARIMA']`, whose forecasts are addressed behind one
flat model-major index.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…tatus trap

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`hue=`/`cluster=` cut a dataset into one drawn trace per contiguous
category run, so that reveal is expressed in RUNS and builds
`for_regrouped`, which got no grid and kept the raw-row anchor -- leaving
exactly the defect just fixed everywhere else. Measured on a 12-row,
two-category spiral over 81 frames: the forecast trailed the drawn head by
up to 0.51 (mean 0.22) on 75 of them.

`DatasetRevealSchedule` already finds the run holding each dataset's head in
order to count revealed rows; it now keeps WHICH run won and where that
run's polyline ends, so `for_regrouped` can read the head vertex out of the
same per-run arrays the backends draw. `_serial_counts` returns the same
three quantities for the serial sweep, where the last run with anything on
screen holds the head.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The gallery thumbnail and the tutorial video were rendered while a frame's
forecast hung off the last raw observation behind the drawn head. That
example animates 60 monthly observations over 180 frames (grid stride 4), so
its forecast trailed the head by up to 10.7% of the data diagonal on 134 of
its 180 frames.

Rebuilt by the verification pipeline: `sphinx -W` html (0 warnings, 51
gallery examples) -> scripts/generate_gallery_thumbs.py -> re-execution of
docs/tutorials/animate_forecast.ipynb against the real cached archive (three
60x6 regions, not the synthetic fallback). The notebook's only content change
is the `execution` timestamps on five code cells; sources and stored outputs
are byte-identical, because the animation is written to the mp4 rather than
embedded.

Measured on the example's own configuration after the rebuild: 87 live
forecasts across 29 sampled frames, every one starting exactly on a drawn
endpoint (worst distance 0.000000000).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant