Skip to content

docs(guide): how to load many sheets without paying for it (HF-360) - #1759

Open
marcin-kordas-hoc wants to merge 5 commits into
developfrom
docs/perf-guide-bulk-load
Open

marcin-kordas-hoc wants to merge 5 commits into
developfrom
docs/perf-guide-bulk-load

Conversation

@marcin-kordas-hoc

@marcin-kordas-hoc marcin-kordas-hoc commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

Context

docs/guide/performance.md says nothing about loading. The page covers useColumnIndex, address-mapping policies, lazy-transformation cleanup and suspending recalculation, and stops there — so the one decision an integrator makes before any of that advice applies, how to get the data in, is undocumented.

It matters more than it looks. With a dense cross-sheet dependency graph, loading sheets one at a time is quadratic: as an example, on the released 3.4.0 with 500 cross-referencing sheets the per-sheet loop takes over a hundred times as long as a single buildFromSheets call.

This adds a "Loading multiple sheets" section and two subsections for the findings that came out of the same investigation:

  • Build once. buildFromSheets against a per-sheet addSheet + setSheetContent loop, with the mechanism: every setSheetContent recalculates the loaded cells that depend on the sheet it just filled, so the work grows with each step. addSheet adds to this whenever the loaded formulas already point at the sheet being added, which is why pre-registering the sheet names removes about half the cost and not the growth.
  • Load order does not matter. A formula referencing a sheet that does not exist yet evaluates to #REF! and is repaired when that sheet arrives, with no re-parse. This shipped in 3.1.1 and is documented nowhere. It also means ordering the inserts by dependency is not a fix for the cost above — it holds only while every reference points one way.
  • Pass an instance, not the class. An integration handed the HyperFormula class calls buildEmpty and then adds sheets one at a time, which is the slow path by construction.

The section also states that reads throw while the evaluation is suspended, and links to batch operations — that is what turns a batch wrapped around a load loop into a crash in a host application that renders from the engine.

How did you test your changes?

Documentation only, no production code touched.

  • Both code samples were executed against the current develop source rather than written from memory. Reading Sheet1!B1 after the first sample gives 10; reading Hub!A1 through the second gives #REF!, then 1 after addSheet('Later'), then 42 after setSheetContent — the values the sample's comments claim. The samples themselves print nothing; the reads were added to run them.
  • The #REF!-repair claim was checked twice: the behaviour on the released 3.4.0, and the mechanism still present on develop (SheetMapping placeholder handling, isPlaceholder).
  • The performance claim comes from a benchmark of 500 sheets x 60 x 10 with 49,450 cross-sheet references, all pointing forward, medians of five interleaved runs per mode. Every run is gated on a correctness check that resolves one reference row per sheet — 4,945 of them, a tenth of the total — and no run was counted unless the gate passed. The naive loop measures 168,085 ms against 1,183 ms for one buildFromSheets.
  • grep over docs/guide/ confirmed the placeholder-sheet behaviour appears in no other page, so this does not contradict or duplicate an existing statement.
  • The frontmatter gains buildFromSheets and addSheet as search tags. The page keeps ten tags, which is the maximum every other guide page observes.

Bugbot's findings and a self-review pass are applied in 5f4513b: the tag count, the duplication with the page's own suspension section, the 3.1.1 version note on the placeholder behaviour, a noun-phrase heading, one cell-value convention across the samples, and dropping a general claim about third-party integrations in favour of the condition that actually matters.

Types of changes

  • Breaking change (a fix or a feature because of which an existing functionality doesn't work as expected anymore)
  • New feature or improvement (a non-breaking change that adds functionality)
  • Bug fix (a non-breaking change that fixes an issue)
  • Additional language file, or a change to an existing language file (translations)
  • Change to the documentation

Related issues:

  1. HF-360 — describe the optimal initialization strategy in the docs

Checklist:

  • I have reviewed the guidelines about Contributing to HyperFormula and I confirm that my code follows the code style of this project.
  • I have signed the Contributor License Agreement.
  • My change is compliant with the OpenDocument standard.
  • My change is compatible with Microsoft Excel.
  • My change is compatible with Google Sheets.
  • I described my changes in the CHANGELOG.md file.
  • My changes require a documentation update.
  • My changes require a migration guide.

🤖 Generated with Claude Code


Note

Low Risk
Documentation-only change with no runtime or API behavior modifications.

Overview
Adds a Loading multiple sheets section to the performance guide so integrators know how to initialize multi-sheet workbooks without quadratic recalculation cost.

It recommends buildFromSheets over per-sheet addSheet + setSheetContent, explains why incremental loads get slower as cross-sheet dependencies grow (including addSheet dirtying when formulas already point at the new sheet), and points runtime loaders at batch operations plus avoiding reads while evaluation is suspended. Subsections cover that load order is flexible since 3.1.1 (#REF! with live repair when a missing sheet arrives), that dependency-ordered inserts do not fix the incremental cost, that one engine instance is required for cross-sheet formulas, and that integrations should receive a pre-built instance rather than the class so they do not rebuild via the slow path.

Frontmatter search tags buildFromSheets and addSheet are added.

Reviewed by Cursor Bugbot for commit f9326a9. Bugbot is set up for automated code reviews on this repo. Configure here.

marcin-kordas-hoc and others added 2 commits September 3, 2026 09:09
…-15559)

Loading sheets one by one is the slowest way to get data into the
engine, and the guide never said so. Every setSheetContent call
recalculates the loaded cells that depend on the sheet it just filled,
so with cross-sheet references the cost of a per-sheet loop grows with
every step; a customer case measured it at over a hundred times the
cost of a single buildFromSheets call for 500 cross-referencing sheets.

The section states the rule, explains the mechanism, notes that
pre-registering sheet names only removes the extra recalculation
addSheet triggers and not the growth itself, and points to batch
operations for the case where the data is not known upfront.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…OT-15559)

Three more findings from the same customer case, all of them things the
guide never said:

- A formula that references a sheet added later is repaired when that
  sheet arrives, without re-parsing, so load order is free. It also
  means ordering the inserts by dependency is not a fix for the cost of
  incremental loading: it holds only while every reference points one
  way.
- Reads throw while the evaluation is suspended, which is what turns a
  batch around a load loop into a crash in a host application that
  renders from the engine.
- Integrations that accept the HyperFormula class build an empty engine
  and add sheets one at a time; hand them an instance instead.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 4dd93e4. Configure here.

Comment thread docs/guide/performance.md
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Sep 3, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
hyperformula-docs f9326a9 Commit Preview URL

Branch Preview URL
Sep 11 2026, 03:27 AM

- Drop the `loading` tag: every other guide page stops at ten tags and
  this page was the only one at eleven. The word is already in the
  heading it would index.
- Don't restate what "Suspending automatic recalculations" further down
  the same page already says; link to it instead.
- Attribute the doubling to the measurement it comes from rather than
  stating it as a property of addSheet.
- "Order of loading" as a noun phrase, matching every other heading on
  the page, and it no longer reads as an absolute the next paragraph
  walks back.
- Say that the placeholder-repair behaviour arrived in 3.1.1; without
  it the advice is false on earlier releases.
- One cell-value convention across both samples.
- Don't assert what third-party integrations do in general; state the
  condition and what to do about it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown

Performance comparison of head (f9326a9) vs base (c920375)

                                     testName |    base |    head | change
--------------------------------------------------------------------------
                                      Sheet A |  504.17 |  508.74 | +0.91%
                                      Sheet B |  164.81 |  158.03 | -4.11%
                                      Sheet T |  142.63 |  137.81 | -3.38%
                                Column ranges |  528.31 |  518.85 | -1.79%
                                Sorted lookup | 15800.9 | 14915.8 | -5.60%
Sheet A:  change value, add/remove row/column |   15.77 |   16.72 | +6.02%
 Sheet B: change value, add/remove row/column |  154.54 |  142.79 | -7.60%
                   Column ranges - add column |  165.96 |  169.97 | +2.42%
                Column ranges - without batch |  496.58 |  500.24 | +0.74%
                        Column ranges - batch |  124.05 |  128.68 | +3.73%

"it has to call buildEmpty" asserts a necessity the API does not
impose — a library handed the class could collect the data and call
buildFromSheets itself. What is true is narrower: the engine gets built
on the library's terms, and one that receives sheets one at a time adds
them one at a time. Passing a prepared instance is what removes the
choice.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@marcin-kordas-hoc marcin-kordas-hoc changed the title docs(guide): how to load many sheets without paying for it (HOT-15559) docs(guide): how to load many sheets without paying for it (HF-360) Sep 11, 2026
@qunabu

qunabu commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

The HOT-15559 investigation this section comes from ruled that route
out, and the guide left it out. Anyone optimising a slow multi-sheet
load will reach for sharding into several instances, so the guide
should say what it costs: a formula can only reference a sheet in its
own instance, and pointing at one held elsewhere is #REF!.

Measured, not carried over from the investigation's own citation:
instance A holding Sheet1!A1 = 10, instance B evaluating =Sheet1!A1
against it = #REF!, and the same reference inside one instance = 10.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@codecov

codecov Bot commented Sep 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.32%. Comparing base (c920375) to head (f9326a9).

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff            @@
##           develop    #1759   +/-   ##
========================================
  Coverage    97.32%   97.32%           
========================================
  Files          195      195           
  Lines        15739    15739           
  Branches      3390     3461   +71     
========================================
  Hits         15318    15318           
+ Misses         421      413    -8     
- Partials         0        8    +8     

see 5 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants