Measure what a query costs as the corpus grows - #143
Merged
Merged
Conversation
#21 asks for a measurement before anything is built, so this is the measurement and nothing else. The corpus gains a size parameter, the query runner moves beside it so both benches share one, and a target of its own seeds 1 000, 5 000 and 20 000 notes — kept out of `commands`, whose baseline has to stay comparable and whose corpus is one size on purpose. Each size is timed twice: the whole query, and the same query stopped before the view is built and serialised. The gap between the two is what a lighter wire shape could take away and what a LIMIT could not.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #21, and deliberately not closing it: the ticket asks for a measurement before anything is built, so this is the measurement and nothing else. The numbers and the reading are in the issue; the decision between its three fixes is not mine to make.
What it adds
cargo bench --bench corpus_sizeseeds 1 000, 5 000 and 20 000 notes and times each twice: the whole query, and the same query stopped beforeview::buildand serde. The gap between the two is what a lighter wire shape could take away and what aLIMITcould not — which is the question that decides between the ticket's options.It also prints the serialised payload per size, because half of what #21 is about is what crosses the bridge, and a size is not a duration.
Why a target of its own
commandskeeps one corpus size on purpose — its baseline is the table indocs/architecture.mdand has to stay comparable from run to run. And seeding 26 000 notes of ~13 kB would make a plaincargo benchunusable. So:[[bench]] name = "corpus_size", run on its own.Shape
corpus::build()becomesbuild_of(notes)withbuild()calling it atNOTES, so every existing group is untouched.run_querymoves fromcommands.rsintocorpus.rs: two targets needed it, and a copy in each is the duplication this repository has spent the day removing.#![allow(dead_code)]oncorpus.rs, with the reason: each bench target compiles its own copy of the module and uses a part of it —commandsnever callsbuild_of,corpus_sizenever readsnote_ids.