fix(query): print wide decimals at full precision - #299
Conversation
A DECIMAL needing more than ~17 significant digits printed as a rounded
float: `SELECT CAST('99999999999999999999.99' AS DECIMAL(38,2))` came
out as `1e20` in every output format, while the same query against the
HTTP API returned every digit. Wrong money on screen, with no error and
no exit-code change.
Two independent losses, both on this side. Result rows arrived already
rounded, because they were deserialized into `serde_json::Value`, which
has no arbitrary-precision number. And the Arrow path re-introduced the
same loss on data it held exactly, by rendering each cell to JSON text
and parsing it back into a `Value`.
Rows now carry `hotdata::JsonCell` (0.17.0), which holds a cell's JSON
text, so nothing is decided at parse time and a number is printed with
the digits the service sent. `-o json` still emits it unquoted, so the
output stays a JSON number rather than becoming a string.
`kind()` identifies each cell before the accessors are called, so the
accessors cannot decline; they now panic on that unreachable state
rather than substituting an empty string or list, which would put a
value on screen that the service never sent.
One visible change: a list or struct cell prints on a single line in
`-o json`. Re-indenting it would mean parsing the text, which is what
loses the digits. Scalars are unaffected and the JSON is unchanged to
any parser.
There was a problem hiding this comment.
Reviewed the full diff plus the surrounding code in src/commands/query.rs and src/output/table.rs. Both loss sites are fixed and covered by tests. Two non-blocking inline comments.
Checked and found correct:
-o csv,-o tableand-o jsonall read the cell text, so no render path re-parses throughserde_json::Value.- The composite-on-one-line change does not affect the integration tests.
tests/results_arrow.rsandtests/query_async_polling.rscompare parsed values, not raw text. - The
RawValueprivate-struct hazard described forarbitrary_precisiondoes not reach-o yaml.print_resultaccepts only json, csv and table, andJsonCellappears in no model thatworkspace.rs,jobs.rsorresults.rsserializes to YAML. - CHANGELOG needs no manual edit. Entries are generated from the conventional commit subject, and
CI / changelogpassed.
CI / test was still queued at review time, so the test results are unverified here.
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
`truncate_array` formats the short and head/tail branches separately, so a regression could land on one alone; assert a wide decimal survives both, and that the elision counts every element. The `encode_cell` doc said a wide result does not allocate per value. It copies the cell's text out of the shared buffer, so it allocates one String per cell; only the encode buffer is reused.
There was a problem hiding this comment.
Prior nits addressed in 0a5950a. The encode_cell doc comment now matches the code. truncate_array_keeps_element_precision_on_both_branches covers both formatting branches, including the 7-element case.
No new findings. CI was still queued or in progress at review time, so this approval does not assert the test result.
The bug
A DECIMAL wide enough to need more than ~17 significant digits printed as a rounded float, in every output format, with no error and no exit-code change:
The
exactcolumn is the same value cast to text, in the same row — proof the digits reached the CLI intact. The HTTP API returns99999999999999999999.99for the same query. So the platform was right and the screen was wrong, and checking against the API made it more confusing rather than less.DECIMAL is where money lives, so this printed wrong money into
-o csvexports and-o jsonpipelines.Cause
Two independent losses, both client-side.
serde_json::Value, which has no arbitrary-precision number variant — anything that does not fit ani64/u64goes throughf64. The digits were gone before the CLI saw them.Decimal128values, rendered each to JSON text, then parsed that text back into aValue— reintroducing the same loss on data it already had right.Integers were unaffected (
Valuehas reali64/u64variants). In practice this is DECIMAL.Fix
Rows carry
hotdata::JsonCell(new in 0.17.0), which holds the cell's JSON text. Nothing is decided at parse time, so a number prints with the digits the service sent.-o jsonstill emits it unquoted — it stays a JSON number, not a string, sojqpipelines are unaffected.While translating the render paths, three
unwrap_or_default()fallbacks were replaced withexpect.kind()has already identified the cell before the accessor is called, soNoneis unreachable — and substituting an empty string or empty list would put a value on screen the service never sent, which is the same defect this PR exists to remove.One visible change
A list or struct cell now prints on a single line in
-o json:Scalars are unaffected, and the JSON is identical to any parser. Re-indenting would require parsing the cell text, which is the step that loses the digits. Pinned by
json_output_writes_a_composite_cell_on_one_lineso it is a documented fact rather than a surprise.Verification
Manual, against a local server, with the fixed binary:
moneymatches theexacttext cast in all three, and the raw API body agrees.Both loss sites are covered by tests: the inline path, and the Arrow path via a truncated 200k-row result followed through
/results/{id}. The previousa_wide_decimal_is_rounded_a_known_limitation, which pinned the lossy behaviour, is replaced bya_wide_decimal_keeps_every_digit.582 tests pass against the published 0.17.0. clippy clean, fmt clean.