Skip to content

feat(frontend): add a panel that opens a workflow's own record (Hackathon UP) - #8525

Draft
Brb6754 wants to merge 1 commit into
apache:mainfrom
Brb6754:feat-spy-panel
Draft

feat(frontend): add a panel that opens a workflow's own record (Hackathon UP)#8525
Brb6754 wants to merge 1 commit into
apache:mainfrom
Brb6754:feat-spy-panel

Conversation

@Brb6754

@Brb6754 Brb6754 commented Sep 12, 2026

Copy link
Copy Markdown

Built for Hackathon UP and submitted as its entry.

Draft on purpose. The panel works end to end, but where a kept run should live
is an open design question and I would rather settle it with the committers
than guess. See the last section.

What changes were proposed in this PR?

A panel, opened from a button in the workspace menu, that shows the user what
Texera already records about a workflow and currently keeps to itself: every
save, and every run.

Nothing here is new data. workflow_version holds one inverse JSON patch per
save, so the whole history of a canvas is recoverable by walking the patches
backwards. A run's results are dropped after thirty seconds, but its runtime
statistics are not: they stay in an Iceberg table with a row per operator per
sample, which is enough to say how many rows went in and out of every step of
every run ever made.

The panel reads both and answers four questions in plain language, which is what
its five views are named after:

  • How does this work? The whole flow as one drawing, laid out by the panel
    rather than taken from the canvas, with each pipe as thick as the rows that
    run through it, so where the flow narrows reads before any number does. A step
    inspector alongside gives one step at a time: what it does, rows in and out,
    its settings in words, and sample rows.
  • What did I change? Every save as a block on a ribbon, coloured by what
    kind of save it was, with the canvas rebuilt at whichever block is picked and
    that save's edits listed in words.
  • What has been tried? Every run read as an experiment rather than a log
    line, so the same settings run five times count as one thing tried. On the
    workflow I developed against, twenty-one runs collapse to four.
  • Why did my results change? Two runs compared step by step, naming the
    first step whose output differs and the edit behind it.
  • Report. The same facts written up as a handover document, asked for by
    hand because it costs a model call.

Two rules hold throughout, and they are the part I would most like reviewed:

  • Every claim is derived and checkable. A model is used only to put
    already-derived facts into prose. It never decides which step diverged, which
    edit caused it, or what any count means. With no model configured the panel
    loses its sentences and nothing else, so it does not become a feature that
    stops working when a key expires.
  • Ordering is not a change. A step returning the same rows in a different
    order is reported as exactly that. The engine splits work between workers
    non-deterministically, and an earlier version of this did blame a join for it,
    hiding the real edit further downstream.

Everything the panel writes is in plain language rather than in the system's own
terms: canvas names instead of identifiers, property paths translated through
the same JSON schema that draws the operator form (so a filter reads
amount > 400, never its JSON), and dates in words. The technical form is kept,
one disclosure down.

Files. One new component under frontend/src/app/workspace/component/spy/,
three added lines' worth of button in the workspace menu, and a new
spy-service that answers it: it reads the database directly, holds no Texera
session of its own, and its single writing route carries the token of the user
who has Texera open, so whoever cannot run a workflow cannot run one from here.

Any related issues, documentation, discussions?

Proposed in #8524, which has the reasoning and the open question in full.
spy-service/README.md documents the modules and every environment variable.

How was this PR tested?

No automated tests yet, which is the other thing I would like guidance on: the
parts worth testing are the patch application and the topological comparison,
and I would rather write them where the committers expect them to live than
guess at a home for them.

What was verified, by hand, against a workflow with 61 saves and 21 runs:

  • The rebuild is exact. All 61 versions rebuild, and each one matches what
    GET /api/version/{wid}/{vid} returns, compared one by one. That check turned
    up a real bug in the process: Texera sometimes writes the same inverse patch
    on several consecutive saves, typically remove /operators/0/operatorProperties/envName,
    which appears because opening a user function's property panel writes the key
    empty and the next save removes it. The first patch applies and the second
    finds nothing to remove, which broke the chain and made everything earlier
    look unrecoverable. Removing what is already gone leaves exactly the document
    the patch describes, so it is tolerated and counted. With that, one workflow
    went from 2 readable versions out of 56 to all 56. Worth flagging on its own:
    restoring an old version has the same exposure.
  • The comparison names the right edit in two prepared cases: a filter
    threshold moved from 100 to 400, and a grouping key moved from segment to
    city where one branch changes and the other does not.
  • Browser check of all five views, the step inspector, the ribbon, and the
    run-and-keep button (9 steps, 2,731 rows kept).

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Claude Opus 5)

The open design question

Comparing two runs row by row needs the rows, and Texera drops them thirty
seconds after the workflow goes idle. This draft keeps a copy as one JSON file
per run on the service's own disk. That is the weakest part of it: it does not
survive a restarted container, it does not respect the workflow's access
control, and nothing ever cleans it up. It should not survive review.

The rows do not need copying at all. When a run finishes they are already in an
Iceberg table. They disappear because WorkflowService.clearExecutionResources
clears them once executionStateCleanUpInSecs has passed. So keeping a run for
comparison is a flag rather than a write: mark the execution as retained and let
the cleanup skip it, the way it already skips per-user warehouses through
WarehouseReadGuard.skipWhileDisabled. A retention policy would be needed
alongside, since nothing would otherwise free the space.

That is likely worth having beyond this panel. "Keep this execution's results"
is something users ask for on its own.

I have left it as it is rather than guessing at the shape of that flag, and
would rather agree it first.

🤖 Generated with Claude Code

https://claude.ai/code/session_01QqWWyuFAFLVswDYKXYyCwx

Texera writes down every save and every run of a workflow and shows the
user neither. `workflow_version` keeps one inverse JSON patch per save, and
the runtime statistics of every run stay in Iceberg long after the results
themselves are dropped. This adds a panel that reads both and answers, in
plain language, the four questions people actually ask about a workflow
they did not build, or built weeks ago:

- How does this work? The flow drawn as one picture, each pipe as thick as
  the rows running through it, with a step inspector beside it.
- What did I change? Every save as a block on a ribbon, and the canvas
  rebuilt at whichever one is picked.
- What has been tried? Every run read as an experiment, so twenty-one runs
  collapse into the four distinct things that were actually tried.
- Why did my results change? Two runs compared step by step, naming the
  first step whose output differs and the edit behind it.

Two rules hold throughout. Every claim is derived from the record and can
be checked; a model only puts those derived facts into words and never
decides anything, so with no model the panel loses its sentences and
nothing else. And a step that returns the same rows in a different order is
not reported as a change, because that ordering comes from how work was
split between workers rather than from any edit.

The panel is opened from a button in the workspace menu. It is served by a
new `spy-service`, which reads the database directly and holds no Texera
session of its own; its single writing route carries the token of the user
who has Texera open.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QqWWyuFAFLVswDYKXYyCwx
@github-actions github-actions Bot added feature frontend Changes related to the frontend GUI docs Changes related to documentations labels Sep 12, 2026
@github-actions

Copy link
Copy Markdown
Contributor

👋 Thanks for your first contribution to Texera, @Brb6754!

If you're looking for a good place to start, browse issues labeled starter-task; they're scoped to be approachable for newcomers.

You can drive common housekeeping yourself by commenting one of these commands on its own line:

  • Issues. Comment /take to assign an open issue to yourself, or /untake to release it. You can find unclaimed work with the search filter is:issue is:open no:assignee.
  • Sub-issues. To link issues into a parent/child hierarchy, comment /sub-issue #5166 #5222 on the parent to attach those children (or /unsub-issue #5166 #5222 to detach them). From a child issue, comment /parent-issue #5166 to set its parent, or /unparent-issue to clear it (the current parent is detected automatically). References may be written as #5166 or as a bare 5166; cross-repository references are not supported.
  • Pull requests (author only). Comment /request-review @user to request a review from someone, or /unrequest-review @user to withdraw that request.

Each command must match exactly: /take this will not work, only /take does. For the full contribution flow, see CONTRIBUTING.md.

@github-actions

Copy link
Copy Markdown
Contributor

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

  • Contributors with relevant context: @PG1204, @yangzhang75
    You can notify them by mentioning @PG1204, @yangzhang75 in a comment.

@Brb6754 Brb6754 changed the title feat(frontend): add a panel that opens a workflow's own record feat(frontend): add a panel that opens a workflow's own record (Hackathon UP) Sep 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Changes related to documentations feature frontend Changes related to the frontend GUI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant