Skip to content

optimization: Build the G-code preview in C++ - #4481

Open
alex-pres wants to merge 21 commits into
LinuxCNC:masterfrom
alex-pres:pr-render-preview-cpp
Open

alex-pres wants to merge 21 commits into
LinuxCNC:masterfrom
alex-pres:pr-render-preview-cpp

Conversation

@alex-pres

@alex-pres alex-pres commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Follows PR##4293

Adds C++ GCodeRenderer, which builds the whole preview - transform, arcs, taps, suppression, vertices, extents, lengths and event records - during gcode.parse and hands it over once, replacing the Python rendering engine. C++ handles Vec[9] -> X,Y,Z transform completely, following all interpretator details, and Python renders it into OpenGL using shaders.

Parsing is ~2.9x faster on move-shaped programs and peaks at less memory; and specially for arcs, the last per-move Python call kept in code, go from 1.58 s to 0.075 s at 1.28M segments.

Even without speed benefit it is anyway better because resulting code is much more clean and easy to follow. Previous code was hard to read, understand and test. Right now we can test GCodeRenderer engine directly, it is independent of presentation.

Benchmark results, compare to current master

Headless, fractal-1M.ngc (1,000,149 moves):

master this branch
parse+build 2.705 s 0.732 s (3.7x)
total load 2.712 s 0.745 s
peak RSS 166.8 MB 163.5 MB
VBO bytes 20,003,000 20,003,000
CPU geometry 28,004,200 28,004,200
extents identical identical

Byte-identical geometry out, ~3.7x less time to produce it.

Results could be much better if ngc file use arcs (current master calculates arcs in Python which was bad idea, after moving to C++ it is 100x faster).

@alex-pres
alex-pres marked this pull request as draft August 27, 2026 15:50
@alex-pres
alex-pres force-pushed the pr-render-preview-cpp branch from bd037e7 to c0bf905 Compare August 27, 2026 16:01
@alex-pres
alex-pres marked this pull request as ready for review August 27, 2026 16:06
@BsAtHome

Copy link
Copy Markdown
Contributor

What are all those json files doing in there? It seems you want to do some "bake" test, but I did not see a test{.hal,.sh,} in that directory. Did I miss it?
You also need to disable any test(s) that cannot run in the headless CI run. All GUI/Graphics related dependencies are not installed in the headless CI run.

Another point, you are "redecorating" the gcodemodule. Would it be an idea if we move this to pybind11 at the same time? Doing this work twice may be more work and may also be error prone.

@alex-pres
alex-pres force-pushed the pr-render-preview-cpp branch from c0bf905 to 0b8fa5f Compare August 27, 2026 16:20
@alex-pres

alex-pres commented Aug 27, 2026

Copy link
Copy Markdown
Contributor Author

What are all those json files doing in there? It seems you want to do some "bake" test, but I did not see a test{.hal,.sh,} in that directory. Did I miss it? You also need to disable any test(s) that cannot run in the headless CI run. All GUI/Graphics related dependencies are not installed in the headless CI run.

My fault, forgot to add test.sh - done now. JSON files are reference results for related .ngc files from another folder.

No problem with headless - these tests are for C++ routines that do math only - no GUI, no graphics.

Another point, you are "redecorating" the gcodemodule. Would it be an idea if we move this to pybind11 at the same time? Doing this work twice may be more work and may also be error prone.

Any WIP on this or samples?

Yes, I can work on pybind11 conversion, but any existing refrerences to such change in LinuxCNC will be helpful. If any

@alex-pres

Copy link
Copy Markdown
Contributor Author

I agree test files do not look good. I will rework it to easily see what each case is testing.

In time being, I'd appreciate feedback on the rest of PR.

@alex-pres

Copy link
Copy Markdown
Contributor Author

Another point, you are "redecorating" the gcodemodule. Would it be an idea if we move this to pybind11 at the same time? > Doing this work twice may be more work and may also be error prone.

OK, I have found halquery is using pybind now. I will try to rework gcodemodule to use it too in this branch.

@alex-pres
alex-pres marked this pull request as draft August 27, 2026 19:49
@alex-pres
alex-pres force-pushed the pr-render-preview-cpp branch from 0b8fa5f to 7597f74 Compare August 28, 2026 12:44
@alex-pres
alex-pres marked this pull request as ready for review August 28, 2026 12:45
@alex-pres

alex-pres commented Aug 28, 2026

Copy link
Copy Markdown
Contributor Author

Reworked with pybind11 , it is a pleasure to work with. Resulting code is much easier to read and follow! Added some modern C++ which gcodemodule.cc deserve )

Tests reworked too.

Comment thread src/emc/rs274ngc/gcodemodule.cc
Comment thread src/emc/rs274ngc/Submakefile
@alex-pres

Copy link
Copy Markdown
Contributor Author

Last commit is breaking compatibility - returns tuples instead of arrays is not good idea, breaks code on Python side. I will revert this at Monday.

@alex-pres
alex-pres marked this pull request as draft August 28, 2026 18:58
@alex-pres
alex-pres marked this pull request as ready for review August 31, 2026 08:25
@alex-pres

Copy link
Copy Markdown
Contributor Author

Last commit is breaking compatibility - returns tuples instead of arrays is not good idea, breaks code on Python side. I will revert this at Monday.

It was false alarm, problem not caused by this code. Everything seems to be OK and ready for review and testing.

Current live testing results: loading of 418Mb .ngc file (real work file sent by customer, with G2 arcs): 25 seconds to first render.

@grandixximo

Copy link
Copy Markdown
Contributor

418Mb .ngc file (real work file sent by customer, with G2 arcs): 25 seconds

🤤

@alex-pres alex-pres changed the title Optimization: Build the G-code preview in C++ optimization: Build the G-code preview in C++ Aug 31, 2026
@alex-pres

alex-pres commented Aug 31, 2026

Copy link
Copy Markdown
Contributor Author

418Mb .ngc file (real work file sent by customer, with G2 arcs): 25 seconds

🤤

Together with PR #4491 : 12 seconds!

P.S. As said before AXIS [specially AXIS] is using not optimised file editor code that now eats more memory and CPU than graphical preview. So for benchmarking I cut it off. Worth to be optimised too, may be later.

@alex-pres

alex-pres commented Sep 1, 2026

Copy link
Copy Markdown
Contributor Author

And with both #4491 and #4492 - 5.7 seconds to first frame on same file.
Peak RSS after file load: +1.04GB.

@grandixximo

grandixximo commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Could you please fetch master, then merge or rebase and push, just to let the CI run again, and we shall get the shots of all the UIs to make sure nothing regressed.

@alex-pres
alex-pres force-pushed the pr-render-preview-cpp branch from 2564163 to 075e6b8 Compare September 2, 2026 14:27
@alex-pres

Copy link
Copy Markdown
Contributor Author

Could you please fetch master, then merge or rebase and push, just to let the CI run again, and we shall get the shots of all the UIs to make sure nothing regressed.

Done. And squashed to one commit.

@BsAtHome

BsAtHome commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

And the screenshots seem to be similar to what we see elsewhere.

Comment thread src/rtapi/rtapi_string.h Outdated
Comment thread lib/python/rs274/glcanon.py Outdated
Comment thread lib/python/rs274/glcanon.py Outdated
Comment thread lib/python/rs274/glcanon.py Outdated
Comment thread src/emc/rs274ngc/gcode_renderer.cc Outdated
Comment thread src/emc/rs274ngc/gcode_renderer.cc
Comment thread src/emc/rs274ngc/gcode_renderer.cc Outdated
Comment thread src/emc/rs274ngc/gcode_renderer.cc
Comment thread src/emc/rs274ngc/gcodemodule.cc
@alex-pres

Copy link
Copy Markdown
Contributor Author

Finished with changes, hope all mentioned is fixed now

@grandixximo

Copy link
Copy Markdown
Contributor

You might want to look at CI the clang is reporting failures, we have it fail on warnings, gotta clean those up.

@BsAtHome BsAtHome self-assigned this Sep 13, 2026
Comment thread src/emc/rs274ngc/gcode_renderer.cc Outdated
Comment thread src/emc/rs274ngc/gcode_renderer.cc Outdated
Comment thread src/emc/rs274ngc/gcode_renderer.cc Outdated
Comment thread src/emc/rs274ngc/gcode_renderer.cc Outdated
Comment thread tests/gcode-renderer/test_bake.py
Comment thread tests/gcode-renderer/test.sh Outdated
@BsAtHome

Copy link
Copy Markdown
Contributor

BTW, you need to rebase because there is a conflict now.

Adds C++ GCodeRenderer, which builds the whole preview - transform, arcs, taps,
suppression, vertices, extents, lengths and event records - during gcode.parse
and hands it over once, replacing the Python rendering engine. C++ handles Vec[9] -> X,Y,Z transform completely, and Python renders it into OpenGL using shaders.

2 protocols to interact with python: CallbackCanon (compat) and GCodeRenderer (new). Selected when "parse" method called based on canon properties.
This change belongs to the separate pr-perf-rtapi-strlcpy branch and was
merged into this one by mistake. It is submitted as its own pull request,
so drop it here to keep this branch limited to the C++ preview work.
Replaces use_gcode_renderer, a flag a canon with a catch-all __getattr__ could not opt out of.
The renderer transforms in C++ and forwards no offsets, so nothing read them.
set_plane, set_feed_rate, set_spindle_rate and select_plane, with the plane and feedrate they wrote to.
progress_ is a bound method, so it held the canon and its whole program until the next parse.
Drops lo/first_move/xo..wo from the canon contract; the current position reaches a preview as the caller's G53 G0 initcode.
gcode.linecode() constructs again, snapshotting the interpreter the way a
delivery does rather than handing back zeros no later next_line could fill
in. Outside a parse it raises. snapshot_line() is now the one place a
LineCode is filled in, so the two paths cannot drift.
PreviewGeometry.axes - the machine's letters from [TRAJ]COORDINATES - comes
back from every parse; axis_positions(), each vertex as those axes, is built
only when the canon's program_geometry sets want_axis_positions.
A tool offset draws nothing and is already folded into the positions, so
PreviewGeometry.tool_offsets() records each one with the rows it governs.
@alex-pres
alex-pres force-pushed the pr-render-preview-cpp branch from 07e4603 to a46e1c7 Compare September 14, 2026 11:37
Size every realloc and buffer itemsize from the array it measures rather
than a repeated type name, drop the unused posemath include, and take the
reverse rotation from the forward sin and cos instead of rounding the
angle a second time.
@alex-pres

Copy link
Copy Markdown
Contributor Author

BTW, you need to rebase because there is a conflict now.

done

@alex-pres

alex-pres commented Sep 14, 2026

Copy link
Copy Markdown
Contributor Author

Probably need one more step here: I almost implemented TimeEstimateCanon in C++ which does close-to-real time estimate during parse and return it to GUI.
Compared in SIM and got ~98% time match which I believe is quite enough.

It would deserve separate PR but because it changes gcodemodule.cc internals it may be here too. Let me know if separate PR is better choice.

@BsAtHome

Copy link
Copy Markdown
Contributor

Probably need one more step here: I almost implemented TimeEstimateCanon in C++ which does close-to-real time estimate during parse and return it to GUI. Compared in SIM and got ~98% time match which I believe is quite enough.

A 98% match is better than most people can guess, I guess.
Unless we're talking about a long tail that is "off by progress dialogs from a know Redmond software firm pretending it could calculate time". If that is not the case, then we'd be stupid to complain at this stage.

It would deserve separate PR but because it changes gcodemodule.cc internals it may be here too. Let me know if separate PR is better choice.

I think this PR has been boiling quite enough for now. The speed improvement is a good thing. Improving on it can be the next update.

@alex-pres

alex-pres commented Sep 14, 2026

Copy link
Copy Markdown
Contributor Author

I completely agree: this PR is ready to be served.

I'll add one small refactoring : split GcodeRenderer to 2 classes, one that is base and second is preview-targeted. And it will be enough for this branch and ready to merge.

Next PR for time estimate will use the base class to do its own job, and it will be add-on only not touching the preview itself, I think it will be best that way.

And time estimate definitely will work much better than Win 3.11 install progress )

@BsAtHome

Copy link
Copy Markdown
Contributor

And please squash the commits, if you can.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants