Test fails but doesn't fail as expected (tests/interp/m98m99/12-M99-endless-main-program changed output)

[BsAtHome]

The test in tests/interp/m98m99/12-M99-endless-main-program changed after @grandixximo #4441 merge we get the output:

--- expected.motion-logger	2026-08-26 13:18:16.903654087 +0200
+++ out.motion-logger	2026-09-14 15:41:11.731367523 +0200
@@ -194,6 +194,7 @@
 JOG_ABORT joint=15
 ABORT
 SPINDLE_OFF
+SS_ENABLE
 ENABLE
 COORD
 JOG_ABORT joint=0

The real problem is that the test is marked xfail and that means that it is supposed to fail. However, it has not failed for a very long time and we've all gotten used to the message *** /home/runner/work/linuxcnc/linuxcnc/tests/interp/m98m99/12-M99-endless-main-program: XPASS: Passed, but was expected to fail.

Now the output changed and nobody caught it because CI lets it pass. We told it to let it pass...

Either the program marked xfail does fail and only passes if it fails. Or, we let it pass and it must always pass. This intermediate state we have now is the same as what happened in #4397 and it will happen again and again as long as we let non-failing tests that should fail pass.

The other solution is to disable or remove this test if it does not serve any proper purpose.

[BsAtHome]

BTW, a test that is supposed to fail (using xfail) still needs to test whether the failure mode is exactly the failure expected. That makes things very difficult. Tests should test the result and make the decision on that result. If the result changes, then the test should always be rejected as not passing.

The test in question, tests/interp/m98m99/12-M99-endless-main-program/ is the only one using the xfail marker. It would probably be prudent to remove that functionality completely. The test can do logic negation when necessary. No need for the runtests script to make things complicated by negating negations by negating the negative not negated by negation... (if you still can follow my line of thought)