UI tests: fix six order-dependent flakes and cut 5 s off the suite - #14591
Merged
Conversation
Flakes from the last two days of CI runs (first-attempt failures): - FixCommonErrorsUnfixableErrorsTests / SpeechToTextPostProcessorTests short-duration tests: SplitManagerTests zeroes Se.Settings.General.MinimumBetweenLines in nearly every test and never restores it; the next MainView host mirrors 0 into libse's Configuration.Settings, and FixShortDisplayTimes then extends lines the scenarios rely on being unfixable. Restore in SplitManagerTests, and pin the libse gap in both victims. - UndoRedoManagerTests.CheckForChanges_OverlappingTick: the first tick was queued with Task.Run and starved behind blocked pool threads (the pool is ~40-130 threads deep by then); use a dedicated thread. - SpellCheckPlayCurrentLineTests: with no dictionaries (the CI runner) Initialize posts the get-dictionaries dialog at Background priority and the null service provider throws inside that job, on whatever test pumps next. Point the dictionaries folder at an empty dir so the local run takes the CI path, and seed one dictionary entry so no dialog is posted. - MainMenuKeyboardActivationTests bare-Alt drop-down: wait for the first header to hold focus before sending Down. - TimeCodeUpDownNegativeTests: settle the dispatcher after Focus(). - AudioVisualizerOriginalTextTests: the window was never closed. Speed: - LibMpvEventLoopTests.EventLoop_ForwardsMpvWarningsAndErrors: the missing file had a .wav extension, so LoadFile polled the duration for 2.5 s after the failed open (15 s in one run, behind the starved pool). A .mp4 name takes it to ~30 ms. - FontCollectorEmbeddedMatchTests round trip: Assert.Equal over an IEnumerable<byte> of a ~1 MB font took ~3 s; span compare is ~25 ms. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Triage of the first-attempt failures in the last ~60
tests.ymlruns (via the "Flaky test run" annotations), plus an after-each-test probe run locally that logged libse settings, thread-pool size and the focused window after every test.Flakes fixed
FixCommonErrorsUnfixableErrorsTests.Scan_ReportsErrorsThatCouldNotBeFixedSplitManagerTestssetsMinimumBetweenLines.Milliseconds = 0in nearly every test and never restores it; the next MainView host mirrors 0 into libse, and the "unfixable" 300 ms line becomes extendable by 20 msSpeechToTextQualityReportTests.Fix_FixShortDuration_RunsAfterSplitLinesUndoRedoManagerTests.CheckForChanges_OverlappingTickTask.Runstarved behind blocked pool threads (pool is 40-130 threads deep by then); now a dedicated threadSpellCheckPlayCurrentLineTests.*(cleanup failures)Initializeposts the get-dictionaries dialog at Background priority → null service provider throws inside the posted job. Reproduced locally by pointingSe.DictionariesFolderat an empty dir; a seeded dictionary entry skips the dialogMainMenuKeyboardActivationTests.BareAlt_ClosesTheMenu_WhileFocusIsInsideAnOpenDropDownTimeCodeUpDownNegativeTests.SteppingAcrossZeroUpdatesSignAndCaretAlso closes the window
AudioVisualizerOriginalTextTestsleft open (the only unclosed window in the suite).Not touched:
BurnInWindowTestsfailures were both on one PR branch (a real regression there, since fixed).Speed
LibMpvEventLoopTests.EventLoop_ForwardsMpvWarningsAndErrorsFontCollectorEmbeddedMatchTests.EmbeddedFontRoundTrip_DecodesToLoadableTypefaceNothing was removed; the remaining slow classes (
MultipleReplacePreviewTests~9 s on a real 250 ms timer,MainReadOnlyOriginalTests~8 s of MainView hosting) are legitimately waiting.Verification
Full
UITestslocally: two runs before the change failed one test each (menu drop-down; STT short duration), two runs after are clean (4894 passed, 1 skipped).🤖 Generated with Claude Code