Accept CancelTimer for a timer that fired during a workflow task

[cameronhotchkies]

What changed?

The in-process test server (temporal-test-server) now accepts a CancelTimer command for a timer that fired in real time while the current workflow task was still in progress, instead of failing the workflow task completion with INVALID_ARGUMENT "invalid history builder state for action".

• processCancelTimer: when there is no state machine for the timer, it searches the buffered events for that timer's TIMER_FIRED event, removes it, and records a TIMER_CANCELED event in its place (new StateMachines.cancelFiredTimer helper, using the fired timer's startedEventId), mirroring the real server's AddTimerCanceledEvent, which calls GetAndRemoveTimerFireEvent on the buffered TimerFired and still writes a TimerCanceled.
• unhandledCommand: a buffered TIMER_FIRED event that a CancelTimer command in the same response replaces no longer counts as an unhandled new event, so a workflow completion command can be accepted alongside the cancel. This matches the real server, where handleCommandCancelTimer clears the handler's buffered-events flag and the completion command (validated to be the last command) checks it afterwards.

Why?

A timer that fires while a workflow task is in progress has its TIMER_FIRED event buffered until the task completion (and its state machine removed from the timers map). A worker that still sees the timer as pending can respond with CancelTimer, which the test server rejected — and kept rejecting on every retry, leaving the workflow task outstanding forever and wedging the workflow (see #2606 for the stuck histories). This affects tests using TestWorkflowEnvironment that start a timer and cancel it, with the cancel racing the timer firing during a long-running workflow task.

The buffered TIMER_FIRED must be removed rather than committed ahead of the TIMER_CANCELED: the SDK's TimerStateMachine has no transition for a TIMER_FIRED in the CANCEL_TIMER_COMMAND_SENT state, so committing both events would break replay determinism. The removal is safe because buffered events carry no real event id yet (ids are assigned at save time).

Breaking changes?

None. All changes are confined to the test server's internal classes (io.temporal.internal.testservice), which are not part of the public API.

Server PR

None. The change aligns the test server with existing real server behavior; no coordinated server update is needed.

Testing

• New CancelFiredTimerTest in temporal-test-server/src/test/java/io/temporal/testserver/functional/ reproduces the race deterministically at the gRPC level (a short real-time timer fired while a workflow task is in progress; time skipping stays locked by an outstanding activity so the timer only fires in real time):
  • cancelFiredTimerWithoutWorkflowCompletion fails with the exact invalid history builder state for action error without the fix, and with it verifies the buffered TIMER_FIRED is replaced by TIMER_CANCELED and the workflow still progresses and completes on the next workflow task.
  • cancelFiredTimerWithWorkflowCompletion fails with UnhandledCommand without the fix, and with it verifies [CancelTimer, CompleteWorkflowExecution] is accepted in one response.
• Both tests were verified to fail on main and pass with the fix.
• ./gradlew :temporal-test-server:test, ./gradlew :temporal-testing:test, the timer-related temporal-sdk workflow tests (TimerTest, MultipleTimersTest, TimerCallbackBlockedTest), and ./gradlew spotlessApply all pass.

Fixes #2606

[CLAassistant]

All committers have signed the CLA.