fix(android): keep a scroll's swipe out of the IME window - #2514
Open
thymikee wants to merge 1 commit into
Open
Conversation
Android is the case the clip exists for beyond iOS: an `adjustPan` or `adjustNothing` activity keeps a window whose recorded bounds already run under the IME, so a plan built from them aims at the keys. The helper now reports the largest input method window beside the application window, in absolute screen pixels like the window next to it, and `scroll` clips its band with the shared rule or refuses when the keyboard owns the surface. An older helper reports no keyboard keys, and a provider-supplied viewport has no IME channel at all. Both read as "nothing to avoid", which is what the shared rule already does with a missing frame; neither turns into a refusal. `UiAutomation.getWindows()` answers with an empty list until the service asks for interactive windows, so the read applies the seam the tree capture already uses rather than depending on a snapshot capture having run first in the same instrumentation; the one-shot fallback below it has no such neighbour. Measured on a Pixel 7 emulator with an `adjust=pan` contact editor, the application window keeps its full 2400px height while the IME window reports `[0,1517][1080,2400]`, and `scroll down` answers with `referenceHeight: 1505`, `keyboardMinY: 1517`, `keyboardAvoided: true` and a swipe ending at 301 instead of starting at 1920 under the keys.
Size Report
Startup median (7 runs, lower is better):
|
|
Member
Author
|
The IME window selection can still leave a scroll inside the keyboard at f63e72f. GestureViewportReader.java:54-61 keeps only the largest input-method rectangle, even though the composer and key plane may be separate windows. A larger lower key plane discards a taller composer’s earlier top edge, so the computed swipe band can overlap that composer. Preserve the earliest top edge of the IME windows intersecting the swipe path and add a two-window regression. Current-head checks pass and the reported Pixel 7 adjustPan run covers the single-window case. This layer also depends on #2503. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Second half of #2500, stacked on #2503 (the shared rule, the iOS runner clip, and the golden table land there). Android had no keyboard input into a scroll's plan at all: an
adjustPanoradjustNothingactivity reports window bounds that already run under the IME, so the swipe was planned into the keys, the surface never moved, and the edge loop reported a stuck container.The helper now reports the largest input method window beside the application window, in the same absolute screen pixels, and
scrollclips its band with the sharedcontracts/scroll-gesturerule or refuses withscroll_keyboard_occludes_surfacewhen the keys own the surface. 10 files, 463 gross lines.Depends on #2503. Do not merge before it.
Behaviour
keyboardAvoidedandkeyboardMinYappear only when the clip ran, matching the Apple side, so the two platforms answer with the same shape.fill/scroll/fillloop.keyboard dismissstays explicit.Validation
pnpm checkgreen on this branch.Live on a Pixel 7 emulator, Contacts editor with
sim={adjust=pan}(fromdumpsys window windows), LatinIME bound:[0,1517][1080,2400]while the application window keeps its full1080x2400;scroll down --json→referenceHeight: 1505,keyboardAvoided: true,keyboardMinY: 1517, swipe1205 -> 301;1920 -> 480, starting under the keys.UiAutomation.getWindows()answers with an empty list until the service asks for interactive windows, so the viewport read applies the seam the tree capture already uses instead of trusting that a snapshot capture ran earlier in the same instrumentation. Without it the read sees no windows at all, fails open, and looks identical to "no keyboard on screen".Regression proof: neutering the clip fails 2 scroll tests and restores green.
Harness note for the next agent
openactivates agent-device's headlessimehelper, which draws no keyboard, and the emulator defaults to no soft keyboard while a hardware keyboard is attached. Bind a real IME afteropen:Restore
com.callstack.agentdevice.imehelper/.TestInputMethodServiceafterwards, or typing determinism in later sessions is silently different.Size
10 files, 463 gross lines, one command family. The Android-shaped rows it satisfies (
adjustPan window with only 88px of 900px left above the IME is refused) are already in the shared table landed by #2503.