From fba78ccbd68a86b158b51dff099fbea998c08612 Mon Sep 17 00:00:00 2001 From: GenericJam Date: Fri, 11 Sep 2026 18:23:51 -0600 Subject: [PATCH] MOB-183: mob.deploy verifies release bootfile on Android, not just ERTS MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Non-native mix mob.deploy on Android already had a per-app OTP-runtime guard that ran `ls .../otp/erts-*/bin/erl_child_setup` on the target and refused a push when the file was missing. That covered "no OTP at all" but missed the partial-runtime state a reporter hit: ERTS was present but `otp/releases/*/start_clean.boot` was absent. The check passed, the deploy reported "Deployed to 1 device(s) / Apps restarted", and the app crash-dumped at boot with: {'cannot get bootfile', .../otp/releases/29/start_clean.boot} That's the state a device lands in when it was installed via `adb install` without a matching `mix mob.deploy --native`, or when a prior --native pushed ERTS but was interrupted before writing the release tree. Third sighting of this failure family (see mob_dev#54). The probe now checks BOTH globs in one run-as-wrapped `ls` and classifies the output. The classifier is public (`MobDev.Deployer.classify_android_runtime_ls/3`) so failure-mode matrix is tested without a device — pattern used for adb / simctl parsers elsewhere in this file. Classification order: run_as_unavailable → erts_missing → bootfile_missing. erts_missing wins the both-missing case: provisioning ERTS is what --native does first, and pointing a caller at the bootfile they don't have yet would send them chasing the wrong file. The pre-review revision fell through to `:ok` on the exact adb-shell arm the bug fires on — modern adb-shell (v2, default since Android 7) forwards the inner shell's non-zero exit as the session code, so a partial-runtime device would come back as `{:error, out}` from `run_adb` and skip the classifier entirely. Fixed: `ensure_erts_on_device` feeds `out` into the classifier regardless of status, and the classifier's empty-output branch handles the true transport-failure case as `:ok` (best-effort pass; downstream push failures still surface the real problem). Also recognises three `ls: : No such file` shapes seen in the wild (Toybox default, older Toybox `not found`, GNU coreutils `cannot access`). Anchoring on the exact glob is what lets the classifier distinguish erts-missing vs bootfile-missing in the both-missing case. iOS is not covered by this change; its runtime lives host-side under `~/.mob/runtime/ios-sim//`. Separate follow-up. Decision record: decisions/2026-09-11-android-runtime-check-covers-bootfile.md. Co-Authored-By: Claude Opus 4.7 Claude-Session: https://claude.ai/code/session_01Ezeu2FEZfRajeoL4ywEUDJ --- ...1-android-runtime-check-covers-bootfile.md | 72 ++++++++++ lib/mob_dev/deployer.ex | 136 +++++++++++++++--- test/mob_dev/deployer_test.exs | 98 +++++++++++++ 3 files changed, 283 insertions(+), 23 deletions(-) create mode 100644 decisions/2026-09-11-android-runtime-check-covers-bootfile.md diff --git a/decisions/2026-09-11-android-runtime-check-covers-bootfile.md b/decisions/2026-09-11-android-runtime-check-covers-bootfile.md new file mode 100644 index 0000000..96c3fee --- /dev/null +++ b/decisions/2026-09-11-android-runtime-check-covers-bootfile.md @@ -0,0 +1,72 @@ +# mob.deploy Android runtime check verifies release bootfile, not just ERTS + +- Date: 2026-09-11 +- Status: accepted +- Linear: MOB-183 +- GitHub: mob_dev#54 + +## Context + +`mix mob.deploy` (non-native, Android) already had an `ensure_erts_on_device/2` +guard that ran `ls .../otp/erts-*/bin/erl_child_setup` on the target and +refused a push when the file was missing. That covered the "no OTP at all" +case a caller hits when they've never run `mix mob.deploy --native` on that +device. + +It did *not* cover a partial-runtime state, which the reporter of mob_dev#54 +hit on a physical Android 11: the device had `erts-14.2.5/bin/` populated +(the APK ships those binaries as `.so` in `jniLibs/`), but the release +directory under `.../otp/releases/*/` was empty. `ls erts-*/bin/erl_child_setup` +returned the file, so the check passed. `mob.deploy` reported "Deployed +to 1 device(s) / Apps restarted" and the app crash-dumped at boot with: + + {'cannot get bootfile', .../otp/releases/29/start_clean.boot} + +That's the failure family the issue calls out — "green deploy that yields +a boot-crashing app costs a full diagnosis cycle every time." The check +was too narrow to see the release-dir-missing subset. Third sighting of +this class of bug (sim variant tracked as `app-ii3`). + +## Decision + +Widen the Android runtime probe to verify **both** files in one `ls` +round-trip: + +- `.../otp/erts-*/bin/erl_child_setup` — the ERTS binary (existing check). +- `.../otp/releases/*/start_clean.boot` — the OTP release the emulator + boots from. + +Classification is split out to `MobDev.Deployer.classify_android_runtime_ls/3` +(pure, tested) so the ordering (`run_as_unavailable` → `erts_missing` → +`bootfile_missing`) is asserted against real fixtures rather than mental +model. `erts_missing` wins over `bootfile_missing` in the both-missing case: +provisioning ERTS is what `--native` does first, and pointing a reader at +the bootfile they don't have yet would send them chasing the wrong file. + +The check now recognises three `ls`-missing message variants seen in the +wild: + +- `ls: : No such file or directory` (Toybox, most Android userland) +- `: not found` (older Toybox / minimal shells) +- `ls: cannot access '': No such file or directory` (GNU coreutils) + +Anchoring on the exact glob rather than substring "No such file" is what +lets the classifier distinguish which of the two globs is the one that +failed to expand. + +## Consequences + +- Non-native `mix mob.deploy` now fails with a targeted message and exit + code non-zero when a device is in the release-dir-missing state, instead + of a green deploy followed by a boot crash the user has to diagnose from + logcat. +- The check remains best-effort: if `run_adb` itself errors (adb offline, + device unauthorised), the deploy proceeds and the current downstream + errors take over. Fixing that fallback is a separate concern. +- iOS is not covered by this change. The sim variant (`app-ii3`) has the + same failure family but the runtime path is different (`sim_runtime_dir` + under `~/.mob/runtime/ios-sim//` on the host, not `/data/data/...` + on the device). Follow-up ticket. +- The classifier is public so future adb output shapes can be regressed + against without needing a device — pattern already established in this + repo for `parse_devices_output/1`, `parse_simctl_json/1`, etc. diff --git a/lib/mob_dev/deployer.ex b/lib/mob_dev/deployer.ex index b2368dd..f642a02 100644 --- a/lib/mob_dev/deployer.ex +++ b/lib/mob_dev/deployer.ex @@ -282,41 +282,104 @@ defmodule MobDev.Deployer do end end - # Verify the OTP runtime (erts-X.Y/bin/erl_child_setup) is present on - # the device. Without this, the BEAM can't start — symlinks fail with - # ENOENT, the app crashes immediately. This typically happens when the - # device wasn't connected during a previous `mix mob.deploy --native`. + # Verify the OTP runtime (ERTS binary + release bootfile) is present on + # the device. Without both, the BEAM can't start: # - # Returns :ok if ERTS is present, {:error, message} with a helpful hint - # if missing. + # - Missing `erts-*/bin/erl_child_setup` — symlinks fail with ENOENT, the + # app crashes at BEAM launch. + # - Missing `releases/*/start_clean.boot` — the OTP dir has ERTS but no + # release, so `erl_child_setup` starts and the emulator loads, but + # boot-time reports "cannot get bootfile" and the app dies (MOB-183 / + # mob_dev#54). This is the state a device lands in when the APK ships + # ERTS in jniLibs but the release dir was never pushed via + # `mix mob.deploy --native`. + # + # This typically happens when the device wasn't connected during a + # previous `mix mob.deploy --native`. Returns :ok when both are present, + # {:error, message} otherwise. defp ensure_erts_on_device(serial, pkg) do - # The wildcard must be expanded *inside* the run-as sandbox — `run-as` + # The wildcards must be expanded *inside* the run-as sandbox — `run-as` # itself does not invoke a shell, and the outer adb-shell shell can't # see /data/data//, so a literal "erts-*" gets passed to ls if we - # don't wrap with `sh -c` here. - cmd = - "run-as #{pkg} sh -c 'ls /data/data/#{pkg}/files/otp/erts-*/bin/erl_child_setup' 2>&1" + # don't wrap with `sh -c` here. `run_adb` already sets + # `stderr_to_stdout: true`, so the ls diagnostic lines land in the + # `out` we classify either way. + erts_glob = "/data/data/#{pkg}/files/otp/erts-*/bin/erl_child_setup" + boot_glob = "/data/data/#{pkg}/files/otp/releases/*/start_clean.boot" + + cmd = "run-as #{pkg} sh -c 'ls #{erts_glob} #{boot_glob}'" + + # The MOB-183 case sits on the `{:error, out}` arm: `ls` exits non-zero + # when a glob is missing, `run-as` propagates it, `sh -c` propagates + # it, and modern adb-shell (shell v2, default since Android 7) forwards + # it as the session exit code — so `run_adb` returns `{:error, out}` + # with the ls diagnostic lines still in `out`. We MUST classify that + # `out` too, or the exact bug this ticket fixes falls through silent. + # The classifier's own empty-output branch handles the true transport + # failure case (device disappeared between `list_devices` and here). + {_status, out} = run_adb(["-s", serial, "shell", cmd]) + + case classify_android_runtime_ls(out, erts_glob, boot_glob) do + :ok -> :ok + {:error, :run_as_unavailable} -> {:error, run_as_unavailable_message(serial, pkg, out)} + {:error, :erts_missing} -> {:error, erts_missing_message(serial, pkg)} + {:error, :bootfile_missing} -> {:error, bootfile_missing_message(serial, pkg)} + end + end - case run_adb(["-s", serial, "shell", cmd]) do - {:ok, out} -> - cond do - String.contains?(out, "run-as:") -> - {:error, run_as_unavailable_message(serial, pkg, out)} + @doc """ + Classifies the output of the Android per-app runtime `ls` probe. + + Returns `:ok` when both `erts_glob` and `boot_glob` resolve to a real + file. Returns one of the failure tags in preference order: + + - `{:error, :run_as_unavailable}` — the shell couldn't `run-as` the + package (release build, non-debuggable APK). Nothing under + `/data/data//files/otp/` is reachable to `ls` at all. + - `{:error, :erts_missing}` — no `erts-*/bin/erl_child_setup`. The + BEAM can't launch; the app crashes at ERTS start. + - `{:error, :bootfile_missing}` — ERTS is present but no + `releases/*/start_clean.boot`. The emulator starts, then dies with + "cannot get bootfile". MOB-183. + + Public so the classifier can be tested against captured adb output + without an emulator. + """ + @spec classify_android_runtime_ls(String.t() | nil, String.t(), String.t()) :: + :ok | {:error, :run_as_unavailable | :erts_missing | :bootfile_missing} + def classify_android_runtime_ls(out, erts_glob, boot_glob) do + cond do + # No output to classify — a true adb transport failure (device + # disappeared between `list_devices` and this probe). Best-effort + # pass; downstream push failures still surface the real problem. + out in [nil, "", " "] -> + :ok - String.contains?(out, "No such file") or String.contains?(out, "not found") -> - {:error, erts_missing_message(serial, pkg)} + String.contains?(out, "run-as:") -> + {:error, :run_as_unavailable} - true -> - :ok - end + output_names_missing?(out, erts_glob) -> + {:error, :erts_missing} - _ -> - # adb shell failed entirely — let the deploy proceed and fail later - # if needed; this check is best-effort. + output_names_missing?(out, boot_glob) -> + {:error, :bootfile_missing} + + true -> :ok end end + # A missing glob shows as an `ls: : No such file or directory` + # (or "not found") line — the `` in that message is verbatim the + # glob we passed since it never expanded. Match on the exact glob so + # ERTS-missing vs bootfile-missing don't confuse each other in the + # both-missing case. + defp output_names_missing?(out, glob) do + String.contains?(out, "ls: #{glob}: No such file") or + String.contains?(out, "#{glob}: not found") or + String.contains?(out, "ls: cannot access '#{glob}'") + end + # `run-as` fails silently downstream: every beams/priv/exqlite push shells # out to `run-as #{pkg} tar xf ... 2>/dev/null; true`, where the trailing # `; true` is there to swallow Toybox tar's benign chown-to-macOS-UID @@ -361,6 +424,33 @@ defmodule MobDev.Deployer do """ end + # MOB-183: separate the "release dir missing" case from the "ERTS + # missing" case. Both are provisioned by `mix mob.deploy --native` but + # a device in the bootfile-missing state has ERTS binaries and would + # otherwise pass a check that only looked for `erl_child_setup` — the + # bug the reporter hit was a green deploy that yielded an app + # crash-dumping at boot with "cannot get bootfile" because + # `otp/releases/*/start_clean.boot` was absent. + defp bootfile_missing_message(serial, pkg) do + """ + OTP release missing on device #{serial}. + + The ERTS binaries are present under /data/data/#{pkg}/files/otp/erts-*/ + but the release directory is empty — no /data/data/#{pkg}/files/otp/releases/*/start_clean.boot. + The emulator will load but crash-dump at boot with "cannot get bootfile" + (see MOB-183 / mob_dev#54). + + This is the state a device lands in when the APK was installed via + `adb install` without a matching `mix mob.deploy --native` — or when + a previous `--native` pushed ERTS but was interrupted before writing + the release tree. Provision the release now: + + mix mob.deploy --native --device #{serial} + + Subsequent `mix mob.deploy` runs (without --native) will work normally. + """ + end + # If the Elixir stdlib on the device was installed by a different Elixir version # than the host (e.g. after `asdf` upgrade), regex literals and other stdlib # internals will be incompatible. Detect the mismatch and push updated BEAMs. diff --git a/test/mob_dev/deployer_test.exs b/test/mob_dev/deployer_test.exs index 1219283..1a043d3 100644 --- a/test/mob_dev/deployer_test.exs +++ b/test/mob_dev/deployer_test.exs @@ -367,4 +367,102 @@ defmodule MobDev.DeployerTest do assert Deployer.__sqlite_nif_target__(lines) == "/data/app/x/lib/arm/libsqlite3_nif.so" end end + + # MOB-183: non-native mob.deploy used to report success on a device + # whose per-app OTP release directory was missing; the app then + # boot-crashed with "cannot get bootfile". The old check only verified + # ERTS binaries — this classifier verifies both ERTS and the release + # bootfile, and distinguishes the failure modes. + describe "classify_android_runtime_ls/3" do + @erts_glob "/data/data/com.example.demo/files/otp/erts-*/bin/erl_child_setup" + @boot_glob "/data/data/com.example.demo/files/otp/releases/*/start_clean.boot" + + test "ok when both erts and bootfile exist" do + out = """ + /data/data/com.example.demo/files/otp/erts-14.2.5/bin/erl_child_setup + /data/data/com.example.demo/files/otp/releases/29/start_clean.boot + """ + + assert Deployer.classify_android_runtime_ls(out, @erts_glob, @boot_glob) == :ok + end + + test "run_as failure surfaces before glob checks" do + out = "run-as: unknown package: com.example.demo" + + assert Deployer.classify_android_runtime_ls(out, @erts_glob, @boot_glob) == + {:error, :run_as_unavailable} + end + + test "erts missing when erts glob has no match" do + out = """ + ls: #{@erts_glob}: No such file or directory + ls: #{@boot_glob}: No such file or directory + """ + + # Both missing prefers the ERTS tag — provisioning ERTS is the + # first thing --native does; there is no way to have the bootfile + # without ERTS in a real deploy, so reporting bootfile first would + # send the reader chasing the wrong file. + assert Deployer.classify_android_runtime_ls(out, @erts_glob, @boot_glob) == + {:error, :erts_missing} + end + + test "bootfile missing when only the boot glob has no match — MOB-183" do + out = """ + /data/data/com.example.demo/files/otp/erts-14.2.5/bin/erl_child_setup + ls: #{@boot_glob}: No such file or directory + """ + + assert Deployer.classify_android_runtime_ls(out, @erts_glob, @boot_glob) == + {:error, :bootfile_missing} + end + + test "recognises the Toybox 'not found' variant of the missing-file message" do + # Some Android userland (Toybox `ls`) says ": not found" rather + # than "ls: : No such file or directory". The classifier must + # match either shape or the check silently passes on that userland. + out = "#{@erts_glob}: not found" + + assert Deployer.classify_android_runtime_ls(out, @erts_glob, @boot_glob) == + {:error, :erts_missing} + end + + test "recognises the GNU coreutils 'cannot access' variant" do + out = "ls: cannot access '#{@boot_glob}': No such file or directory" + + # This message doesn't contain the "ls: : No such file" prefix + # the primary branch looks for, so a naive substring on "No such file" + # would misfire between erts and bootfile in the both-missing case. + # Anchoring on the specific glob is the point of the classifier. + assert Deployer.classify_android_runtime_ls(out, @erts_glob, @boot_glob) == + {:error, :bootfile_missing} + end + + # The pre-review revision of the fix fell through to `:ok` whenever + # `run_adb` returned `{:error, out}` — which is exactly the arm that + # fires on the MOB-183 failure mode, because modern adb-shell (v2, + # default since Android 7) forwards the inner shell's non-zero exit as + # the session's own exit. `ensure_erts_on_device` now feeds `out` into + # the classifier regardless of the adb-shell status, so a real + # partial-runtime device reaches this classifier — the pathway must + # produce a verdict, not a silent pass. These fixtures are the exact + # `ls: ... No such file` strings adb-shell hands back on that arm. + test "MOB-183 regression: classifier fires on adb-shell error output" do + out = """ + /data/data/com.example.demo/files/otp/erts-14.2.5/bin/erl_child_setup + ls: #{@boot_glob}: No such file or directory + """ + + assert Deployer.classify_android_runtime_ls(out, @erts_glob, @boot_glob) == + {:error, :bootfile_missing} + end + + test "empty output passes through as ok (adb transport failure fallback)" do + # A true adb transport failure (device disappeared, offline race) has + # no output to classify. Best-effort: let the deploy proceed and let + # the actual push failure surface the real problem. + assert Deployer.classify_android_runtime_ls("", @erts_glob, @boot_glob) == :ok + assert Deployer.classify_android_runtime_ls(nil, @erts_glob, @boot_glob) == :ok + end + end end