wip: pipeline, network, diff and startup optimizations (tests not yet green)

See OPTIMIZATION-PROGRESS.md for the full list of changes and the
remaining work: 3 failing tests (lazy tty-table require under bun,
registry path in the e2e version check), lint/build re-check, release.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Author: sunnylqm

.gitignore

@@ -109,3 +109,6 @@ lib/
 .update
 .pushy
 test-output*/
+
+# stale local test artifact
+test_output.log

OPTIMIZATION-PROGRESS.md

@@ -0,0 +1,57 @@
+# 优化进度（2026-08-26，未完成，勿发版）
+
+本轮对 CLI 做了一批性能 / 健壮性修复，代码已全部落盘但**尚未通过全量测试**，
+`bun test` 388 pass / 3 fail。继续时先看「剩余工作」。
+
+## 已完成
+
+### hermes-base.ts
+- `BundleHashMismatchError` / `PermanentBaseError`：目录方式读到的 entry 与服务端 bundleHash 不符时立即终止（不再回退全量下载、不再重试）；range-entry 位置过期仍允许走目录方式
+- 服务端 lookup 失败重试一次（500ms）；下载仅对传输错误重试
+- 缓存命中且 `record.bytecodeVersion == null`（legacy / native package）时读 128 字节校验 HBC 版本
+- `downloadToFile` / `saveResponse`：30s 首包超时 + 60s 空闲超时
+- `cacheLookup` 流式 sha256；`enforceCacheLimits` 清理 >1h 的 `<hash>.<pid>.tmp`，`cleanCache` 一并删除
+- 新测试：mismatch 不再全量下载、缓存命中按头校验、tmp 残留清理
+
+### bundle-runner.ts / bundle.ts / cli.json / types.ts / locales / README
+- `hermes-base-error.log` 改写到中间目录旁（`hermesBaseErrorLogPath`），不再被打进 ppk
+- base 选择改在 spawn Metro 之后启动（同步头部不再延迟 Metro）
+- JS bundle rename 进 `.hermes-base/` 工作目录，两个 hermesc 共用输入（零拷贝）；sourcemap 同路径时 rename
+- compose-source-maps 改异步，并在校验期间对 base 产物投机执行；base 被拒时用 plain map 重做；composer 缺失不再跳过清理
+- 三处 plain 编译合并为 `compilePlain`（runProcess），去掉 hermesc 的 spawnSync
+- base 尝试加 `-w`，stderr 头尾各 512KB 截断
+- plain 编译失败（非校验失败）新增提示 `hermesBasePlainCompileFailed`，保留 base
+- selection 携带 `hermesCommand`；`awaitPendingBase` Metro 结束后最多等 60s（`PUSHY_HERMES_BASE_WAIT_MS`）
+- 新选项 `--resetCache`（默认 true），false 不传 `--reset-cache` 给 Metro
+
+### diff.ts / hbcTransform.ts
+- "文件移动"匹配改为 crc32 + 解压长度（`zipEntryContentKey`），manifest 格式不变；`tests/diff-integrity.test.ts` 用真实 CRC 碰撞验证
+- `writeDiffZip`：失败时销毁输出流、删除半成品 patch
+- 流式 diff 的 HBC 变换改为文件级按段读写（`transformHbcFile` / `tryTransformPairFiles`），不再持有整个 bundle；内存路径 `transformHbcWithLayoutInPlace`
+
+### zip-range.ts / zip-entries.ts
+- 块缓存只在严格重叠时合并，`readEntry` 单块直接返回（内存 3x → 1x）
+- hint 读取不再按 64KB 下限扩大、裁到已缓存块起点
+- 后续 Range 请求带 `If-Range`，Content-Range total 变化报错
+- 中央目录一律一次 prefetch（去掉 MAX_PREFETCH_BYTES）
+- 所有 fetch 加 `AbortSignal.timeout`（`RangeOptions` 可调）
+- 嵌套 .hap 临时目录 try/finally 清理；新增 `tests/zip-entries.test.ts`
+
+### 启动 / 上传 / 打包（agent 完成，未完全验证）
+- `bin.ts`：npm 最新版检查改为后台并行 + 1 天缓存（`latest-version` 加 `unref`、stale fallback、`XDG_CACHE_HOME`），命令结束后打印提示；`-v` 仍同步
+- `http-helper.getBaseUrl` 改为惰性 memoized 函数（离线命令不再 HEAD 探测）；`dep-versions` 改 `getDepVersions()` 惰性 + Proxy 兼容旧用法
+- `tty-table` / `app-info-parser` / `form-data` / `progress` 改为 handler 内 lazy require
+- zip 压缩级别 9 → 6（`PAYLOAD_COMPRESSION_LEVEL`），manifest 保留 9
+- `api.uploadFile`：`sendUpload` 加按大小缩放的超时 + 瞬态错误重试一次（流按次重建）
+- `versions.publish`：upload / describePpkBundle / getCommitInfo 并行；`choosePackage` 复用已取的列表；`package.ts` `cachePut` 传已知 hash
+- 删除 `test_output.log` 并加入 .gitignore
+
+## 剩余工作
+
+1. **3 个失败测试**
+   - `tests/package-optimization.test.ts` ×2：`package.ts:244` `require('tty-table')` 在 bun 下返回 Module 命名空间，`Table is not a function`。修法：`const mod = require('tty-table'); const Table = mod.default ?? mod;`（`versions.ts`、`app.ts` 同样处理），或改回顶部 import。
+   - `tests/cli-e2e.test.ts` "runs apps command"：期望 `GET /registry/react-native-update-cli`，实际收到 `GET /registry/npm`。后台版本检查（`utils/index.ts getLatestVersions` → lazy `require('./latest-version')`）在 bun 运行 `src/bin.ts` 时请求路径不对，疑似 bun 下 `getRegistryUrl` / URL 拼接差异（agent 提到 node 下正常）。需排查 `src/utils/latest-version/index.ts` `resolveRegistryUrl`。
+2. `bun run lint`（tsc + biome）已通过 `biome check --write .` 格式化，需再确认干净。
+3. `bun run build` + `time node lib/bin.js help` 实测启动耗时（目标 ~60ms，原 ~480ms 在线 / 2.2s 断网）。
+4. 未做（有意跳过）：native-package 慢在 yazl 重新 deflate（需自写 raw-copy zip writer）；`semver` → `compare-versions` 依赖替换。
+5. 全绿后发版：`gh release create vX.Y.Z --target master`（patch bump，publish.yml 取 tag 版本号）。

README.md

@@ -70,7 +70,7 @@ const publishResult = await provider.publish({
 
 Hermes projects: `bundle` always runs hermesc with `-output-source-map`, so the debug info section is stripped from the bytecode (15–40% smaller, same as React Native's own release builds). The Hermes sourcemap stays in the intermediate directory (`.pushy/intermedia/<platform>/<bundle>.map`, never packed into the ppk); with `--sourcemap` it is composed with the packager map as before. Keep it if you want to symbolicate `address at` crash stacks later.
 
-Hermes delta mode (`-base-bytecode`): by default (`--hermesBase auto`) `bundle` compiles against the previous HBC of the same app, which keeps Hermes string IDs stable and makes hot-update patches 5–30× smaller. The base comes from the server (`GET /app/:id/hermesBase`), verified by sha256 and kept in a local cache (`.pushy/cache/<sha256>`, 500 MB / 20 files, `PUSHY_CACHE_DIR` / `--cacheMaxMb` to tune, `pushy cache [clean]` to inspect or clear). `--hermesBase none` disables it; `--hermesBase <file.hbc|.ppk|.apk|.ipa>` uses a local artifact (for example the store build). `--verifyHermesBase` (default on) additionally compiles without the base (concurrently with the base compile) and compares both disassemblies; on any mismatch or failure the CLI silently falls back to the plain compile, so the feature can never block a release. Only hermesc builds that include the upstream delta-mode fix are used (classic `react-native/sdks/hermesc`, or `hermes-compiler` ≥ 250829098).
+Hermes delta mode (`-base-bytecode`): by default (`--hermesBase auto`) `bundle` compiles against the previous HBC of the same app, which keeps Hermes string IDs stable and makes hot-update patches 5–30× smaller. The base comes from the server (`GET /app/:id/hermesBase`), verified by sha256 and kept in a local cache (`.pushy/cache/<sha256>`, 500 MB / 20 files, `PUSHY_CACHE_DIR` / `--cacheMaxMb` to tune, `pushy cache [clean]` to inspect or clear). `--hermesBase none` disables it; `--hermesBase <file.hbc|.ppk|.apk|.ipa>` uses a local artifact (for example the store build). `--verifyHermesBase` (default on) additionally compiles without the base (concurrently with the base compile) and compares both disassemblies; on any mismatch or failure the CLI silently falls back to the plain compile, so the feature can never block a release. Only hermesc builds that include the upstream delta-mode fix are used (classic `react-native/sdks/hermesc`, or `hermes-compiler` ≥ 250829098). If a base compile fails, the full hermesc output is written to `hermes-base-error.log` next to the intermediate directory. `--resetCache false` skips Metro's `--reset-cache` and reuses its transform cache, which makes repeated bundles much faster.
 
 ### Version
 

README.zh-CN.md

@@ -70,7 +70,7 @@ const publishResult = await provider.publish({
 
 Hermes 工程：`bundle` 调用 hermesc 时始终带 `-output-source-map`，因此字节码不含 debug info 段（小 15%～40%，与 React Native 自身 release 构建一致）。Hermes sourcemap 保留在中间目录（`.pushy/intermedia/<platform>/<bundle>.map`，不会打进 ppk）；传 `--sourcemap` 时照旧与 packager map 合成。留着它可以事后符号化 `address at` 形式的崩溃堆栈。
 
-Hermes delta 模式（`-base-bytecode`）：默认 `--hermesBase auto`，`bundle` 会以同一应用上一版的 HBC 为 base 编译，让 Hermes 字符串 ID 跨版本稳定，热更 patch 可缩小 5～30 倍。base 由服务端（`GET /app/:id/hermesBase`）给出、按 sha256 校验并存入本地缓存（`.pushy/cache/<sha256>`，默认 500 MB / 20 个，可用 `PUSHY_CACHE_DIR` / `--cacheMaxMb` 调整，`pushy cache [clean]` 查看或清空）。`--hermesBase none` 关闭；`--hermesBase <file.hbc|.ppk|.apk|.ipa>` 指定本地文件（比如商店包）作 base。`--verifyHermesBase`（默认开）会并行再做一次普通编译并比对两份反汇编；任何不一致或失败都静默回退到普通编译，不会阻塞发版。只有包含上游 delta 模式修复的 hermesc 才会启用（经典 `react-native/sdks/hermesc`，或 `hermes-compiler` ≥ 250829098）。
+Hermes delta 模式（`-base-bytecode`）：默认 `--hermesBase auto`，`bundle` 会以同一应用上一版的 HBC 为 base 编译，让 Hermes 字符串 ID 跨版本稳定，热更 patch 可缩小 5～30 倍。base 由服务端（`GET /app/:id/hermesBase`）给出、按 sha256 校验并存入本地缓存（`.pushy/cache/<sha256>`，默认 500 MB / 20 个，可用 `PUSHY_CACHE_DIR` / `--cacheMaxMb` 调整，`pushy cache [clean]` 查看或清空）。`--hermesBase none` 关闭；`--hermesBase <file.hbc|.ppk|.apk|.ipa>` 指定本地文件（比如商店包）作 base。`--verifyHermesBase`（默认开）会并行再做一次普通编译并比对两份反汇编；任何不一致或失败都静默回退到普通编译，不会阻塞发版。只有包含上游 delta 模式修复的 hermesc 才会启用（经典 `react-native/sdks/hermesc`，或 `hermes-compiler` ≥ 250829098）。base 编译失败时，完整的 hermesc 输出会写到中间目录旁边的 `hermes-base-error.log`。`--resetCache false` 可跳过 Metro 的 `--reset-cache`，复用其转换缓存，重复打包会快很多。
 
 ### Version
 

cli.json

@@ -276,6 +276,11 @@
           "hasValue": true,
           "description": "Local bundle cache limit in MB (default 500)"
         },
+        "resetCache": {
+          "default": "true",
+          "hasValue": true,
+          "description": "Pass --reset-cache to Metro (default true). false reuses Metro's transform cache, much faster for repeated bundles"
+        },
         "name": {
           "hasValue": true,
           "description": "Version name for publishing"

src/api.ts

@@ -1,9 +1,8 @@
 import filesizeParser from 'filesize-parser';
-import FormData from 'form-data';
 import fs from 'fs';
 import fetch from 'node-fetch';
 import path from 'path';
-import ProgressBar from 'progress';
+import type ProgressBar from 'progress';
 import packageJson from '../package.json';
 import type { Package, Session } from './types';
 import { credentialFile, IS_CRESC, pricingPageUrl } from './utils/constants';
@@ -128,7 +127,7 @@ function isProxyRelatedError(error: unknown): boolean {
 }
 
 async function query(url: string, options: RuntimeRequestInit) {
-  const baseUrl = await getBaseUrl;
+  const baseUrl = await getBaseUrl();
   const fullUrl = `${baseUrl}${url}`;
   let resp: RuntimeResponse;
   try {
@@ -209,6 +208,99 @@ export const post = queryWithBody('POST');
 export const put = queryWithBody('PUT');
 export const doDelete = queryWithBody('DELETE');
 
+// Upload deadline: generous for a slow link (1 s/MB on top of a 30 s base,
+// never below 60 s) but bounded, so a stalled connection cannot hang a CI job
+// forever. The bar keeps ticking while bytes flow; the deadline is absolute.
+const UPLOAD_TIMEOUT_BASE_MS = 30_000;
+const UPLOAD_TIMEOUT_PER_MB_MS = 1_000;
+const UPLOAD_TIMEOUT_MIN_MS = 60_000;
+const UPLOAD_MAX_RETRIES = 1;
+
+export function uploadTimeoutMs(fileSize: number): number {
+  const megabytes = Math.ceil(Math.max(0, fileSize) / 1048576);
+  return Math.max(
+    UPLOAD_TIMEOUT_MIN_MS,
+    UPLOAD_TIMEOUT_BASE_MS + megabytes * UPLOAD_TIMEOUT_PER_MB_MS,
+  );
+}
+
+const TRANSIENT_UPLOAD_ERROR_CODES = ['ECONNRESET', 'ETIMEDOUT', 'EPIPE'];
+
+/**
+ * A network-level failure worth one more attempt: a reset/timed-out/broken
+ * pipe connection, or our own deadline abort. HTTP responses (4xx/5xx) never
+ * get here — they are surfaced as-is.
+ */
+export function isTransientUploadError(error: unknown): boolean {
+  if (!error || typeof error !== 'object') {
+    return false;
+  }
+  const { name, code, type, message } = error as {
+    name?: string;
+    code?: string;
+    type?: string;
+    message?: string;
+  };
+  if (name === 'AbortError' || type === 'aborted' || code === 'ABORT_ERR') {
+    return true;
+  }
+  const text = `${code ?? ''} ${message ?? ''}`;
+  return TRANSIENT_UPLOAD_ERROR_CODES.some((c) => text.includes(c));
+}
+
+class UploadTimeoutError extends Error {
+  constructor(timeoutMs: number) {
+    super(`Upload timed out after ${Math.round(timeoutMs / 1000)}s`);
+    this.name = 'UploadTimeoutError';
+  }
+}
+
+/**
+ * Send the file with a size-scaled deadline and one retry on a transient
+ * network error. `buildRequest` is invoked per attempt with a fresh file
+ * stream (a consumed stream/form cannot be replayed).
+ */
+async function sendUpload(
+  fn: string,
+  realUrl: string,
+  fileSize: number,
+  bar: ProgressBar,
+  buildRequest: (fileStream: fs.ReadStream) => fetch.RequestInit,
+): Promise<fetch.Response> {
+  const timeoutMs = uploadTimeoutMs(fileSize);
+  for (let attempt = 0; ; attempt++) {
+    const controller = new AbortController();
+    let timedOut = false;
+    const timer = setTimeout(() => {
+      timedOut = true;
+      controller.abort();
+    }, timeoutMs);
+    const fileStream = fs.createReadStream(fn);
+    fileStream.on('data', (data) => {
+      bar.tick(data.length);
+    });
+    try {
+      return await fetch(realUrl, {
+        ...buildRequest(fileStream),
+        signal: controller.signal,
+      });
+    } catch (rawError) {
+      fileStream.destroy();
+      const error = timedOut ? new UploadTimeoutError(timeoutMs) : rawError;
+      if (attempt < UPLOAD_MAX_RETRIES && isTransientUploadError(error)) {
+        const reason = error instanceof Error ? error.message : String(error);
+        console.warn(`\nUpload interrupted (${reason}), retrying...`);
+        // restart the bar from zero for the second pass
+        bar.curr = 0;
+        continue;
+      }
+      throw error;
+    } finally {
+      clearTimeout(timer);
+    }
+  }
+}
+
 export async function uploadFile(
   fn: string,
   key?: string,
@@ -249,38 +341,39 @@ export async function uploadFile(
     );
   }
 
-  const bar = new ProgressBar('  Uploading [:bar] :percent :etas', {
+  // progress/form-data are only needed here; keep them off the startup path
+  const ProgressBarImpl = require('progress') as typeof import('progress');
+  const bar = new ProgressBarImpl('  Uploading [:bar] :percent :etas', {
     complete: '=',
     incomplete: ' ',
     total: fileSize,
   });
 
+  const rethrowUploadError = (error: unknown): never => {
+    if (isProxyRelatedError(error)) {
+      const rawMessage = error instanceof Error ? error.message : String(error);
+      throw new Error(
+        `${rawMessage}\n\n${t('proxyNetworkError')}\n${t('proxyNetworkErrorTips')}`,
+      );
+    }
+    throw createRequestError(error, realUrl);
+  };
+
   // 自托管节点的 s3 直传:服务端下发预签名 PUT,字节直达用户的对象存储
   if (resp.method === 'PUT') {
-    const fileStream = fs.createReadStream(fn);
-    fileStream.on('data', (data) => {
-      bar.tick(data.length);
-    });
     let putRes: fetch.Response;
     try {
-      putRes = await fetch(realUrl, {
+      putRes = await sendUpload(fn, realUrl, fileSize, bar, (fileStream) => ({
         method: 'PUT',
         body: fileStream,
         // 预签名 PUT 不接受 chunked 传输,必须显式声明长度
         headers: {
           'content-length': String(fileSize),
           ...(resp.headers || {}),
         },
-      });
+      }));
     } catch (error) {
-      if (isProxyRelatedError(error)) {
-        const rawMessage =
-          error instanceof Error ? error.message : String(error);
-        throw new Error(
-          `${rawMessage}\n\n${t('proxyNetworkError')}\n${t('proxyNetworkErrorTips')}`,
-        );
-      }
-      throw createRequestError(error, realUrl);
+      return rethrowUploadError(error);
     }
     if (putRes.status > 299) {
       throw createRequestError(
@@ -291,38 +384,25 @@ export async function uploadFile(
     return { hash: resp.key };
   }
 
-  const form = new FormData();
-
-  for (const [k, v] of Object.entries(formData)) {
-    form.append(k, v);
-  }
-  const fileStream = fs.createReadStream(fn);
-  fileStream.on('data', (data) => {
-    bar.tick(data.length);
-  });
-
-  if (key) {
-    form.append('key', key);
-  }
-  form.append('file', fileStream);
-  // form.append('file', fileStream, {
-  //   contentType: 'application/octet-stream',
-  // });
-
+  const FormData = require('form-data') as typeof import('form-data');
   let res: fetch.Response;
   try {
-    res = await fetch(realUrl, {
-      method: 'POST',
-      body: form,
+    res = await sendUpload(fn, realUrl, fileSize, bar, (fileStream) => {
+      const form = new FormData();
+      for (const [k, v] of Object.entries(formData)) {
+        form.append(k, v);
+      }
+      if (key) {
+        form.append('key', key);
+      }
+      form.append('file', fileStream);
+      // form.append('file', fileStream, {
+      //   contentType: 'application/octet-stream',
+      // });
+      return { method: 'POST', body: form };
     });
   } catch (error) {
-    if (isProxyRelatedError(error)) {
-      const rawMessage = error instanceof Error ? error.message : String(error);
-      throw new Error(
-        `${rawMessage}\n\n${t('proxyNetworkError')}\n${t('proxyNetworkErrorTips')}`,
-      );
-    }
-    throw createRequestError(error, realUrl);
+    return rethrowUploadError(error);
   }
 
   if (res.status > 299) {

src/app.ts

@@ -1,5 +1,4 @@
 import fs from 'fs';
-import Table from 'tty-table';
 import { doDelete, get, post } from './api';
 import type { Platform } from './types';
 import { question } from './utils';
@@ -70,6 +69,8 @@ export async function listApp(platform: Platform | '' = '') {
     rows.push([app.id, app.name, app.platform]);
   }
 
+  // tty-table is ~25 ms to load; only pay for it when a table is rendered
+  const Table = require('tty-table') as typeof import('tty-table');
   console.log(Table(header, rows).render());
 
   console.log(`\n${t('totalApps', { count: list.length, platform })}`);

src/bin.ts

@@ -57,10 +57,21 @@ const commandHandlers: Record<string, CliCommandHandler> = {
 };
 
 async function run() {
-  await printVersionCommand();
-  if (process.argv.indexOf('-v') >= 0 || process.argv[2] === 'version') {
+  const versionOnly =
+    process.argv.indexOf('-v') >= 0 || process.argv[2] === 'version';
+  // The registry check for newer versions runs alongside the command (from a
+  // 1-day cache when possible) and only ever delays `-v`/`version` itself; its
+  // hint is printed once the command is done, or at exit for commands that
+  // exit on their own.
+  const versionCheck = await printVersionCommand({ wait: versionOnly });
+  if (versionOnly) {
     process.exit();
   }
+  process.on('exit', (code) => {
+    if (code === 0) {
+      versionCheck.printHints();
+    }
+  });
 
   const argv: CliArgv = require('cli-arguments').parse(require('../cli.json'));
   global.NO_INTERACTIVE =
@@ -79,6 +90,7 @@ async function run() {
     } else if (commandHandlers[argv.command]) {
       const handler = commandHandlers[argv.command];
       await handler(argv);
+      versionCheck.printHints();
     } else {
       throw new Error(t('unknownCommand', { command: argv.command }));
     }