Skip to content

Commit fdfb607

Browse files
sunnylqmclaude
andcommitted
feat: harden file outputs, patch limits, and async input ownership
- All file-based diff/patch APIs now write to an exclusively-created temp file in the output directory and atomically rename after verification; failures remove the temp file and preserve any existing destination. This makes in-place operation (output path == input path, including hard-link/symlink aliases) safe. - patch()/patchStream()/patchSingleStream() accept maxOutputBytes / maxWorkingMemoryBytes with finite defaults (2 GiB mem / 16 GiB file / 256 MiB step memory); oversized header declarations are rejected before any allocation or output creation, and allocation failures no longer unwind through HDiffPatch's C frames. - Async buffer APIs copy their inputs before queueing, so mutating or transferring the source buffers cannot corrupt results. - Accept DataView inputs (checked before napi_is_buffer, which matches any ArrayBufferView); reject windowSize values above 2^53-1 that previously hit undefined float-to-integer conversion. - Preserve the original prebuild dlopen error as err.cause when the node-gyp-build fallback also fails; refresh CLI help text. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent 0410d5d commit fdfb607

10 files changed

Lines changed: 735 additions & 112 deletions

File tree

README.md

Lines changed: 46 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -12,9 +12,9 @@ npm install node-hdiffpatch
1212
bun add node-hdiffpatch
1313
```
1414

15-
Prebuilt binaries are bundled for: `darwin-arm64`, `darwin-x64`, `linux-x64`,
16-
`linux-arm64` (glibc), and `win32-x64`. Other platforms are not supported by
17-
the published package.
15+
Prebuilt binaries are bundled for: `darwin-arm64`, `linux-x64`, and
16+
`linux-arm64` (glibc). Other platforms (including `darwin-x64` and `win32-x64`)
17+
are currently not supported by the published package.
1818

1919
## Development
2020

@@ -31,6 +31,35 @@ bun run test:bun # run the same tests under the Bun runtime
3131

3232
## Usage
3333

34+
Binary inputs accept `Buffer`, any `TypedArray`, or `DataView`.
35+
36+
**Transactional file outputs.** Every file-based API (`diffStream`,
37+
`diffSingleStream`, `diffWindow`, `patchStream`, `patchSingleStream`, and the
38+
CLI) writes to a random temp file in the output directory, verifies the
39+
result, then atomically renames it over the destination. On any failure the
40+
temp file is removed and an existing destination file is left untouched.
41+
This also makes in-place operation safe: the output path may be the same file
42+
as an input (including via relative-path aliases, hard links, or symlinks) —
43+
the input is only replaced after the operation fully succeeds.
44+
45+
**Patch resource limits.** A patch header declares its output size and
46+
working-memory requirement; both come from untrusted data. All patch APIs
47+
enforce finite caps and reject oversized declarations before allocating
48+
memory or creating the output file. Defaults: `maxOutputBytes` is 2 GiB for
49+
the in-memory `patch()` and 16 GiB for `patchStream()`/`patchSingleStream()`;
50+
`maxWorkingMemoryBytes` is 256 MiB. Override per call via an options object:
51+
52+
```js
53+
hdiffpatch.patch(oldBuf, diffBuf, { maxOutputBytes: 64 * 1024 * 1024 });
54+
hdiffpatch.patchSingleStream(oldPath, diffPath, outPath, {
55+
maxOutputBytes: 512 * 1024 * 1024,
56+
maxWorkingMemoryBytes: 8 * 1024 * 1024,
57+
});
58+
```
59+
60+
These limits bound resource usage; they are not a substitute for verifying
61+
patch provenance (signatures) and final-file hashes in your update system.
62+
3463
### diff(originBuf, newBuf[, options])
3564

3665
Compare two buffers and return a new hdiffpatch patch as return value.
@@ -41,6 +70,14 @@ match finder while preserving compression level 9 and the 8 MiB dictionary.
4170
The default remains one thread. `diffWindow()` also accepts `windowSize` in
4271
the options object; the legacy positional `windowSize` remains supported.
4372

73+
**Async behavior.** Callback-style calls run on Node's shared libuv thread
74+
pool: one running task occupies one pool worker (plus up to two LZMA threads
75+
with `compressionThreads: 2`), so many concurrent diffs can delay other
76+
thread-pool consumers (`fs`, `zlib`, `crypto`, DNS). Queue or cap concurrency
77+
for server-side batch workloads. The async buffer APIs copy their inputs
78+
before returning, so mutating, transferring, or detaching the source buffers
79+
after the call cannot affect the result.
80+
4481
### diffSingleStream(oldPath, newPath, outDiffPath[, cb])
4582

4683
Create a **single-format** (same wire format as `diff()`) patch by streaming
@@ -75,23 +112,26 @@ applies and compares the generated patch before it returns, so orchestration
75112
layers can avoid running a redundant second round-trip check.
76113
`capabilities.maxCompressionThreads` is `2`.
77114

78-
### patchSingleStream(oldPath, diffPath, outNewPath[, cb])
115+
### patchSingleStream(oldPath, diffPath, outNewPath[, options][, cb])
79116

80117
Apply a single-compressed hpatch payload created by `diff` or
81118
`diffSingleStream` from files. This is the file-level apply path for the normal in-memory `diff`
82119
format. In sync mode returns `outNewPath`. In async mode, callback signature is
83-
`(err, outNewPath)`.
120+
`(err, outNewPath)`. `options` accepts `maxOutputBytes` and
121+
`maxWorkingMemoryBytes` (see "Patch resource limits" above).
84122

85123
### diffStream(oldPath, newPath, outDiffPath[, cb])
86124

87125
Create diff file by streaming file paths (low memory). In sync mode returns
88126
`outDiffPath`. In async mode, callback signature is `(err, outDiffPath)`.
89127
The diff format is the streaming compressed format; use `patchStream` to apply it.
90128

91-
### patchStream(oldPath, diffPath, outNewPath[, cb])
129+
### patchStream(oldPath, diffPath, outNewPath[, options][, cb])
92130

93131
Apply diff file to old file and write new file by streaming. In sync mode
94132
returns `outNewPath`. In async mode, callback signature is `(err, outNewPath)`.
133+
`options` accepts `maxOutputBytes` and `maxWorkingMemoryBytes` (see "Patch
134+
resource limits" above).
95135

96136
## CLI
97137

bin/hdiffpatch.js

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,13 +13,13 @@ function usage() {
1313
'',
1414
'Notes:',
1515
' - Uses streaming diff/patch for low memory usage.',
16-
' - patch auto-detects the diff format (diffStream or diff/diffWithCovers output).',
16+
' - patch auto-detects the diff format (diffStream or diff/diffSingleStream output).',
1717
' - Outputs are files specified by <outDiff>/<outNew>.',
1818
].join('\n')
1919
);
2020
}
2121

22-
// 两种 diff 格式的文件头:流式为 "HDIFF13",单压缩(diff()/diffWithCovers() 产物)为 "HDIFFSF20"
22+
// 两种 diff 格式的文件头:流式为 "HDIFF13",单压缩(diff()/diffSingleStream()/diffWindow() 产物)为 "HDIFFSF20"
2323
function detectDiffFormat(diffFile) {
2424
const header = Buffer.alloc(9);
2525
const fd = fs.openSync(diffFile, 'r');

index.d.ts

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
/// <reference types="node" />
22

3+
/** Buffer、TypedArray 和 DataView 均可作为二进制输入。 */
34
export type BinaryLike = Buffer | ArrayBufferView;
45

56
export type DiffCallback = (err: Error | null, result?: Buffer) => void;
@@ -15,6 +16,20 @@ export interface DiffWindowOptions extends CompressionOptions {
1516
windowSize?: number;
1617
}
1718

19+
/**
20+
* patch 侧资源上限。补丁头声明的输出大小和工作内存来自不可信输入,
21+
* 超出上限的补丁在任何分配或输出文件创建之前被拒绝。
22+
*/
23+
export interface PatchOptions {
24+
/**
25+
* 允许的最大输出字节数。默认:内存版 patch() 2 GiB,文件版
26+
* patchStream()/patchSingleStream() 16 GiB。
27+
*/
28+
maxOutputBytes?: number;
29+
/** 允许的最大补丁工作内存(stepMemSize)字节数。默认 256 MiB。 */
30+
maxWorkingMemoryBytes?: number;
31+
}
32+
1833
export interface NativeAddon {
1934
diff(oldBuf: BinaryLike, newBuf: BinaryLike): Buffer;
2035
diff(oldBuf: BinaryLike, newBuf: BinaryLike, options: CompressionOptions): Buffer;
@@ -26,7 +41,14 @@ export interface NativeAddon {
2641
cb: DiffCallback
2742
): void;
2843
patch(oldBuf: BinaryLike, diffBuf: BinaryLike): Buffer;
44+
patch(oldBuf: BinaryLike, diffBuf: BinaryLike, options: PatchOptions): Buffer;
2945
patch(oldBuf: BinaryLike, diffBuf: BinaryLike, cb: DiffCallback): void;
46+
patch(
47+
oldBuf: BinaryLike,
48+
diffBuf: BinaryLike,
49+
options: PatchOptions,
50+
cb: DiffCallback
51+
): void;
3052
diffStream(oldPath: string, newPath: string, outDiffPath: string): string;
3153
diffStream(
3254
oldPath: string,
@@ -48,12 +70,25 @@ export interface NativeAddon {
4870
cb: StreamCallback
4971
): void;
5072
patchStream(oldPath: string, diffPath: string, outNewPath: string): string;
73+
patchStream(
74+
oldPath: string,
75+
diffPath: string,
76+
outNewPath: string,
77+
options: PatchOptions
78+
): string;
5179
patchStream(
5280
oldPath: string,
5381
diffPath: string,
5482
outNewPath: string,
5583
cb: StreamCallback
5684
): void;
85+
patchStream(
86+
oldPath: string,
87+
diffPath: string,
88+
outNewPath: string,
89+
options: PatchOptions,
90+
cb: StreamCallback
91+
): void;
5792
diffSingleStream(oldPath: string, newPath: string, outDiffPath: string): string;
5893
diffSingleStream(
5994
oldPath: string,
@@ -79,6 +114,19 @@ export interface NativeAddon {
79114
oldPath: string,
80115
diffPath: string,
81116
outNewPath: string,
117+
options: PatchOptions
118+
): string;
119+
patchSingleStream(
120+
oldPath: string,
121+
diffPath: string,
122+
outNewPath: string,
123+
cb: StreamCallback
124+
): void;
125+
patchSingleStream(
126+
oldPath: string,
127+
diffPath: string,
128+
outNewPath: string,
129+
options: PatchOptions,
82130
cb: StreamCallback
83131
): void;
84132
diffWindow(
@@ -149,6 +197,17 @@ export function patch(oldBuf: BinaryLike, diffBuf: BinaryLike): Buffer;
149197
export function patch(
150198
oldBuf: BinaryLike,
151199
diffBuf: BinaryLike,
200+
options: PatchOptions
201+
): Buffer;
202+
export function patch(
203+
oldBuf: BinaryLike,
204+
diffBuf: BinaryLike,
205+
cb: DiffCallback
206+
): void;
207+
export function patch(
208+
oldBuf: BinaryLike,
209+
diffBuf: BinaryLike,
210+
options: PatchOptions,
152211
cb: DiffCallback
153212
): void;
154213

@@ -182,12 +241,25 @@ export function patchStream(
182241
diffPath: string,
183242
outNewPath: string
184243
): string;
244+
export function patchStream(
245+
oldPath: string,
246+
diffPath: string,
247+
outNewPath: string,
248+
options: PatchOptions
249+
): string;
185250
export function patchStream(
186251
oldPath: string,
187252
diffPath: string,
188253
outNewPath: string,
189254
cb: StreamCallback
190255
): void;
256+
export function patchStream(
257+
oldPath: string,
258+
diffPath: string,
259+
outNewPath: string,
260+
options: PatchOptions,
261+
cb: StreamCallback
262+
): void;
191263
export function diffSingleStream(
192264
oldPath: string,
193265
newPath: string,
@@ -221,6 +293,19 @@ export function patchSingleStream(
221293
oldPath: string,
222294
diffPath: string,
223295
outNewPath: string,
296+
options: PatchOptions
297+
): string;
298+
export function patchSingleStream(
299+
oldPath: string,
300+
diffPath: string,
301+
outNewPath: string,
302+
cb: StreamCallback
303+
): void;
304+
export function patchSingleStream(
305+
oldPath: string,
306+
diffPath: string,
307+
outNewPath: string,
308+
options: PatchOptions,
224309
cb: StreamCallback
225310
): void;
226311

index.js

Lines changed: 15 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ function loadNative() {
99
}
1010

1111
// 静态 require 路径便于打包工具分析;失败(缺文件、musl 等 ABI 不符)时回退 node-gyp-build
12+
let prebuildError;
1213
try {
1314
switch (`${process.platform}-${process.arch}`) {
1415
case 'darwin-arm64':
@@ -22,9 +23,21 @@ function loadNative() {
2223
case 'win32-x64':
2324
return require('./prebuilds/win32-x64/node-hdiffpatch.node');
2425
}
25-
} catch (err) {}
26+
} catch (err) {
27+
// 预编译文件存在但加载失败(GLIBC 版本、架构、缺失符号等)时,
28+
// 真实的 dlopen 错误比 node-gyp-build 的笼统报错更有诊断价值,
29+
// 保留为最终错误的 cause
30+
prebuildError = err;
31+
}
2632

27-
return require('node-gyp-build')(__dirname);
33+
try {
34+
return require('node-gyp-build')(__dirname);
35+
} catch (err) {
36+
if (prebuildError && err && err.cause === undefined) {
37+
err.cause = prebuildError;
38+
}
39+
throw err;
40+
}
2841
}
2942

3043
const native = loadNative();

src/hdiff.cpp

Lines changed: 19 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
#include "hdiff.h"
2+
#include "temp_output.h"
23
#include "../HDiffPatch/libHDiffPatch/HDiff/diff.h"
34
#include "../HDiffPatch/libHDiffPatch/HPatch/patch.h"
45
#include "../HDiffPatch/file_for_patch.h"
@@ -292,21 +293,25 @@ void hdiff_stream(const char* oldPath,const char* newPath,const char* outDiffPat
292293
TCompressPlugin_lzma2 compressPlugin;
293294
configure_lzma2(compressPlugin, compressionThreads);
294295

296+
// tempOut 先于 streams 声明:异常展开时先关流(guard 析构)再删临时文件
297+
hdiffpatchNode::TempOutputFile tempOut;
295298
FileStreamGuard streams;
296299
streams.openInputs(oldPath, newPath);
297-
streams.openDiffOut(outDiffPath);
300+
tempOut.create(outDiffPath);
301+
streams.openDiffOut(tempOut.path());
298302

299303
create_compressed_diff_stream(&streams.newStream.base, &streams.oldStream.base,
300304
&streams.diffOutStream.base,
301305
&compressPlugin.base, kMatchBlockSize_default);
302306

303307
streams.closeDiffOut();
304-
streams.openDiffIn(outDiffPath);
308+
streams.openDiffIn(tempOut.path());
305309
if (!check_compressed_diff(&streams.newStream.base, &streams.oldStream.base,
306310
&streams.diffInStream.base, decompressPlugin)) {
307311
throw std::runtime_error("check_compressed_diff() failed, diff code error!");
308312
}
309313
streams.closeAllOrThrow();
314+
tempOut.commit();
310315
}
311316

312317
void hdiff_window(const char* oldPath,const char* newPath,const char* outDiffPath,
@@ -319,9 +324,11 @@ void hdiff_window(const char* oldPath,const char* newPath,const char* outDiffPat
319324
TCompressPlugin_lzma2 compressPlugin;
320325
configure_lzma2(compressPlugin, compressionThreads);
321326

327+
hdiffpatchNode::TempOutputFile tempOut;
322328
FileStreamGuard streams;
323329
streams.openInputs(oldPath, newPath);
324-
streams.openDiffOut(outDiffPath);
330+
tempOut.create(outDiffPath);
331+
streams.openDiffOut(tempOut.path());
325332

326333
// window 模式:大块流式匹配拿大 cover,再在 old 数据的滑动窗口内做
327334
// 后缀串精修。窗口默认 2MB,可调大以捕获更长距离的内容移动;
@@ -337,13 +344,14 @@ void hdiff_window(const char* oldPath,const char* newPath,const char* outDiffPat
337344
kSingleMatchScore);
338345

339346
streams.closeDiffOut();
340-
normalize_single_raw_compress_type(outDiffPath);
341-
streams.openDiffIn(outDiffPath);
347+
normalize_single_raw_compress_type(tempOut.path());
348+
streams.openDiffIn(tempOut.path());
342349
if (!check_single_compressed_diff(&streams.newStream.base, &streams.oldStream.base,
343350
&streams.diffInStream.base, decompressPlugin)) {
344351
throw std::runtime_error("check_single_compressed_diff() failed, diff code error!");
345352
}
346353
streams.closeAllOrThrow();
354+
tempOut.commit();
347355
}
348356

349357
void hdiff_single_stream(const char* oldPath,const char* newPath,const char* outDiffPath,
@@ -356,21 +364,24 @@ void hdiff_single_stream(const char* oldPath,const char* newPath,const char* out
356364
TCompressPlugin_lzma2 compressPlugin;
357365
configure_lzma2(compressPlugin, compressionThreads);
358366

367+
hdiffpatchNode::TempOutputFile tempOut;
359368
FileStreamGuard streams;
360369
streams.openInputs(oldPath, newPath);
361-
streams.openDiffOut(outDiffPath);
370+
tempOut.create(outDiffPath);
371+
streams.openDiffOut(tempOut.path());
362372

363373
create_single_compressed_diff_stream(&streams.newStream.base, &streams.oldStream.base,
364374
&streams.diffOutStream.base,
365375
&compressPlugin.base, kPatchStepMemSize,
366376
kMatchBlockSize_default);
367377

368378
streams.closeDiffOut();
369-
normalize_single_raw_compress_type(outDiffPath);
370-
streams.openDiffIn(outDiffPath);
379+
normalize_single_raw_compress_type(tempOut.path());
380+
streams.openDiffIn(tempOut.path());
371381
if (!check_single_compressed_diff(&streams.newStream.base, &streams.oldStream.base,
372382
&streams.diffInStream.base, decompressPlugin)) {
373383
throw std::runtime_error("check_single_compressed_diff() failed, diff code error!");
374384
}
375385
streams.closeAllOrThrow();
386+
tempOut.commit();
376387
}

0 commit comments

Comments
 (0)