fix(split): visited reporting non existing refs

[ShaMan123]

IfcSplitter.split() reports ids that no line in the file defines

Caught when reviewing code with claude.

My take

I believe this is why the parser should use web-ifc instead of re-inventing the wheel - and address perf using the stream parser #252 and other dedicated fixes

Summary

split() returns Map<groupId, { path, ids }>. The ids set includes express ids that are not written to the output file and do not exist in the input file at all — ids that only ever appeared as a reference, never as an entity definition.

extract() is affected the same way; it shares collectDeps.

Reproduction

Self-contained, public API only:

import { IfcSplitter } from "@thatopen/fragments";
import { readFileSync, writeFileSync } from "node:fs";

const io = {
  async readableStream(p: string) {
    const lines = readFileSync(p, "utf-8").split("\n");
    return new ReadableStream<string>({
      start(c) { for (const l of lines) c.enqueue(l); c.close(); },
    });
  },
  async writableStream(p: string) {
    const out: string[] = [];
    return new WritableStream<string>({
      write: (chunk) => void out.push(chunk),
      close: () => writeFileSync(p, out.join("\n")),
    });
  },
};

writeFileSync("in.ifc", `ISO-10303-21;
HEADER;
FILE_DESCRIPTION((''),'2;1');
FILE_SCHEMA(('IFC4'));
ENDSEC;
DATA;
#1=IFCPROJECT('p',$,'Project',$,$,$,$,$,$);
#2=IFCSITE('s',$,'Site',$,$,$,$,$,$,$,$,$,$,$);
#3=IFCBUILDING('b',$,'Building',$,$,$,$,$,$,$,$,$);
#4=IFCBUILDINGSTOREY('st',$,'Storey',$,$,$,$,$,$,$);
#5=IFCRELAGGREGATES('r1',$,$,$,#1,(#2));
#6=IFCRELAGGREGATES('r2',$,$,$,#2,(#3));
#7=IFCRELAGGREGATES('r3',$,$,$,#3,(#4));
#10=IFCCARTESIANPOINT((0.,0.,0.));
#11=IFCPRODUCTDEFINITIONSHAPE($,$,(#10));
#12=IFCWALL('wa',$,'Wall A',$,$,#999,#11,$,$);
#13=IFCCARTESIANPOINT((1.,0.,0.));
#14=IFCPRODUCTDEFINITIONSHAPE($,$,(#13));
#15=IFCWALL('wb',$,'Wall B (see #99999)',$,$,$,#14,$,$);
#17=IFCRELCONTAINEDINSPATIALSTRUCTURE('rc',$,$,$,(#12,#15),#4);
ENDSEC;
END-ISO-10303-21;
`);

const result = await new IfcSplitter(io).split("in.ifc", 2, (g) => `out_${g}.ifc`);
for (const [groupId, { ids }] of result) {
  console.log(groupId, [...ids].sort((a, b) => a - b).join(","));
}

Two ways to trigger it, one in each wall:

• #12 holds #999 in an attribute slot — a reference to a line the file never defines (ordinary in exports that drop entities).
• #15 has #99999 inside a quoted string, 'Wall B (see #99999)'.

Actual

group 0: 1,2,3,4,5,6,7,10,11,12,17,999
group 1: 1,2,3,4,5,6,7,13,14,15,17,99999

The highest id defined anywhere in the input is 17. Neither 999 nor 99999 is written to out_0.ifc / out_1.ifc.

Expected

group 0: 1,2,3,4,5,6,7,10,11,12,17
group 1: 1,2,3,4,5,6,7,13,14,15,17

Root cause

collectDeps (and collectDepsAll) mark an id visited before discovering whether it resolves:

const id = stack.pop()!;
if (visited.has(id)) continue;
visited.add(id);              // <-- unconditional
const refs = index.getRefs(id);
if (!refs) continue;          // <-- only now do we learn no line defines it

getRefs returns null only for an id that was never passed to LineIndex.set, i.e. one with no line of its own — set records a zero-length ref list for defined lines with no references, so an empty result is distinguishable from a missing one. By then the id is already in visited, which is the group's fileIds set and is what split() returns as ids.

The ids get there because extractRefs scans a line's raw text for # + digits with no validation that the target exists.

Impact

• ids is the natural input for building an id → group map. Sized by max(ids) it allocates for ids nothing can ever query; that bound is attacker/exporter-controlled through string content — a 9-digit number in a description field asks for a multi-GB array.
• Resolving such an id yields a group whose output file provably does not contain it, rather than "not found".
• GroupData.totalIds over-reports.

Fix

     const id = stack.pop()!;
     if (visited.has(id)) continue;
-    visited.add(id);
     const refs = index.getRefs(id);
     if (!refs) continue;
+    visited.add(id);

in both collectDeps and collectDepsAll. Nothing downstream regresses: no line is ever emitted for these ids, so removing them changes only what is reported. It also shrinks the sets slightly.

Related, not fixed by the above

extractRefs (packages/fragments/src/Utils/ifc-parsing-utils.ts) is not string-aware, so a #N in any quoted text is read as a reference. When N happens to name a real entity, that entity and its dependency subtree are pulled into the group — a text mention changes the output file's contents. Changing 'Wall B (see #99999)' to 'Wall B (see #10)' in the reproduction above adds 10 to group 1, which otherwise has no claim on it.

Whether the parser should skip single-quoted strings is a separate call — it touches every caller of extractRefs, not just the splitter.