Skip to content

feat(commonlisp): resolve calls to functions defined in other files - #3545

Open
fade wants to merge 3 commits into
Graphify-Labs:v8from
fade:commonlisp-crossfile-calls
Open

fade wants to merge 3 commits into
Graphify-Labs:v8from
fade:commonlisp-crossfile-calls

Conversation

@fade

@fade fade commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Builds on #3543 and #3544. Those commits are included here as a base; the change this PR is about is the final commit, 7222d75.

A defun calling a function defined in another file produces no calls edge.

The call pass builds label_to_nid from nodes, which for a per-file extractor holds only the
definitions in the file being parsed. A Common Lisp system spreads its functions across many files
of one package and calls them by bare name, with no receiver at the call site and no import naming
the target, so the per-file pass resolves a minority of the real calls and drops the rest. The call
graph looks complete and is mostly missing.

Approach

The extractor now reports what it cannot see as raw_calls instead of dropping it, and a resolver
registered in resolver_registry binds those against the merged corpus once every file is known.

pascal_resolution.py was the model. It closes the same shape of gap, a call whose target lies
structurally outside any one file's scope, and the call shapes common in the projects it was written
for are close enough to Lisp's to make it the right thing to build on. The resolution rule differs
because the languages do: Pascal walks an inherits chain to mirror method lookup, while a bare
name in a Lisp package denotes one function, so the corpus-wide name is the resolution.

A name matching more than one definition goes through the shared tie-breakers and is dropped unless
exactly one survives, so an ambiguous name yields no edge rather than a guess. Resolved edges are
INFERRED, since a name match across the corpus is weaker evidence than a call the extractor
resolved inside one file.

Calls into the standard library need no special handling and get none. Nothing in the corpus defines
car or format, so they match no candidate and add no node. That falls out of matching against
definitions rather than against a name list, which is why no such list is maintained.

Reproduction

;; a.lisp
(defun helper (x) (* x 2))
(defun same-file-caller (x) (helper x))   ; edge present

;; b.lisp
(defun cross-file-caller (x) (helper x))  ; no edge

Build the graph over a directory holding both files. Extracting b.lisp alone cannot show this
either way, which is the point: the information is not in that file.

Six tests in tests/test_commonlisp_resolution.py, three of them controls: a same-file call stays
EXTRACTED, an ambiguous name produces no edge, and standard-library calls add no nodes.

…ther file

A defmethod dispatching on a class defined in another file lost its specializes
edge. The specializer resolved to a file-scoped id with no backing node, so the
dangling-edge filter pruned the edge. Resolve through the same sourceless stub
the cross-file superclass path uses, so the corpus rewire collapses it onto the
real defclass; a same-file specializer still binds locally.

Most methods in a Common Lisp system live apart from the class they dispatch
on, so this removed nearly every specializes edge in a real codebase.

Add specializes to the supertype relations at the same time. Functions and
types occupy separate namespaces in Common Lisp, so one symbol is routinely
both, and without the guard a specializer stub can collapse onto a same-named
function and assert a dispatch relationship that does not exist.
…nt types

define-condition was routed to the generic definer path, which records the name
and never reads the parent list, so a condition hierarchy reached the graph as
unrelated nodes even with both ends in the same file. It shares defclass's
shape, (NAME (PARENTS) (SLOTS) ...), so the defclass handler covers it and the
class path keeps working unchanged.

Conditions are how a Common Lisp program signals, so for a codebase that leans
on them this left a large part of its structure invisible.
A Common Lisp system spreads its functions across many files of one package and
calls them by bare name, with no receiver at the call site and no import naming
the target. The per-file extractor sees only the file it is parsing, so it could
resolve a minority of the real calls and silently dropped the rest: a call graph
that looked complete and was mostly missing.

The extractor now reports what it cannot see as raw_calls, and a resolver runs
over the merged corpus and binds a bare name to its definition when exactly one
definition carries that name. An ambiguous name goes through the shared
tie-breakers and is dropped unless one candidate survives, so a guess is never
emitted. Resolved edges are INFERRED, since a name match across the corpus is
weaker evidence than a call resolved inside a single file.

Modelled on the Pascal resolver, which closes the same shape of gap for a call
whose target lies outside any one file's scope; the resolution rule differs
because Pascal walks an inherits chain to mirror method lookup, while a bare
name in a Lisp package denotes one function.

Calls into the standard library need no special handling and get none: nothing
in the corpus defines car or format, so they match nothing and add no node.

@graphify-labs graphify-labs Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Graphify reviewed this change.

Worth a look — the grounded gate found no coupling regressions or blocking issues, but 1 advisory finding(s) below merit a look before merge.

Formal verification. No changes could be formally verified in this run.


Graphify review — findings

Adds cross-file call resolution for Common Lisp: extract_commonlisp now reports bare-name calls it can't resolve locally as raw_calls, and the new resolve_commonlisp_calls resolver (registered for .lisp/.cl/.lsp/.asd) binds them against the merged corpus, emitting INFERRED calls edges only when exactly one definition matches — ambiguous names go through the shared tie-breakers and are dropped otherwise, and standard-library names match nothing and produce no edge. Routes define-condition through the defclass handler so condition hierarchies get their inherits edges, and points method specializers at a class ref via ensure_class_ref instead of a name-derived id. Treats specializes as a supertype relation in _rewire_unique_stub_nodes so a specializer stub can't rewire onto a same-named function, since CL methods dispatch on types that share names with functions.

Worth a look

  • Common Lisp resolver can bind calls to non-Common-Lisp nodesgraphify/commonlisp_resolution.py:66 · Escalate · medium
    • agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 2301 functions depend on the 838 functions this change touches.

Health — this change adds coupling hotspots:

  • new: extract() — 596 callers, 44 callees
  • new: _rebuild_code() — 116 callers, 51 callees
  • new: extract_js() — 85 callers, 4 callees
  • new: extract_xaml() — 19 callers, 17 callees
  • new: dispatch_command() — 2 callers, 124 callees
  • new: extract_commonlisp() — 31 callers, 6 callees
  • new: _get_extractor() — 26 callers, 6 callees
  • new: run_pipeline() — 8 callers, 13 callees
  • …and 33 more — each is listed as a finding

Verification — 2301 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

  • verification_scope: 2126 function(s) in the blast radius were not formally verified this run

Test selection

Test selection

110 of 277 test file(s) selected (40%) via static blast radius.

  • tests/test_astro_extraction.py — impact
  • tests/test_astro_import_ids.py — impact
  • tests/test_build.py — impact
  • tests/test_builtin_global_type_refs.py — impact
  • tests/test_case_sensitive_resolution.py — impact
  • tests/test_cjs_module_extension.py — impact
  • tests/test_commonlisp_resolution.py — impact, changed-test
  • tests/test_commonlisp_specializer_binding.py — impact, changed-test
  • tests/test_cpp_nested_and_cli.py — impact
  • tests/test_cpp_objc_cross_file_calls.py — impact
  • tests/test_cross_extension_reexport_self_cycle.py — impact
  • tests/test_cross_language_call_resolution.py — impact
  • tests/test_cross_repo_member_calls.py — impact
  • tests/test_csharp_call_site_generic_args.py — impact
  • tests/test_csharp_enum_members.py — impact
  • tests/test_csharp_field_generic_args.py — impact
  • tests/test_csharp_generic_callsites.py — impact
  • tests/test_csharp_interface_dispatch.py — impact
  • tests/test_csharp_member_calls.py — impact
  • tests/test_csharp_member_nodes.py — impact
  • tests/test_csharp_object_creation.py — impact
  • tests/test_csharp_partial_classes.py — impact
  • tests/test_csharp_type_resolution.py — impact
  • tests/test_definition_file_portability.py — impact
  • tests/test_detect.py — impact
  • tests/test_dotnet.py — impact
  • tests/test_duplicate_annotation_edges.py — impact
  • tests/test_extract.py — impact
  • tests/test_extract_cache_location.py — impact
  • tests/test_file_label_disambiguation.py — impact
  • tests/test_file_node_id_spec.py — impact
  • tests/test_forwarding_review_findings.py — impact
  • tests/test_go_builtin_call_targets.py — impact
  • tests/test_go_qualified_resolution.py — impact
  • tests/test_import_extension_resolution.py — impact
  • tests/test_import_self_loops.py — impact
  • tests/test_imported_export_forwarding.py — impact
  • tests/test_incremental.py — impact
  • tests/test_indirect_call_arrow_single_param_shadow.py — impact
  • tests/test_indirect_call_catch_binding_shadow.py — impact
  • tests/test_indirect_call_external_import_shadow.py — impact
  • tests/test_indirect_call_for_of_binding_shadow.py — impact
  • tests/test_indirect_call_function_expression_shadow.py — impact
  • tests/test_indirect_call_nested_closure_shadow.py — impact
  • tests/test_indirect_dispatch.py — impact
  • tests/test_indirect_dispatch_assign_return.py — impact
  • tests/test_indirect_dispatch_getattr.py — impact
  • tests/test_inferred_confidence_rubric.py — impact
  • tests/test_inherited_field_receivers.py — impact
  • tests/test_issue_3405_python_resolution.py — impact
  • … and 60 more

Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.

Formal verification

Could not verify: Could not verify \_rewire\_unique\_stub\_nodes.

The verifier did not have enough to check \_rewire\_unique\_stub\_nodes, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: not verifiable: the input domain has 81 values but only 9 distinct were tested — a small finite domain must be EXHAUSTED, not sampled (an untested input could invert the result)

Could not verify: Could not verify extract\_commonlisp.

The verifier did not have enough to check extract\_commonlisp, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set

· 41 more finding(s) on lines outside this diff (see the check run).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant