Problem
The same builtin function gets two distinct can:// identities, because Jedi and PyCG spell
the builtins module differently and nothing normalizes it before ids are minted.
Measured on test/fixtures/whole_applications/requests at -a 2 (main @ 6f02581):
external_symbols under @external/<builtin>/ : 29
external_symbols under @external/builtins/ : 14
names present under BOTH spellings : 12
all, any, chr, getattr, hasattr, isinstance, iter, len, max, print, setattr, sorted
call_graph edges -> @external/<builtin>/... : 130 all prov=['pycg']
call_graph edges -> @external/builtins/... : 79 all prov=['jedi']
The provenance split is exact: <builtin> is PyCG's spelling, builtins is Jedi's.
core.py:_home_external_symbols derives the id by splitting the dotted signature on its last
dot (module, name = sig.rsplit(".", 1)) and never canonicalizes module, so both spellings
mint their own @external home.
Two consequences:
- A consumer asking "who calls
len" gets two disjoint answers, and neither is complete. - Provenance can never merge for builtins.
merge_edges coalesces on (src, dst); because
the two backends produce different dst ids for the same target, a call both resolvers agree
on can never reach prov: ["jedi", "pycg"]. In the same run, 198 non-builtin edges do carry
both — so the merge works, and builtins are structurally excluded from it.
PyCG only ever emits the bare <builtin> module (69 occurrences across requests + flask);
every dotted form (builtins.str, builtins.dict, …) comes from Jedi. So an exact-match alias
is sufficient — no prefix rewriting needed.
Scope boundary
Canonicalizes PyCG's module spelling so one symbol has one id. Does not change how either
resolver resolves calls, does not touch the @external id grammar, does not address Jedi's
resolution failures on attribute calls (a capability limit, not a bug), and does not change
body.callee backfill — which was verified correct: across all 234 unresolved call sites in
requests, every one has callee_signature is None, and there are zero cases where a resolved
signature failed to reach its body node.
Goals
Caveats and known risks
- Normalization must happen before
merge_edges, not at id-minting time.core.py:598
merges Jedi and PyCG edges on raw dotted signatures, and ids are minted later at line 633.
Canonicalizing at the minting site would leave two already-merged edges with identical
endpoints and split provenance — the symptom would move rather than disappear. - Edge weights change. Coalescing two spellings into one sums their weights, so any test
pinning a builtin edge weight will need updating. That is the correct new value, not a
regression. _coalesce_edges (pycg_analysis.py:468) is broken and sits on this path (reachable from
line 710): it constructs PyCallEdge(source=…, target=…) and reads existing.source, but the
model's fields are src/dst with no aliases — so it raises whenever two shards produce the
same (src, dst). This fix must not route through it. Tracked separately.<builtin> is PyCG's convention and could change if PyCG is ever replaced; the alias map
should be named for what it is (a backend spelling quirk), not treated as a general grammar.
Definition of done
@external/<builtin>/ appears zero times in analysis.json for requests, flask, xarray- The 12 doubly-spelled names collapse to one id each, with weights summed
- At least one builtin edge carries
prov: ["jedi", "pycg"] where previously none could - A regression test asserts a hand-written expected id for a builtin call resolved by both backends
- Existing call-graph tests pass, with any weight changes updated deliberately
Problem
The same builtin function gets two distinct
can://identities, because Jedi and PyCG spellthe builtins module differently and nothing normalizes it before ids are minted.
Measured on
test/fixtures/whole_applications/requestsat-a 2(main@6f02581):The provenance split is exact:
<builtin>is PyCG's spelling,builtinsis Jedi's.core.py:_home_external_symbolsderives the id by splitting the dotted signature on its lastdot (
module, name = sig.rsplit(".", 1)) and never canonicalizesmodule, so both spellingsmint their own
@externalhome.Two consequences:
len" gets two disjoint answers, and neither is complete.merge_edgescoalesces on(src, dst); becausethe two backends produce different
dstids for the same target, a call both resolvers agreeon can never reach
prov: ["jedi", "pycg"]. In the same run, 198 non-builtin edges do carryboth — so the merge works, and builtins are structurally excluded from it.
PyCG only ever emits the bare
<builtin>module (69 occurrences acrossrequests+flask);every dotted form (
builtins.str,builtins.dict, …) comes from Jedi. So an exact-match aliasis sufficient — no prefix rewriting needed.
Scope boundary
Canonicalizes PyCG's module spelling so one symbol has one id. Does not change how either
resolver resolves calls, does not touch the
@externalid grammar, does not address Jedi'sresolution failures on attribute calls (a capability limit, not a bug), and does not change
body.calleebackfill — which was verified correct: across all 234 unresolved call sites inrequests, every one hascallee_signature is None, and there are zero cases where a resolvedsignature failed to reach its body node.
Goals
<builtin>→builtinson PyCG edge endpoints, at the single exit point(
PyCG.build_call_graph_edges) so every shard strategy is coveredweight instead of emitting two edges with identical endpoints
prov: ["jedi", "pycg"]now appears for builtins both resolvers agree on@external/<builtin>/id remains in any emitted payloadCaveats and known risks
merge_edges, not at id-minting time.core.py:598merges Jedi and PyCG edges on raw dotted signatures, and ids are minted later at line 633.
Canonicalizing at the minting site would leave two already-merged edges with identical
endpoints and split provenance — the symptom would move rather than disappear.
pinning a builtin edge weight will need updating. That is the correct new value, not a
regression.
_coalesce_edges(pycg_analysis.py:468) is broken and sits on this path (reachable fromline 710): it constructs
PyCallEdge(source=…, target=…)and readsexisting.source, but themodel's fields are
src/dstwith no aliases — so it raises whenever two shards produce thesame
(src, dst). This fix must not route through it. Tracked separately.<builtin>is PyCG's convention and could change if PyCG is ever replaced; the alias mapshould be named for what it is (a backend spelling quirk), not treated as a general grammar.
Definition of done
@external/<builtin>/appears zero times inanalysis.jsonforrequests,flask,xarrayprov: ["jedi", "pycg"]where previously none could