Skip to content

Decorator call edges are incomplete and attributed to the enclosing scope #129

Description

@rahlk

Problem

Decorator application produces call-graph edges that are both incomplete and attributed
to the wrong node. On main (6f02581), -a 2 over a fixture whose module-level
plain_function carries @trace('hot') and @functools.lru_cache(maxsize=128), and
whose method Account.fetch carries @trace('method'):

app -> trace(tag) ['pycg']
Account -> trace(tag) ['pycg']
deco(fn) -> wraps ['jedi', 'pycg']

Two problems.

Misattributed. The edges hang off the enclosing module (app) and class (Account),
not off the decorated callable. That is faithful to evaluation order — a decorator does
run in the enclosing scope at definition time — but it means the graph cannot answer
"what decorates fetch", which is the question consumers actually ask. Both trace
edges also collapse to the same shape, so the two distinct decoration sites are
indistinguishable.

Incomplete.functools.lru_cache produced no edge at all, while functools.wraps in
the same file did. One of two decorators on the same function is represented.

Scope boundary

Makes decoration recoverable from the graph, and makes the edge set complete. Does not
add PyClass.decorators (#127) or resolve decorator strings (#128) — though a decoration
edge needs #128's ids to have a dst worth pointing at, so that one lands first. Does not
change how PyCG or Jedi resolve ordinary calls.

Goals

  • A decorated callable is reachable from its decorators in the graph, keyed on the
    decorated callable — not only on its enclosing scope
  • Every decorator in a stack is represented, including ones PyCG currently misses
    (functools.lru_cache in the fixture above)
  • Multiple decoration sites for the same decorator stay distinguishable
  • Decide and document whether decoration is a call_graph edge with distinct prov,
    or a separate overlay — a decoration is not an ordinary call site and conflating
    them changes what call-graph consumers count
  • Whichever it is, mirrored in the Neo4j projection

Caveats and known risks

  • The existing edges are not wrong, just not useful.app -> trace(tag) describes
    real evaluation order. If decoration becomes its own overlay these should probably
    stay, which means consumers see both — state that explicitly rather than silently
    double-counting.
  • PyCG's gap needs diagnosing before it is patched.lru_cache missing while wraps
    is present is unexplained; it may be PyCG's handling of decorators-with-arguments. Fix
    the cause, not the symptom — the same gap likely affects non-decorator calls.
  • L2 contract. This is call-graph shape, so it must not perturb L1 ⊆ L2. New edges
    at L2 are additive and safe; changing existing src endpoints is not, and would be a
    breaking change needing its own decision.
  • Depends on Structured decorator representation: PyDecorator on callable, class, attribute, parameter #128 for resolved decorator ids.

Definition of done

  • Given the fixture, the exact expected edge set is written down by hand first, then
    matched — not "non-empty"
  • Account.fetch is reachable from trace in the graph, and plain_function from both
    trace and functools.lru_cache
  • The two trace decoration sites are distinguishable from each other
  • analysis.json(-a 1) ⊆ analysis.json(-a 2) still holds
  • The PyCG lru_cache gap has a written root cause, not just a passing test

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions