Uh oh!
There was an error while loading. Please reload this page.
feat: L1 emission in canonical schema v2 (opt-in via --schema v2) - #188
Open
sinha108 wants to merge 22 commits into
Open
feat: L1 emission in canonical schema v2 (opt-in via --schema v2)#188sinha108 wants to merge 22 commits into
sinha108 wants to merge 22 commits into
Conversation
…l id (#180) Key body nodes by bare local id (line:col) per the keystone, and emit call nodes for constructor invocations and this()/super() chaining so L2 can resolve those edges.
…s_variadic (#180) Add V2Json (snake_case keys, no nulls emitted), module span/imports/ content_hash, field kind discriminator and parameter is_variadic, with a serialization-contract test covering the emitted key names.
…ode_start_line, comments (#180)
) Types, supertypes, error_channel and refs.types are now resolved to qualified names, and callable signatures use erased resolved parameter types, matching the v1 symbol table. Tests parse with a symbol solver so the resolution path is actually exercised; unresolvable types degrade to their AST spelling.
… callee signature, flags) (#180)
…tion (#180) v2 is opt-in (v1 stays the default until the rest of the migration lands). Unsupported combinations (-a > 1, --emit neo4j, --source-analysis, --target-files, unknown --schema) fail with a clear non-zero error rather than silently emitting a different shape. stdout carries compact JSON only.
Adds the strict in-repo schema used as the L1 oracle until the SDK's v2 models exist, a gate over the in-repo fixtures in the default suite, and a realWorldConformanceTest task for the submodule applications (too slow for the inner loop, but required).
…ag (#180) L1 now downloads the project's dependencies and puts a JarTypeSolver on the solver path, so third-party types resolve to qualified names as they do in v1 (verified on spring-petclinic: Model, Pageable, Page<Owner>, and callee_signature on 99% of call sites). Reflection is restricted to the JRE so the analyzer's own dependencies can no longer be resolved as if the analysed project depended on them. is_static_call becomes a Boolean that is omitted when the callee is unresolved rather than reported as false.
The repo is deployment tooling (Makefile, docker-compose, helm charts) with zero .java files, so it cannot exercise the analyzer. Also ignores output/, used for ad-hoc v1/v2 comparison runs.
Anonymous class bodies now get their own type node ($anon$N) under the
callable that declares them, like named local classes — v1 mis-attributed
their initializers and locals to the enclosing type and the first v2 attempt
dropped them. Re-measured, initializer blocks and local variables are back
at parity with v1.
body_span delimits the { ... } block so source[body_span.bytes] reproduces
v1's per-callable code byte for byte without duplicating the text; the
callable's own span covers the whole declaration. Pinned by a test that
compares against the v1 emitter directly.…arison report Anonymous classes occur in two places: inside a callable body and inside a field initializer, which belongs to no callable. The latter was missed, so commons-lang's AnnotationUtils lost the double-brace initializer configuring its ToStringStyle. Adds docs/design/notes/l1-v1-v2-comparison.md, generated from twenty runs (ten applications x both schemas). v2 matches or exceeds v1 on every structural metric; the two remaining negative deltas are v1 counting bugs (anonymous-class fields reported as method locals, nested initializer blocks counted twice) that v2 does not reproduce.
…, --eager) Reuses modules whose files are byte-for-byte unchanged, skipping the parse as well as the build: commons-lang goes from 130s cold to 4s warm. Caching is opt-in, and the cache is discarded wholesale when the app name or analyzer version changes since both are baked into every can:// id. The extractor now enumerates and hashes files itself instead of parsing whole source roots up front; module discovery is unchanged (commons-lang still yields 625 modules, matching v1).
- Constructor callee_signature normalises to <init> so it joins against the target callable's signature; otherwise L2 drops every constructor edge (88 of petclinic's call sites). - Expression-type resolution no longer memoises failures by expression text: the same text can denote different types in different scopes of one file, so a failure blanked later resolvable occurrences. - metrics.cyclomatic is scope-filtered like every other callable fact, so branches inside a nested or anonymous class are no longer double-counted. - Call sites with no source range are skipped rather than colliding on 0:0 and silently overwriting one another. - Module span end position is computed for universal newlines and for files with no trailing newline. - Corrected the byteOffsets javadoc: the range is end-exclusive.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements L1 emission in the canonical schema v2: the additive CPG tree (
application → module → type → callable → body) withcan://ids, byte-offset spans, and a per-modulesourceblob.v2 is opt-in via
--schema v2; v1 remains the default, so no existing consumer is affected. Unsupported combinations (-a > 1,--emit neo4j,--source-analysis,--target-files, an unknown--schemavalue) exit non-zero with a clear message rather than silently emitting a different shape.Delivered
syntactic_analysis/) — one per node kind, sharing anL1BuildContext. Built from the JavaParser AST rather than mapped from the v1 entities, which cannot supply byte offsets or annotation arguments.org.springframework.ui.Model,Page<…Owner>); reflection is restricted to the JRE so the analyzer's own dependencies are never mistaken for the project's. Signatures use erased resolved parameter types.kind:"initializer"callables), field initializers, modifiers,declaration,code_start_line,is_implicit, entrypoint flags, and resolved call-site facts (receiver, argument types, callee signature, static/constructor flags).typenodes ($anon$N), in callable bodies and field initializers.callable.body_span, sosource[body_span.bytes]reproduces v1's per-callablecodebyte for byte without duplicating text.-c/--cache-dir,--eager) keyed oncontent_hash, skipping the parse as well as the build:commons-lang130s cold → 4s warm../gradlew realWorldConformanceTest.Verification
docs/design/notes/l1-v1-v2-comparison.md— twenty runs (ten real-world applications × both schemas), generated from the payloads so it cannot drift from the data.v2 matches or exceeds v1 on every structural metric: modules 1081 = 1081, enum constants / record components / entrypoints exact, and more types (+233), callables (+256) and call sites (+416) — v1 dropped local classes and missed
this(...)/super(...)chaining. Resolution is at parity (95.8% qualified) with callee signatures on 99% of call sites. All 1081 v1symbol_tablekeys are absolute paths; v2 has none.Of four negative deltas, none is a v2 defect:
local variables(−11) andinitializer blocks(−1) are v1 counting bugs v2 declines to reproduce;commentsis mostly v1 duplication (2.09×) plus body-internal comments, which need L3's statement nodes;CRUDis deferred to #187.Design decisions:
.claude/SCHEMA_DECISIONS.md. Spec:docs/design/specs/schema-v2-l3-l4-design.md.Not in this PR
call_graphedges and thecalleebackfill — codeanalyzer-java: L2 v2 call_graph (identity edges + external_symbols) #181L1 ⊆ L2) cannot be exercised until L2 exists;-jfan-out is not implemented, though ordering is sorted throughout so parallelising later stays deterministic.Closes#180