Dataflow: Overlay informed dataflow.

[alexet]

This adds overlay informed Dataflow.

The idea is that for non-diff-informed configurations we use the diff informed mechanism to filter the sources and sinks to the overlay and then merge those results with the base results. This can lead to FPs/FNs due to flow paths via the overlay but the accuracy seems well within what we are targetting for overlay mode and saves lots of time.

Unfortunately the merging code cannot instantiated with CPP analysis as it uses EquivalenceRelation internally. This means we need separate parametrised modules (as we can't just guard with a predicate).

The local results are also exposed for cases where the pure local results are desired (e.g. for regexes where we are only going to care about analysing new regexes in the overlay).

[Copilot]

Pull Request Overview

This PR implements overlay-informed dataflow analysis capabilities for CodeQL. The changes introduce support for evaluating dataflow in overlay mode, where base and overlay database results can be merged, providing incremental analysis capabilities.

Key changes:

• Added overlay-informed filtering logic that complements the existing diff-informed mode
• Introduced new predicate variants for determining overlay evaluation context
• Created overlay-aware modules that merge base and overlay results for dataflow and taint tracking

Reviewed Changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
shared/util/codeql/util/AlertFiltering.qll	Added predicates to check diff information availability and file inclusion in diff ranges
shared/dataflow/codeql/dataflow/internal/DataFlowImplStage1.qll	Implemented overlay-informed filtering logic for sources and sinks with dual evaluation modes
shared/dataflow/codeql/dataflow/internal/DataFlowImpl.qll	Added OverlayImpl module that merges base and overlay dataflow results
shared/dataflow/codeql/dataflow/TaintTracking.qll	Refactored modules and added overlay-aware taint tracking implementations
shared/dataflow/codeql/dataflow/DataFlow.qll	Split into core and overlay-specific modules with new overlay evaluation support
java/ql/lib/semmle/code/java/dataflow/internal/DataFlowImplSpecific.qll	Added Java-specific overlay evaluation predicate implementation

Comments suppressed due to low confidence (1)

shared/dataflow/codeql/dataflow/TaintTracking.qll:1

• The condition should use Config::observeOverlayInformedIncrementalMode() without negation to match the logic in the flow predicate above. Currently this will include local results when overlay mode is disabled, which contradicts the expected behavior.

/**

[alexet]

@aschackmull Can you have a quick look over this to check that the general codee organisation is reasonable. This is the least duplication I could come up with.

[aschackmull]

Suggested change

	predicateflowLocal(Nodesource,Nodesink)= forceLocal(Base::flow/2)(source,sink)
	privatepredicateflowLocal(Nodesource,Nodesink)= forceLocal(Base::flow/2)(source,sink)

[alexet]

These were purposefully exported so they could be used in places like usedAsRegex without needing a new force local. (the force local could have been one predicate back but I was careful).

[aschackmull]

You made both local versions private - I only meant for the binary version to be private. I think it makes sense to expose the unary flowToLocal, as that has a reasonable local semantics, whereas a the binary predicate really only makes sense as a global predicate.

[alexet]

Ah I understand.

C++ has (had? it's been a while since I looked at that code) some cases where I think this would potentially be useful as they have Local dataflow done using global dataflow (they put a barrier at all calls, because they select source sink pairs using things like dominace, but use global dataflow because they want field flow). However I think that you are correct in that it is likely a trap and we can easily change it if we do find it useful.

[aschackmull]

Can you have a quick look over this to check that the general codee organisation is reasonable. This is the least duplication I could come up with.

Yeah, looks reasonable for now. I guess the plan is that the duplication can go away once all languages have an overlay-local Node?