Skip to content

unified: use a vendored-in copy of tree-sitter-swift - #21819

Merged
tausbn merged 4 commits into
mainfrom
tausbn/unified-vendor-in-tree-sitter-swift
May 12, 2026
Merged

unified: use a vendored-in copy of tree-sitter-swift#21819
tausbn merged 4 commits into
mainfrom
tausbn/unified-vendor-in-tree-sitter-swift

Conversation

@tausbn

Copy link
Copy Markdown
Contributor

For ease of iteration on the prototype.

@tausbntausbn added the no-change-note-required This PR does not need a change note label May 8, 2026
@tausbn
tausbn marked this pull request as ready for review May 8, 2026 15:33
CopilotAI review requested due to automatic review settings May 8, 2026 15:33
@tausbn
tausbn requested review from a team as code ownersMay 8, 2026 15:33
@tausbn
tausbn requested a review from asgerfMay 8, 2026 15:33

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR vendors the tree-sitter-swift grammar into unified/extractor to make iterating on the unified extractor’s Swift parsing prototype easier, and updates Rust/Bazel wiring to use the in-repo copy instead of the crates.io package.

Changes:

  • Add a vendored unified/extractor/tree-sitter-swift crate including generated parser sources, scanner, queries, and build scripts.
  • Switch unified/extractor from the registry tree-sitter-swift dependency to a local path dependency and add the new crate to the workspace.
  • Update Bazel third-party wiring to remove the crates.io tree-sitter-swift archive and add required deps (cc, tree-sitter-language) for the new local crate.
Show a summary per file
FileDescription
unified/extractor/tree-sitter-swift/tree-sitter.jsonAdds tree-sitter grammar metadata/config for Swift.
unified/extractor/tree-sitter-swift/src/tree_sitter/parser.hVendors tree-sitter parser API header used by generated sources/scanner.
unified/extractor/tree-sitter-swift/src/tree_sitter/array.hVendors tree-sitter internal array utilities used by generated sources.
unified/extractor/tree-sitter-swift/src/tree_sitter/alloc.hVendors tree-sitter allocator abstraction header.
unified/extractor/tree-sitter-swift/src/scanner.cAdds Swift external scanner implementation (comments/raw strings/semi handling/etc.).
unified/extractor/tree-sitter-swift/README.mdVendors upstream README for the grammar.
unified/extractor/tree-sitter-swift/queries/textobjects.scmAdds textobject queries for Swift.
unified/extractor/tree-sitter-swift/queries/tags.scmAdds tags queries for symbol definitions.
unified/extractor/tree-sitter-swift/queries/outline.scmAdds outline queries for structure extraction.
unified/extractor/tree-sitter-swift/queries/locals.scmAdds locals queries (definitions/scopes).
unified/extractor/tree-sitter-swift/queries/injections.scmAdds injection queries (regex/comment injections).
unified/extractor/tree-sitter-swift/queries/indents.scmAdds indentation queries.
unified/extractor/tree-sitter-swift/queries/highlights.scmAdds syntax highlighting queries.
unified/extractor/tree-sitter-swift/queries/folds.scmAdds folding queries.
unified/extractor/tree-sitter-swift/package.jsonVendors upstream Node package metadata for the grammar.
unified/extractor/tree-sitter-swift/LICENSEAdds upstream MIT license for the vendored grammar.
unified/extractor/tree-sitter-swift/grammar.jsVendors the Swift grammar definition.
unified/extractor/tree-sitter-swift/Cargo.tomlAdds a local Rust crate wrapper for the vendored Swift grammar.
unified/extractor/tree-sitter-swift/BUILD.bazelAdds Bazel rules to build the vendored grammar as a Rust library.
unified/extractor/tree-sitter-swift/bindings/rust/lib.rsProvides LanguageFn and embeds node-types/queries; includes basic tests.
unified/extractor/tree-sitter-swift/bindings/rust/build.rsBuilds parser.c + scanner.c via cc during Rust builds.
unified/extractor/tree-sitter-swift/bindings/node/index.jsVendors Node binding loader.
unified/extractor/tree-sitter-swift/bindings/node/binding.ccVendors Node binding implementation exporting the language.
unified/extractor/tree-sitter-swift/binding.gypVendors Node-gyp build configuration for the Node binding.
unified/extractor/Cargo.tomlSwitches tree-sitter-swift dependency to local path.
unified/extractor/BUILD.bazelAdds the new local tree-sitter-swift Bazel target as a dependency.
MODULE.bazelAdds Bazel module repos for cc and tree-sitter-language; removes crates.io tree-sitter-swift repo.
misc/bazel/3rdparty/tree_sitter_extractors_deps/defs.bzlRemoves vendored crates.io tree-sitter-swift archive; adds mappings for the new local crate + its deps.
misc/bazel/3rdparty/tree_sitter_extractors_deps/BUILD.tree-sitter-swift-0.7.2.bazelDeletes the autogenerated BUILD file for the removed crates.io tree-sitter-swift dependency.
misc/bazel/3rdparty/tree_sitter_extractors_deps/BUILD.bazelAdds aliases for cc and tree-sitter-language.
Cargo.tomlAdds the vendored tree-sitter-swift crate as a workspace member.
Cargo.lockConverts tree-sitter-swift from registry source to a workspace package entry (removes source/checksum).

Copilot's findings

Comments suppressed due to low confidence (1)

unified/extractor/tree-sitter-swift/Cargo.toml:22

  • This crate’s Rust tests and doctest example reference the tree_sitter crate (tree_sitter::Parser), but Cargo.toml doesn’t declare a tree-sitter dependency (and it can’t be used transitively). Add an explicit tree-sitter dependency (or at least a dev-dependency) so cargo test/doctests compile.
# When updating these dependencies, run `misc/bazel/3rdparty/update_cargo_deps.sh`
[dependencies]
tree-sitter-language = "0.1"
[build-dependencies]
cc = "1.2"
  • Files reviewed: 31/35 changed files
  • Comments generated: 1

Comment on lines +705 to +711
#define DIRECTIVE_COUNT 4
const char* DIRECTIVES[OPERATOR_COUNT] = {
"if",
"elseif",
"else",
"endif"
};
@asgerf

Copy link
Copy Markdown
Contributor

This adds 600k lines of code, of which 550k comes from the auto-generated parser.cc and 30k from the generated node-types.json.

I think we should go against tree-sitter conventions and avoid checking it in these generated artifacts, and instead rely on Bazel rules to rebuild when needed. WDYT?

@tausbn

Copy link
Copy Markdown
ContributorAuthor

This adds 600k lines of code, of which 550k comes from the auto-generated parser.cc and 30k from the generated node-types.json.

I think we should go against tree-sitter conventions and avoid checking it in these generated artifacts, and instead rely on Bazel rules to rebuild when needed. WDYT?

Good idea. I'll try to set it up.

tausbn added 2 commits May 12, 2026 11:24
Uses the `tree-sitter-generate` crate to generate these files on the
fly.
@tausbn

tausbn commented May 12, 2026

Copy link
Copy Markdown
ContributorAuthor
Rerun has been triggered: 2 restarted 🚀

@tausbn
tausbn merged commit 9c958a4 into mainMay 12, 2026
80 checks passed
@tausbn
tausbn deleted the tausbn/unified-vendor-in-tree-sitter-swift branch May 12, 2026 12:55
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentationno-change-note-requiredThis PR does not need a change note

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@tausbn@asgerf