Skip to content

JiT Worker in CLI and RPC infrastructure - #2109

Merged
rafal-hawrylak merged 2 commits into
mainfrom
cli_jit_workers
May 20, 2026
Merged

JiT Worker in CLI and RPC infrastructure#2109
rafal-hawrylak merged 2 commits into
mainfrom
cli_jit_workers

Conversation

@rafal-hawrylak

@rafal-hawrylakrafal-hawrylak commented Mar 9, 2026

Copy link
Copy Markdown
Collaborator

This change establishes the foundation for executing JiT compilation in an isolated environment. It introduces the worker process management, the RPC bridge for database access during JiT, and the necessary Bazel targets.

@rafal-hawrylak
rafal-hawrylak marked this pull request as ready for review March 9, 2026 19:59
@rafal-hawrylak
rafal-hawrylak requested a review from a team as a code ownerMarch 9, 2026 19:59
@rafal-hawrylak
rafal-hawrylak enabled auto-merge (squash) March 9, 2026 19:59
@rafal-hawrylakrafal-hawrylak self-assigned this Mar 9, 2026
kolina
kolina requested changes Mar 10, 2026
View reviewed changes
Comment threadcli/api/commands/base_worker.ts Outdated
Comment threadcli/api/commands/jit_compile_child_process.ts Outdated
Comment threadcli/api/commands/jit/compiler.ts
Comment threadcli/api/commands/jit_compile_child_process.ts Outdated
Comment threadcli/api/commands/base_worker.ts
Comment threadcli/api/utils/constants.ts
Comment threadcli/vm/jit_worker.ts
Comment threadcli/vm/jit_worker.ts
Comment threadcli/api/commands/jit/rpc_test.ts Outdated
Comment threadcli/api/commands/jit/rpc.ts
apilaskowski
apilaskowski reviewed Mar 11, 2026
View reviewed changes
Comment threadcli/api/commands/jit_compile_child_process.ts Outdated
ikholopov-omni
ikholopov-omni requested changes Mar 11, 2026
View reviewed changes
Comment threadprotos/jit.proto Outdated
Comment threadcli/api/commands/jit/rpc.ts Outdated
@rafal-hawrylak
rafal-hawrylakforce-pushed the cli_jit_workers branch 10 times, most recently from a05cf58 to be56edbCompareMarch 12, 2026 15:50
kolina
kolina requested changes Mar 14, 2026
View reviewed changes
Comment threadcli/api/commands/jit/rpc.ts
Comment threadcli/api/commands/jit/rpc.ts Outdated
Comment threadcli/api/commands/jit/rpc.ts Outdated
Comment threadcli/api/dbadapters/bigquery.ts Outdated
Comment threadcli/api/dbadapters/bigquery.ts Outdated
Comment threadcli/vm/jit_worker.ts Outdated
Comment threadcli/vm/jit_worker.ts Outdated
Comment threadcli/vm/jit_worker.ts Outdated
@rafal-hawrylak
rafal-hawrylakforce-pushed the cli_jit_workers branch 5 times, most recently from 9ca1f45 to 0fe90aaCompareMarch 27, 2026 18:00
kolina
kolina reviewed Mar 28, 2026
View reviewed changes
Comment threadcli/api/commands/jit/rpc.ts Outdated
Comment threadcli/api/commands/jit/rpc_test.ts
Comment threadcli/api/commands/jit/rpc_test.ts Outdated
Comment threadcli/api/dbadapters/bigquery.ts Outdated
Comment threadcli/api/dbadapters/bigquery.ts Outdated
Comment threadcli/api/dbadapters/bigquery.ts Outdated
Comment threadcli/vm/jit_worker.ts Outdated
@rafal-hawrylak
rafal-hawrylakforce-pushed the cli_jit_workers branch 2 times, most recently from c6650a7 to 22d3661CompareMarch 29, 2026 08:40
rafal-hawrylak
rafal-hawrylak commented Mar 29, 2026
View reviewed changes
Comment threadcli/api/dbadapters/bigquery.ts
@rafal-hawrylak
rafal-hawrylakforce-pushed the cli_jit_workers branch 3 times, most recently from 38433f2 to e687190CompareMarch 30, 2026 11:01
kolina
kolina reviewed Apr 3, 2026
View reviewed changes
Comment threadcli/api/dbadapters/bigquery.ts
Comment threadcli/api/dbadapters/bigquery.ts
Comment threadcli/api/dbadapters/bigquery.ts
Comment threadcli/api/commands/jit/rpc.ts Outdated
kolina
kolina reviewed Apr 11, 2026
View reviewed changes

@kolinakolina left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current PR LGTM, there are a couple of questions about diffs in BigQuery adapters.

After they're responded, I can approve from my side

@rafal-hawrylakrafal-hawrylak mentioned this pull request Apr 28, 2026
Closed
@apilaskowskiapilaskowski mentioned this pull request Apr 29, 2026
Merged
@apilaskowski

apilaskowski commented May 18, 2026

Copy link
Copy Markdown
Collaborator

@kolina can you take a look here?

ikholopov-omni
ikholopov-omni requested changes May 20, 2026
View reviewed changes

@ikholopov-omniikholopov-omni left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM, feel free to submit once the comments are addressed.

Comment threadcommon/protos/structs.ts Outdated
Comment threadcommon/protos/structs.ts Outdated
@rafal-hawrylak
feat: implement JiT worker and RPC infrastructure
8c73246
* feat: add worker process management for JiT compilation
- Introduce base worker and JiT-specific child process logic
- Implement RPC bridge for database access during JiT
- Add VM scripts and loader for isolated execution
- Add unit tests for the RPC mechanism
@rafal-hawrylak

rafal-hawrylak commented May 20, 2026

Copy link
Copy Markdown
CollaboratorAuthor

Overall LGTM, feel free to submit once the comments are addressed.

Addressed. However the merged is blocked due to request for a change.

@rafal-hawrylak
rafal-hawrylak merged commit 433e852 into mainMay 20, 2026
6 checks passed
@rafal-hawrylak
rafal-hawrylak deleted the cli_jit_workers branch May 20, 2026 10:56
@jorgecortesvbjorgecortesvb mentioned this pull request May 25, 2026
Closed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kolinakolinakolina approved these changes
@apilaskowskiapilaskowskiapilaskowski approved these changes
@ikholopov-omniikholopov-omniikholopov-omni approved these changes
@andrzej-grudzienandrzej-grudzienAwaiting requested review from andrzej-grudzien

Assignees

@rafal-hawrylakrafal-hawrylak

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@rafal-hawrylak@apilaskowski@kolina@ikholopov-omni