Skip to content

fix server_native callback wakeups and request uri handling - #12

Draft
kingwill101 wants to merge 14 commits into
masterfrom
fix/10-server-native-isolate-scaling
Draft

kingwill101 wants to merge 14 commits into
masterfrom
fix/10-server-native-isolate-scaling

Conversation

@kingwill101

Copy link
Copy Markdown
Owner

Summary

  • restore absolute HttpRequest.uri / requestedUri reconstruction for server_native bridge requests so the request scheme remains available to framework code
  • replace the idle backoff loop in the default native callback path with callback-driven queue draining so queued direct frames wake Dart immediately
  • extend native callback regression coverage for HttpRequest compatibility and bridge URI behavior

Why

A recent performance refactor left two regressions on the issue #10 branch:

  • BridgeHttpRequest.uri became origin-form for normal requests, which cleared httpRequest.uri.scheme and broke downstream consumers that derive request scheme from HttpRequest.uri
  • the default native callback transport still relied on exponential sleep while directRequestCallback was unset, so the first queued frame after idle could sit until the next poll window expired

Impact

Root Cause

The branch that reduced isolate-scaling overhead switched callback request URI materialization to a relative URI and changed the direct queue drain loop to idle/backoff polling without wiring Rust's existing direct callback wake signal into Dart.

Validation

  • dart analyze packages/server_native/lib/src/bridge/bridge_runtime_request.dart packages/server_native/lib/src/bridge/bridge_runtime_request_uri.dart packages/server_native/lib/src/server_boot_proxy_direct.dart packages/server_native/test/bridge_runtime_test.dart packages/server_native/test/serve_ffi_http_test.dart
  • dart test packages/server_native/test/bridge_runtime_test.dart
  • dart test test/serve_ffi_http_test.dart from packages/server_native
  • dart test test/ffi_http_server_test.dart --plain-name "NativeHttpServer reconstructs requestedUri from forwarded headers" from packages/server_native

Issue #10 Repro Check

Local wrk run on an 8-logical-CPU Linux host with -t8 -c200 -d15s:

Case req/s p50 p99
dart:io x1 18652.21 10.66 ms 15.19 ms
dart:io x8 37173.81 4.74 ms 18.77 ms
server_native x1 29178.05 6.73 ms 10.40 ms
server_native x8 46046.45 3.94 ms 12.18 ms

That reproducer no longer shows the flat multi-isolate behavior from #10; server_native scaled by about 57.8% on this host.

Fixes #10

Cap shared proxy runtimes to one worker thread and back off native direct polling when idle. Also coalesce direct wakeup callbacks and keep regression tests for the new runtime behavior.
Use synchronous request delivery for NativeHttpServer and drain short bursts of queued direct frames before yielding. This removes extra event queue hops in the callback path and improves framework benchmarks in native mode.
Use a direct encoded-payload fast path for non-detached callback responses and fall back to full bridge frames only when detach/tunnel semantics are needed. This cuts temporary response-header allocations in the native callback path.
@vercel

vercel Bot commented Apr 8, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
routed Ready Ready Preview, Comment Apr 9, 2026 7:23pm

@docs-page

docs-page Bot commented Apr 8, 2026

Copy link
Copy Markdown

To view this pull requests documentation preview, visit the following URL:

docs.page/kingwill101/routed~12

Documentation is deployed and generated using docs.page.

@coderabbitai

coderabbitai Bot commented Apr 8, 2026

Copy link
Copy Markdown

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: d4f08d3b-ffd3-4a57-8986-ea53e20c2d47

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/10-server-native-isolate-scaling

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions

github-actions Bot commented Apr 8, 2026

Copy link
Copy Markdown
Contributor

Package publishing

Package Version Status Publish tag (post-merge)
package:property_testing 0.3.2 already published at pub.dev
package:routed 0.3.3 (error) pubspec version (0.3.3) and changelog (0.3.4) don't agree
package:routed_auth 0.1.0 ready to publish routed_auth-v0.1.0
package:routed_hotwire 0.1.2 already published at pub.dev
package:routed_io 0.1.0 ready to publish routed_io-v0.1.0
package:routed_testing 0.3.3 (error) pubspec version (0.3.3) and changelog (0.3.2) don't agree
package:server_native 0.1.3+1 (error) pubspec version (0.1.3+1) and changelog (null) don't agree
package:server_testing 0.3.3 already published at pub.dev
package:server_testing_shelf 0.3.2 already published at pub.dev

Documentation at https://github.com/dart-lang/ecosystem/wiki/Publishing-automation.

@kingwill101 kingwill101 changed the title [codex] fix server_native callback wakeups and request uri handling fix server_native callback wakeups and request uri handling Apr 8, 2026
@coderabbitai

coderabbitai Bot commented Apr 9, 2026

Copy link
Copy Markdown

Note

Unit test generation is a public access feature. Expect some limitations and changes as we gather feedback and continue to improve it.


Generating unit tests... This may take up to 20 minutes.

@coderabbitai

coderabbitai Bot commented Apr 9, 2026

Copy link
Copy Markdown

✅ Created PR with unit tests: #13

@kartikey321

Copy link
Copy Markdown

@kingwill101 Great work on the pr. Love your work . Just had a question what if you used actix or even directly io_uring instead of tokyo. Can server native be scaled even more , as the more costly part of it is the boundary between dart and rust, so i don't think that can be optimised.

I was working on a proof of concept dart with hot paths and http parsing in zig from io_uring in linux.
had gotten initial benchmarks as quite promising on HttpArena comparison between dart io and dart zig done on a 64 core vps

Test Connections dart:io RPS dart-zig RPS Relative
baseline 512 601,780 1,353,265 ~2.25x
baseline 4096 583,020 1,665,927 ~2.86x
pipelined 512 998,153 1,364,400 ~1.37x
pipelined 4096 997,674 1,477,162 ~1.48x
  • This is an initial PoC snapshot for directional signal.
  • Current PoC is not memory-optimized yet; memory tuning is pending.

I even raised a discussion in the dart sdk about a custom server specific runtime. It would be great if dart eventually progresses on the server side and makes it more performant

dart-lang/sdk#63352

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Performance bottleneck when scaling NativeHttpServer across multiple isolates (shared: true)

2 participants