quic: For streams allow ReadableStream as body

[martenrichter]

After the change the body parameter for all
created QuicStreams can take a ReadableStream
of Uint8Array, Arraybuffers etc..

We introduce a DataQueueFeeder class that may be
also used for other related mechanisms.

A locally created unidirectional Stream can not
have a reader. Therefore, the interface is changed
to handle this case. (undercovered when writing
the tests).

Furthermore, a ResumeStream must be added after
AddStream in QuicSession, as the ResumeStream
beforehand triggered with set_outbound is a no-op,
as Stream was not added to Session beforehand.

Fixes: #60234

[martenrichter]

@jasnell This try to add support for streams for outgoing data. I hope the PR is helpful and does not waste too much time. I have contributed long time ago 1-2 patches, but I am not that fluent with the node.js codebase.

[codecov]

Codecov Report

❌ Patch coverage is 73.04786% with 107 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.79%. Comparing base (4f24aff) to head (15d2cc1).
⚠️ Report is 1700 commits behind head on main.

Files with missing lines	Patch %	Lines
src/dataqueue/queue.cc	68.01%	62 Missing and 17 partials ⚠️
src/quic/streams.cc	51.51%	10 Missing and 6 partials ⚠️
src/node_blob.cc	90.90%	2 Missing and 2 partials ⚠️
src/quic/session.cc	66.66%	2 Missing and 2 partials ⚠️
src/dataqueue/queue.h	25.00%	3 Missing ⚠️
src/quic/application.cc	80.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #60237 +/- ##
==========================================
+ Coverage 88.53% 88.79% +0.26% 
==========================================
Files 703 704 +1 Lines 208546 208884 +338 Branches 40217 40339 +122 ==========================================
+ Hits 184634 185479 +845 + Misses 15926 15296 -630 - Partials 7986 8109 +123 

Files with missing lines	Coverage Δ
lib/internal/blob.js	99.81% <100.00%> (+<0.01%)	⬆️
lib/internal/quic/quic.js	100.00% <100.00%> (ø)
src/node_binding.cc	82.74% <ø> (ø)
src/quic/quic.cc	100.00% <ø> (ø)
src/quic/streams.h	0.00% <ø> (ø)
src/quic/application.cc	45.70% <80.00%> (+25.51%)	⬆️
src/dataqueue/queue.h	25.00% <25.00%> (ø)
src/node_blob.cc	76.70% <90.90%> (+0.95%)	⬆️
src/quic/session.cc	42.95% <66.66%> (+13.88%)	⬆️
src/quic/streams.cc	42.58% <51.51%> (+39.38%)	⬆️
... and 1 more

... and 58 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[martenrichter]

There is a crash on some platforms. TODO for next weekend.

[martenrichter]

Locally, I do not have any crashes. Please note, that some of the commits fix errors present in the actual implementation.
I did not make separate PR as these were uncovered by the test included in the commit.
Most of them are annoying race conditions or problems with the lifetime of the objects (use after free!), though I do not see security implications as at the current stage, I do not assume a production usage of quic.

[martenrichter]

It seems, that Linux arm is crashing (or is flaky). So probably, I have to go to a Linux container, next weekend) or something else to figure out, what is happening there. (But it may be a problem not introduced by this PR).
(I hope it is a general flakyness on Linux and not ARM specific, or specific on the timing of the runner).

[jasnell]

Btw I should have time to look at this PR this upcoming weekend

[martenrichter]

Btw I should have time to look at this PR this upcoming weekend

This would be great. I am currently trying to reproduce the crash in a non arm container, no luck so far. I will try to use a github code space for arm, next. (Edit: no arm codespaces, but may be the x86 gives a different timing).

[martenrichter]

This time only unrelated MacOS failures. And I could not reproduce the previous arm faliures locally, may be the rebase
has fixed it.

[martenrichter]

Now, Code coverage is increased, and no failing tests.

[martenrichter]

The newly added test is actually flaky. Sending out data stalls depending on timing. I have to investigate, why SendPendingData is not called any more.

[martenrichter]

The new test is failing as it uncovered a yet-to-be-determined race condition.
What I see is that after submitting 60000, 12, and 50000 bytes, the last submission stalls. On the receive side, 58881 bytes are received. On the send side, it is unclear whether it is flow control or if the sending is just not rescheduled in the Application object. Ideas welcome for the cause, I will continue next weekend. (Review can still start, bug fixing is unrelated.).

Note that I also found another issue (use after free) during debugging related to using next bound to Session and Stream object that are not valid anymore; this should be fixed. It could be a general pitfall when using Bob.

[martenrichter]

I think I finally found the race condition, causing the test to fail.
Hopefully the automated test will go through.
Now I have to decide, whether I will also add tests of bidirectional streams in this PR or another one.

[jasnell]

This using should not be necessary

[martenrichter]

I remove it at my local copy and see what happens in the build process.

[martenrichter]

@jasnell Any chance for a new round of review?

[achingbrain]

I'm looking forward to streaming input making it into the QUIC API here, thanks for taking the time to open this.

From an API perspective, would it make more sense to instead add a writable property to quic.QuicStream of type WritableStream?

This would mirror the WebTransportBidirectionalStreamreadable/writable properties and so be more familiar to developers used to working with duplex streams.

[martenrichter]

Well, I did not touch this. I followed the interface that was already there. And I also asked myself the same question. But I think under the hood in the node.js environment it may be a valid choice, as it was made with normal http3 in mind and not webtransport. Though converting it is easy, see the test, I wanted to use the familiar pattern. My main concern is that things get going at some point. As I was almost forgot what I have done here.

[achingbrain]

For sure, adding the extra body type is less disruptive so I can see why this PR takes that route, but I for one was definitely expecting to see QuicStreams being duplex streams.

As implemented they seem closer to window.fetch which is less useful for my use-case (communicating with rust/go libp2p nodes over the QUIC transport (as a better TCP) which uses the built-in bidirectional stream capability as lightweight application level multiplexed streams).

[martenrichter]

Yes, it is more like fetch. I tried to just fill the gaps that @jasnell left. (As far as I understood them). So that you can also paste later node.js streams and async iterators.
But I do not see a problem with your use cases, I already have an idea how I can use it as a Node.js native backend for my package, where you contributed with the tests. (Note this here are quicstreams, for Webtransport, some tiny bits have to be added to the streams) .However, currently I am waiting for some signals on how the quic support will progress.