Support opt-in psrpc bus compression - #1373
Merged
Merged
Conversation
paulwe
force-pushed
the
psrpc-bus-compression
branch
2 times, most recently
from
September 6, 2026 16:20
57503e5 to
3c57bc5
Compare
psrpc v0.7.6 added opt-in gzip at the bus boundary, and protocol#1771 surfaced it as rpc.PSRPCConfig.Compression with a BusOptions() conversion. The bus was built from the redis client alone, so there was no way to reach the setting. Compression is off by default. An older peer decodes the gzipped bytes as the payload and drops the message without an error, so enabling it is a two-stage operator action.
paulwe
force-pushed
the
psrpc-bus-compression
branch
from
September 6, 2026 16:21
3c57bc5 to
7910d2e
Compare
boks1971
approved these changes
Sep 6, 2026
Contributor
|
AV-sync stats summary: view in run #34045169221 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
psrpc v0.7.6 (psrpc#130) added opt-in gzip at the bus boundary, and protocol#1771 surfaced it as
rpc.PSRPCConfig.Compressionwith aBusOptions()conversion.runServicebuilt the bus from the redis client alone, so there was no way to reach the setting.Shape
Egress carried no
rpc.PSRPCConfigat all, so this adds the whole block under apsrpc:key, matching livekit-server (livekit#4844) and ingress (ingress#488). Onlycompressionis read here — egress uses therpc.NewXClient(bus, opts...)shorthand and its ownio_selection_timeout, somax_attempts,timeout,backoffandbuffer_sizeare accepted for config parity but inert. The README says so.It is seeded in the
NewServiceConfigstruct literal rather thanInitDefaults, so the seed lands before the yaml unmarshal. Without that, a config setting onlyqualitywould leaveThresholdat zero and compress every payload, however small.NewServiceConfigis the sole construction path — nothing else builds aServiceConfigliteral or callsInitDefaults— so the one seed covers the service and the integration runner.PipelineConfigdeliberately gets nothing. The per-session handler subprocess talks to the parent over unix-socket gRPC IPC and never builds a bus (see the comment atpkg/service/handler_proxy.go:27), so nothing else needed threading.Compression is off by default
A peer on a psrpc older than v0.7.6 ignores the compression marker — psrpc deserializes with
DiscardUnknown— and then decodes the gzipped bytes as the payload, so the message is dropped without an error. livekit-server, egress, ingress, SIP and agent workers share this bus, so enabling it is a two-stage operator action: roll v0.7.6+ everywhere, then raisequalityat the publishers. The README says so at the setting.max_decompressed_sizeonly affects reading, so it can be set ahead ofquality.Testing
pkg/config/config_test.gogains coverage of the defaults and of an override that leavesThresholdat its default.Verified the config actually reaches the compressor with a throwaway pub/sub over a local bus: the default config round-trips a 16 KB payload,
quality: 6round-trips it, and addingmax_decompressed_size: 64drops that same payload — which is what proves it was gzipped rather than skipped by the threshold.quality: 0with the same cap delivers it.Not verified against a live livekit-server; that needs a peer already on psrpc v0.7.6+. The integration suite was not run — it needs redis, chrome and a live server, and nothing here changes behavior at the default
quality: 0.Two pre-existing failures in
go test ./...are unrelated and fail identically onmain:m3u8.TestLivePlaylistWriter(wall-clock timestamp) anduploader.TestUploader(needs real AWS credentials).Note on the protocol pin
v1.51.1-0.20260905133529-a4f4b5c0c23fis protocol main at the#1771merge, the same pseudo-version livekit#4844 and ingress#488 use. It is 14 commits ahead of the previous pin and additive for egress — the onlylivekit/package deletions in the range are insip.go, and egress uses neither those nor the data-track packet code that moved in #1757.