Skip to content

Support opt-in psrpc bus compression - #1373

Merged
paulwe merged 1 commit into
mainfrom
psrpc-bus-compression
Sep 6, 2026
Merged

paulwe merged 1 commit into
mainfrom
psrpc-bus-compression

Conversation

@paulwe

@paulwe paulwe commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

psrpc v0.7.6 (psrpc#130) added opt-in gzip at the bus boundary, and protocol#1771 surfaced it as rpc.PSRPCConfig.Compression with a BusOptions() conversion. runService built the bus from the redis client alone, so there was no way to reach the setting.

psrpc:
  compression:
    quality: 6
    threshold: 1024
    max_decompressed_size: 0

Shape

Egress carried no rpc.PSRPCConfig at all, so this adds the whole block under a psrpc: key, matching livekit-server (livekit#4844) and ingress (ingress#488). Only compression is read here — egress uses the rpc.NewXClient(bus, opts...) shorthand and its own io_selection_timeout, so max_attempts, timeout, backoff and buffer_size are accepted for config parity but inert. The README says so.

It is seeded in the NewServiceConfig struct literal rather than InitDefaults, so the seed lands before the yaml unmarshal. Without that, a config setting only quality would leave Threshold at zero and compress every payload, however small. NewServiceConfig is the sole construction path — nothing else builds a ServiceConfig literal or calls InitDefaults — so the one seed covers the service and the integration runner.

PipelineConfig deliberately gets nothing. The per-session handler subprocess talks to the parent over unix-socket gRPC IPC and never builds a bus (see the comment at pkg/service/handler_proxy.go:27), so nothing else needed threading.

Compression is off by default

A peer on a psrpc older than v0.7.6 ignores the compression marker — psrpc deserializes with DiscardUnknown — and then decodes the gzipped bytes as the payload, so the message is dropped without an error. livekit-server, egress, ingress, SIP and agent workers share this bus, so enabling it is a two-stage operator action: roll v0.7.6+ everywhere, then raise quality at the publishers. The README says so at the setting.

max_decompressed_size only affects reading, so it can be set ahead of quality.

Testing

pkg/config/config_test.go gains coverage of the defaults and of an override that leaves Threshold at its default.

Verified the config actually reaches the compressor with a throwaway pub/sub over a local bus: the default config round-trips a 16 KB payload, quality: 6 round-trips it, and adding max_decompressed_size: 64 drops that same payload — which is what proves it was gzipped rather than skipped by the threshold. quality: 0 with the same cap delivers it.

Not verified against a live livekit-server; that needs a peer already on psrpc v0.7.6+. The integration suite was not run — it needs redis, chrome and a live server, and nothing here changes behavior at the default quality: 0.

Two pre-existing failures in go test ./... are unrelated and fail identically on main: m3u8.TestLivePlaylistWriter (wall-clock timestamp) and uploader.TestUploader (needs real AWS credentials).

Note on the protocol pin

v1.51.1-0.20260905133529-a4f4b5c0c23f is protocol main at the #1771 merge, the same pseudo-version livekit#4844 and ingress#488 use. It is 14 commits ahead of the previous pin and additive for egress — the only livekit/ package deletions in the range are in sip.go, and egress uses neither those nor the data-track packet code that moved in #1757.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

@paulwe
paulwe force-pushed the psrpc-bus-compression branch 2 times, most recently from 57503e5 to 3c57bc5 Compare September 6, 2026 16:20
psrpc v0.7.6 added opt-in gzip at the bus boundary, and protocol#1771
surfaced it as rpc.PSRPCConfig.Compression with a BusOptions()
conversion. The bus was built from the redis client alone, so there was
no way to reach the setting.

Compression is off by default. An older peer decodes the gzipped bytes
as the payload and drops the message without an error, so enabling it is
a two-stage operator action.
@paulwe
paulwe force-pushed the psrpc-bus-compression branch from 3c57bc5 to 7910d2e Compare September 6, 2026 16:21
@github-actions

github-actions Bot commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

AV-sync stats summary: view in run #34045169221

@paulwe
paulwe merged commit 6ddee1d into main Sep 6, 2026
17 of 18 checks passed
@paulwe
paulwe deleted the psrpc-bus-compression branch September 6, 2026 16:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants