Uh oh!
There was an error while loading. Please reload this page.
Test the decoupled prebid shim on an rc/july-based tree (do not merge) - #968
Closed
aram356 wants to merge 245 commits into
Closed
Test the decoupled prebid shim on an rc/july-based tree (do not merge)#968aram356 wants to merge 245 commits into
aram356 wants to merge 245 commits into
Conversation
Fastly dynamic backend names embed the first-byte and between-bytes timeouts so a registration can never be silently reused with a different transport configuration. Deriving those timeouts from the remaining wall-clock auction budget minted a new backend name on nearly every request, defeating cross-request TCP/TLS connection reuse (Fastly pools connections per backend name) and accumulating registrations toward the per-service dynamic backend limit. Compute the effective transport timeout from the configured provider timeout verbatim when the budget allows, floor the budget-bound value to 250ms buckets otherwise, and pass sub-quantum remainders through exactly so publishers with sub-250ms configured budgets keep launching. The quantized value feeds both the backend name and the registered configuration, so they cannot diverge. Rounding down never extends a transport cap past the auction deadline, which the mediator and dispatched-collect paths rely on to bound the </body> hold. Also add a Fastly platform test pinning predict_name == ensure for the same spec, since the orchestrator maps responses back to providers by predicted backend name. Fixes#847
Publisher pages were fully buffered before the first byte reached the client: the platform client materialized the origin body (10 MiB cap), the rewrite pipeline ran over an in-memory cursor, and the EdgeZero finalize buffered the assembled response while awaiting auction collection. TTFB therefore tracked full origin transfer plus the auction instead of origin first byte. - Add supports_streaming_responses() to PlatformHttpClient (default false, Fastly true) and request with_stream_response() on the publisher origin fetch only where honored - Teach the pipeline to consume Body::Stream asynchronously: BodyChunkSource (cumulative raw-byte cap via publisher.max_buffered_body_bytes), push-style BodyStreamDecoder/BodyStreamEncoder in streaming_processor - Replace the Fastly buffered finalize with publisher_response_into_streaming_response: a lazy Body::Stream that commits headers at origin first byte, streams rewritten chunks, and holds only the </body> tail for auction collection; bids still inject before body close - Share one hold implementation (hold_step_decoded_chunk / hold_finish_segments) between the lazy body and the writer-driven loop so the paths cannot drift; collect_non_html_auction dedupes the collect-before-stream path - Finalize brotli decode with close() so truncated origin streams error instead of silently truncating; decode failures emit stream_decode_error telemetry - Guard bodiless (HEAD/204/304) responses and log wasted auction dispatch, matching the buffered finalizer Local A/B on a 183 KB gzip publisher page with a live 3-slot auction (release builds, 20 interleaved rounds): TTFB median 741 ms buffered vs 161 ms streamed (-78%); guest wall time and wasm heap unchanged.
Address the deep-review findings on the streaming cutover: - Cap cumulative decoded bytes in BodyStreamDecoder against publisher.max_buffered_body_bytes: the chunk source only bounds raw compressed bytes, so a decompression bomb could expand ~1000x past it and push unbounded decoded volume through the rewrite pipeline - Detect truncated deflate streams: write::ZlibDecoder::try_finish accepts truncated input silently, so the deflate arm now drives flate2::Decompress directly and requires Status::StreamEnd at finalization; trailing bytes after the end marker stay ignored. Add truncated-gzip and truncated-deflate regression tests - Make BodyChunkSource::next_chunk cancellation-safe by polling the body in place instead of moving it out across an await; a cancelled pull no longer turns into a silent EOF - Log dispatched auctions dropped uncollected (client disconnect mid-stream or never-polled body) via DispatchedAuctionGuard; the guard is created before the lazy stream so unpolled drops log too - Share the pull+decode step between the lazy publisher body and the write-sink drivers (hold_step_next_chunk / passthrough_step), removing the unreachable!() error plumbing and the triplicated processor selection; document body_close_hold_loop_stream as groundwork for the buffered adapters' streaming cutover - Pass identity-encoded chunks through zero-copy and finish encoders by consuming them instead of allocating a throwaway replacement - Add a Fastly dispatch test asserting the publisher fallback returns Body::Stream without a stale Content-Length, plus a comment on why the publisher fetch gates streaming on capability while the asset path does not Behavior note: gzip bodies with trailing garbage after the trailer now error mid-stream; the old read-path decoder ignored them.
The buffered finalizer abandons a dispatched auction with processor_init_error telemetry when HTML processor construction fails; the streaming finalizer dropped the in-flight SSP responses silently. Make publisher_response_into_streaming_response async and emit the same abandonment before returning the construction error.
Publisher-specific bundles need diagnostics based on the modules they actually contain. Otherwise, missing identity integrations can be hidden by the default preset. Refresh the comparison when auctions begin so late publisher configuration is visible without repeating warnings. Resolves: #886
The server-side auction stream path only emitted a summary counter (ssp/mediator/winning/time), so an operator seeing winning=0 could not tell whether prebid returned nothing, errored, or bid below the floor. Serialize the full provider_responses (and mediator_response) into the ts-debug HTML comment so the SSAT surfaces the same prebid server response detail available from the /auction endpoint. Bid creative and metadata are attacker/partner-influenced, so neutralize the '-->' and '--!>' comment terminators before embedding to keep the dump inside the comment and out of the live DOM.
When Prebid Server returns a non-2xx status, the parser returned a bare
AuctionResponse::error with empty metadata — indistinguishable in the
ts-debug dump from a transport, parse, or timeout failure, all of which
tag error_type. An operator seeing status=error with metadata={} had no
way to know the upstream HTTP code without log access.
Attach error_type=http_status, the status code, and a 512-byte body
snippet to the error response metadata so the auction dump shows exactly
why prebid errored (e.g. a 4xx from a PBS rejecting the request).Address PR review of the SSAT debug-dump change:
- The PBS non-2xx response body was attached to AuctionResponse.metadata,
which ProviderSummary clones verbatim into ext.orchestrator.provider_details
on the public /auction response — violating the documented invariant in
auction/orchestrator.rs. Drop the body from metadata, keep only the numeric
status, and log the snippet server-side at warn.
- Register ERROR_TYPE_HTTP_STATUS and match it in provider_status so PBS HTTP
errors get their own telemetry bucket instead of the transport_error
fallback.
- Bound the ts-debug dump: compact serialization capped at 256 KiB, and skip
the mediator_response line when no mediator ran.
- Correct the auction_html_comment and prepend_auction_debug_comment docs to
state the comment now embeds raw SSP creative markup (never enable in prod).
- Keep the targeted two-replace terminator neutralisation: a single
replace("--", ...) re-forms -->/--!> at odd dash-run junctions and is not
equivalent. Add a table-driven test over the comment-terminator vectors.
- Hoist test-local imports to module scope per CLAUDE.md.…debug' into feat/ssat-write-prebid-response-debug
- Drop render_adm param: always include adm when creative present
- Gate GAM-bypass on per-bid debug_bid instead of a global window.tsjs flag
(removes SPA-staleness edge case, TsjsApi change, and whole flag-emit task)
- Correct fallback scope: cache fallback only when adm absent; render failure
after adm is supplied is not detectable
- Qualify sandbox claim: TS guarantees script-context escaping; bridge frame
isolation depends on the Prebid Universal Creative
- Reconcile with existing ad_init.test.ts coverage (rename, no duplicates)
- Fix test assertion messages to expect("should ..."); list exact clippy gates- Task 3: assert observable DOM (GAM iframe src) instead of spying on the module-private injectAdmIntoSlot - Use existing make_bid helper + set .creative (make_test_bid_with_creative does not exist on this branch) - Hostile-adm regression test covers both U+2028 and U+2029 - Add 'cd docs && npm run format' to verification
build_bid_map now always inserts the winning creative as adm so the pbRender bridge can render it locally (no PBS Cache round trip); the verbose debug_bid blob and the GAM-bypass gate stay behind inject_adm_for_testing. Rename the param include_adm -> include_debug_bid and thread it through write_bids_to_state. Reconcile the by-default test to the new behavior, drop the now-redundant debug-only-adm test, and pin script-context escaping for a hostile adm (</script> + U+2028/U+2029).
The direct GAM-replace path (injectAdmIntoSlot) now fires only when the bid carries debug_bid, which is present only under inject_adm_for_testing. In production the always-present adm is rendered by the pbRender bridge and GAM stays in the loop. Add observable-DOM tests (GAM iframe src unchanged without debug_bid, rewritten with it), strengthen the bridge test to prove inline adm is preferred even when cache coords are present, and rename debug-adm terminology to inline adm.
Resolve five correctness and resource-safety findings from the PR #867 review of the end-to-end Fastly publisher streaming path. - Drive the deflate decoder to StreamEnd at finalization so a valid stream that exactly fills the internal output buffer is no longer rejected as truncated; the inflater is also drained after all input is consumed within a chunk. - Decode concatenated (multi-member) gzip bodies via MultiGzDecoder on both the streaming decoder and the buffered read pipeline so adapters agree. - Enforce the decoded-body cap during decompression through a bounded sink shared by the gzip and brotli codecs, so a compression bomb errors before its expanded bytes are buffered instead of after a full chunk expands; the deflate codec charges each produced block as it is emitted. - Drop the body of bodiless responses (HEAD, 204, 205, 304) in both the streaming and buffered finalizer Buffered arms, and add RESET_CONTENT to response_carries_body, so a buffered-unmodified stream body is never streamed to the client for a response that must be bodiless. - Keep the dispatched-auction guard armed across the collection await and disarm it only once collection reaches a terminal result, so a body dropped while collection is pending still logs the discarded SSP work. Add regression tests for the deflate output-buffer boundary, multi-member gzip, bodiless buffered stream bodies, and the auction guard sentinel.
Resolve the PR review by making transport-timeout canonicalization a platform capability and hardening auction backend-name correlation. - Move quantization behind PlatformBackend::canonicalize_transport_timeout_ms. Fastly floors budget-derived timeouts to a 250ms quantum with a bounded sub-quantum ladder [200,150,100,50]; other adapters use the exact remaining budget so bidder deadlines (Prebid tmax, APS timeout) are not shortened where no connection-pooling benefit exists. - Bound sub-quantum backend-name cardinality: exact 1-249ms values no longer pass through, capping the budget-derived names a single origin can mint toward the per-service dynamic backend limit. - Add a provider discriminator to PlatformBackendSpec, folded into every adapter's backend name, so two providers sharing one origin no longer collide on the response-correlation key. Reject a duplicate backend_to_provider insertion with an attributed launch failure instead of silently overwriting and misattributing a response. - Make the orchestrator call-site tests deterministic: record predicted and registered transport timeouts separately and assert exact equality via a controllable platform backend, and enumerate the sub-quantum ladder to assert a bounded name cardinality. - Correct the timeout-semantics comments that overstated absolute-deadline enforcement; the Fastly connect/first-byte/between-bytes timeouts bound connection, first-byte, and inactivity, not total response time. A true absolute deadline carried through the platform HTTP API remains follow-up work (#849).
Prebid non-2xx responses were reduced to bare provider errors, making intermittent failures difficult to diagnose. Surface safe HTTP metadata and bounded debug details while correlating server logs with the auction ID.
A configured bidder with no inline params and no matching override
expanded to `"bidder": {}`, which PBS rejects. After applying overrides,
drop fabricated empty bidders, preserve an explicitly supplied empty
object so genuine misconfiguration stays visible, and fall back to the
stored-request path when no eligible bidders remain.…debug' into merge/pr-890-893-899-910-912
rc/july merged the 942-page-bids-single-underscore branch before its second review round; the #971 squash on main includes that round. This merge is a union, not a fast-forward of either side. Resolutions: - publisher.rs takes main's round-two work — PAGE_BIDS_FALLBACK_MARKER, the warn/info split that separates a current bundle falling back from a pre-rename bundle, mark_deprecated_alias on both the 404 and success paths, and the cross-site gate moved above the not-configured 404 so a cross-site caller cannot probe whether creative opportunities are configured. Keeps rc/july's normalize_page_bids_path_and_query, trace_enabled, and request origin. Main's raw query binding is renamed requested_page to feed rc/july's normalize pipeline. - gpt/index.ts takes main's fetchPageBids/requestPageBids alias fallback and keeps rc/july's lastAppliedPath tracking pathname and search. - spa_hook.test.ts keeps rc/july's path assertion, which carries the query string; main's assertion predates that change. - telemetry.rs keeps rc/july's side: AuctionSource moved to types.rs, where the renamed path doc comment already lives. - orchestrator.rs keeps rc/july's side; main's only change there was two path comment renames rc/july already had. - CHANGELOG.md keeps both sets of entries.
Clean auto-merge, no conflicts. Brings over the two commits rc/july was missing from that branch. Enables the Cloudflare cache bypass: the outbound HTTP client maps the platform cache-bypass option to CacheMode::NoStore, but setting the cache field on RequestInit requires the cache_option_enabled compatibility flag, on by default only from compatibility date 2024-11-11. Both wrangler files pin 2024-09-23, so the Workers runtime would have thrown on every auction-eligible navigation, failing the origin fetch and abandoning the auction. The flag is now set explicitly in wrangler.toml and wrangler.ci.toml. Also strips CDN-Cache-Control and Cloudflare-CDN-Cache-Control from synthesized SSAT HTML alongside the existing surrogate headers, closing on Cloudflare the per-navigation document caching this change already closed on Fastly. Lands in handle_publisher_request, disjoint from the page-bids work merged from main immediately before it.
rc/july merged this branch on 2026-07-24, before its review round landed, so it was carrying the pre-review implementation. This brings over the review feedback commit and a doc lint fix. Most of the delta is creative_opportunities.rs, which gains the gam_unit_path template validation and its tests; core test count goes from 1824 to 1837. Also updates publisher.rs, the configuration guide, the example config, and the design spec. Only CHANGELOG.md conflicted, resolved as a union: rc/july's three Added entries kept alongside the branch's gam_unit_path entry. rc/july had no gam_unit_path entry of its own, so nothing is duplicated.
Scope the diagnostics navigation assertion to the fixture navigation landmark so Playwright does not match the separate site-wide Home link in CI.
This reverts commit 653ae8a.
This reverts commit 423e6c5.
This reverts commit fd1cbd8.
This reverts commit 050ad9c.
# Conflicts: # crates/trusted-server-js/lib/src/core/types.ts
Publishers may call googletag.display with a slot or element before the initial request gate is released. Treating that argument as a string can abort the request, while first-prefix slot resolution can target a hidden responsive sibling. Preserve the GPT display contract and resolve stable prefixes only when one active element can be identified.
| for (var i = 0; i < idElements.length; i++) { | ||
| var candidate = idElements[i]; | ||
| if ( | ||
| divId && |
# Conflicts: # crates/trusted-server-js/lib/src/integrations/prebid/index.ts # crates/trusted-server-js/lib/test/integrations/prebid/index.test.ts
# Conflicts: # crates/trusted-server-js/lib/src/integrations/prebid/index.ts # crates/trusted-server-js/lib/test/integrations/gpt/ad_init.test.ts # crates/trusted-server-js/lib/test/integrations/prebid/index.test.ts
# Conflicts: # CHANGELOG.md # crates/trusted-server-cli/tests/config_env_overlay.rs # crates/trusted-server-core/src/auction/README.md # crates/trusted-server-core/src/auction/formats.rs # crates/trusted-server-core/src/auction/orchestrator.rs # crates/trusted-server-core/src/auction_config_types.rs # crates/trusted-server-core/src/config_payload.rs # crates/trusted-server-core/src/creative.rs # crates/trusted-server-core/src/publisher.rs # crates/trusted-server-core/src/settings.rs # docs/guide/auction-orchestration.md # docs/guide/configuration.md # docs/guide/creative-processing.md # trusted-server.example.toml
# Conflicts: # crates/trusted-server-core/src/integrations/gpt_bootstrap.js # crates/trusted-server-js/lib/src/integrations/gpt/index.ts # crates/trusted-server-js/lib/test/integrations/gpt/ad_init.test.ts
Reconciles the squash-merged #945 (deferred initial adInit) with the RC's own evolution of the same files: - build_bids_script, the bundle scheduler, and the gpt_bootstrap.js fallback take main's final form: the SSR bids payload is handed to scheduleInitialAdInit, which applies it and runs adInit only on navigation generation 0; adInit rechecks its generation inside the queued googletag.cmd callback; the head bootstrap keeps the bundle-failure fallback scheduler. - The RC's pathname-plus-query route identity is kept: docblocks and the query-change tests follow the RC semantics (a query change requests fresh page bids and cancels the pending initial pass), and the RC's slot-handoff/initial-request-gate machinery is preserved alongside the new scheduler. - gpt_bootstrap.test.ts is adapted to the RC bootstrap: assertions target the pre-patch spies (the handoff patcher wraps defineSlot/display/ refresh) and expect the TS fallback slot on the publisher's actual inner div per the RC handoff design.
Both branches fixed oversized HEAD response metadata independently with mirrored parameter designs. Adopt main's implementation wholesale — the request_is_head parameter plus response_carries_body(), which also covers 1xx/204/205/304 bodiless statuses — and keep the RC's cache-bypass feature intact: send() retains its bypass_cache read alongside the new request_is_head, and the RC's apply_fastly_cache_bypass tests are restored next to main's bodiless-metadata tests.
# Conflicts: # crates/trusted-server-js/lib/src/integrations/prebid/index.ts # crates/trusted-server-js/lib/test/integrations/prebid/index.test.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Do not merge — testing branch
Draft for validating the decoupled prebid shim (#967) on the current rc/july before it lands anywhere. Not intended to merge.
What this branch is (rebuilt 2026-07-30)
rc/july-aram= currentrc/julytip (0325868cb) + a merge of the three #967 commits:prebid.jstype import before relative importsanycasts with typed helpersUnlike the previous iteration of this branch, nothing is excluded: the base includes the duplicate-GPT-slots fix, the GAM-refresh-exclusion (
excluded_gam_ad_unit_path_suffixes), the #974 publisher ad-unit machinery, the #912 squash-merge (bf61fa189), the responsive-slot GPT display gating fix, the latest main merge, and rc/july's revert of the #961 tracing work.Merge resolution notes
window.pbjsglobal with a fail-loud guard when the external bundle is missing, publicmarkWinningBidAsUsedinstead ofadRenderinginternals with registration-gated field removal, and manifest-basedclient_side_biddersvalidation.eslint-disablesuppressions.Verification
cargo fmt, clippy-fastly-D warnings, core 1826 + fastly 115 + axum/cloudflare/spin + parity 13Deploy coupling for test environments
This tree serves the shim as deferred tsjs and expects the external bundle to be pure Prebid.js. Deploy together with the regenerated bundle and matching
external_bundle_sha256/external_bundle_sri; pointing it at the old combined bundle installs the adapter twice.The previous iteration of this branch (older base + per-PR head sync) is preserved at
aram-sync-backup(536d848f5).