Skip to content

fix(decklink): wait for playback to stop before destroying consumer (teardown-race use-after-free) - #1763

Open
C-alcaide wants to merge 1 commit into
CasparCG:masterfrom
C-alcaide:fix-decklink-consumer-teardown-race
Open

C-alcaide wants to merge 1 commit into
CasparCG:masterfrom
C-alcaide:fix-decklink-consumer-teardown-race

Conversation

@C-alcaide

Copy link
Copy Markdown
Contributor

Summary

The DeckLink consumer can crash the server during teardown with heap corruption (STATUS_HEAP_CORRUPTION / 0xC0000374) or an access violation, because it destroys itself while a frame-completion callback may still be running on the DeckLink driver's thread. This PR makes teardown wait for the driver to confirm playback has stopped, and unregisters the completion callback before destruction — following the sequence in Blackmagic's own SynchronizedPlayback SDK example.

The bug

decklink_consumer (and each decklink_secondary_port) registers itself as the IDeckLinkVideoOutputCallback via SetScheduledFrameCompletionCallback(this). The primary consumer's ScheduledFrameCompleted() runs on the driver's thread and dereferences many members (graph_, mode_, buffer_, secondary_port_contexts_, …).

On destruction the code did:

output_->StopScheduledPlayback(0, nullptr, 0);
output_->DisableVideoOutput();
secondary_port_contexts_.clear();   // members then destroyed

StopScheduledPlayback is asynchronous — the driver can keep delivering ScheduledFrameCompleted callbacks until it fires ScheduledPlaybackHasStopped. The destructor never waited for that, and never unregistered the callback. So if a completion callback is in flight while the consumer is being torn down, it touches members that are being destroyed → use-after-free.

This is timing-dependent (the callback must be in flight at the exact moment of teardown), so it manifests as a rare, log-silent crash — most often at end-of-broadcast teardown (REMOVE / consumer uninitialize) or on channel reconfiguration/restart.

Why the async behaviour is expected

The Blackmagic DeckLink SDK SynchronizedPlayback example maintains a dedicated std::mutex + std::condition_variable + bool m_stopped for exactly this: it calls StopScheduledPlayback, then waitForPlaybackStop() (m_stopCondition.wait), and only proceeds to DisableVideoOutput() + SetScheduledFrameCompletionCallback(nullptr) + release once ScheduledPlaybackHasStopped() has fired. The existence of that wait in the reference code is the confirmation that StopScheduledPlayback does not synchronously fence callbacks.

The fix

Mirror the SDK teardown sequence in both decklink_consumer and decklink_secondary_port:

  1. StopScheduledPlayback(...)
  2. Block until ScheduledPlaybackHasStopped() signals a condition variable (with a 2s timeout as a safety net against a missing callback).
  3. SetScheduledFrameCompletionCallback(nullptr) — unregister so the driver cannot call back into the object.
  4. DisableVideoOutput() and destroy members.

ScheduledPlaybackHasStopped() now sets the flag and notifies; start_playback() resets it before StartScheduledPlayback. For sync-group secondary ports (which are stopped via the primary), the primary already waits before it destroys them, so they only unregister their callback and disable output.

Scope / risk

  • One file, +58/-1; no behavioural change to the running (non-teardown) path.
  • Adds a bounded (≤2s) wait on consumer destruction; the timeout logs a warning and proceeds if the driver never confirms.
  • Builds cleanly on Windows (MSVC).

Validation

Verified by code/SDK analysis and a clean build. Because the underlying crash is rare and timing-dependent, this has not been reproduced on a bench; it is offered as a correctness fix that closes a real use-after-free window and matches the vendor's documented teardown sequence. Production soak (absence of 0xC0000374 at teardown) is the intended real-world validation.

…to fix teardown-race use-after-free

The DeckLink consumer (and each secondary port) registers itself as the
IDeckLinkVideoOutputCallback via SetScheduledFrameCompletionCallback(this).
ScheduledFrameCompleted() runs on the driver's thread and dereferences many
members (graph_, mode_, buffer_, secondary_port_contexts_, ...).

On destruction the code called StopScheduledPlayback() and then immediately
DisableVideoOutput() and destroyed its members, without waiting for the driver
to confirm that playback had stopped. StopScheduledPlayback is asynchronous:
the driver may still deliver ScheduledFrameCompleted callbacks until it fires
ScheduledPlaybackHasStopped. If such a callback is in flight while the consumer
is being torn down, it touches members that are being destroyed - a
use-after-free that can corrupt the heap (STATUS_HEAP_CORRUPTION / 0xC0000374)
or access-violate, typically at end-of-broadcast teardown or channel
reconfiguration.

Fix follows the Blackmagic DeckLink SDK SynchronizedPlayback example: after
StopScheduledPlayback, block until ScheduledPlaybackHasStopped signals a
condition variable, then unregister the completion callback
(SetScheduledFrameCompletionCallback(nullptr)) before DisableVideoOutput and
member destruction. A 2s timeout guards against a missing callback. Applied to
both decklink_consumer and decklink_secondary_port.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant