Start improving high-throughput multi-camera recording path and diagnostics

[C-Achard]

Summary

Improves multi-camera recording behavior under high-throughput conditions by separating recording frame delivery from the heavier GUI/DLC processing path, adding clearer recorder backlog/queue diagnostics, and introducing an optional faster encoding mode for supported FFmpeg codecs.
This is a first quick remediation to the deeper architectural concern requiring the recording to be moved closer to camera SDK frame grabbing.

Main changes

1. Adds a dedicated recording_frame_ready(camera_id, frame, timestamp) signal for full-rate recording frames.
2. Recording frames are now handled by a lean recording-only path instead of _on_multi_frame_processing_ready().
3. Recording-frame emission is disabled by default and only enabled while recording is active.
4. Adds a "Use faster encoding" parameters checkbox in the Recording panel.
   For libx264/libx265, this applies:
   • Preset ultrafast
   • Tune zerolatency
5. Centralizes WriteGear/FFmpeg option generation in RecordingSettings.writegear_options().
6. Adds recorder better diagnostics
7. Updates multi-camera recorder summaries to aggregate queue/backlog/writer throughput across cameras.
8. Switches Basler grabbing to GrabStrategy_OneByOne so recording-oriented acquisition does not silently prefer only the latest image.
9. Reduces noisy Basler trigger debug logs behind an explicit DEBUG_TRIGGER_LOGS flag.
10. Adds profiling artifacts to .gitignore.
11. Updates tests for the new recording signal, fast encoding options, recorder stats formatting, and multi-camera recording aggregation.

What this improves

This makes it easier to diagnose when the recorder is falling behind before drops occur. Previously, dropped = 0 could hide the fact that recorder queues were filling. The new stats expose that state directly through queue fill and backlog.
The fast encoding option also provides a user-controlled way to trade compression efficiency/file size for higher recording throughput.

Not solved yet

• Recording frames still pass through the Qt/main-window path before reaching VideoRecorder.write().
• The actual encoder write is already threaded, but the producer/enqueue side can still add main-thread pressure at very high frame rates.
• This PR does not introduce a dedicated recording router thread or direct camera-worker-to-recorder routing.
• This does not guarantee full-rate recording for extreme cases, but makes bottlenecks more visible and improves slightly the x264 fast path at the cost of worst file size.

Testing

Added/updated unit tests for:

• Recording-frame signal enable/disable behavior
• Recording overlay routing
• RecordingSettings.writegear_options()
• Fast encoding option propagation
• Recorder queue/backlog stats formatting
• Multi-recorder stats aggregation

---

• Note before merge: disable debug logging

[Copilot]

Pull request overview

This PR improves multi-camera recording robustness and diagnostics under high-throughput load by adding a dedicated full-rate recording signal/path, enhancing recorder backlog/queue reporting, and introducing optional “fast encoding” FFmpeg parameters via centralized WriteGear option generation.

Changes:

• Added a dedicated recording_frame_ready(camera_id, frame, timestamp) signal and routed recording through a leaner handler in the main window.
• Expanded recorder stats to include buffer capacity, queue fill, and backlog; updated aggregated multi-camera summaries accordingly.
• Added RecordingSettings.writegear_options() (incl. optional “fast encoding”), updated Basler grab strategy, and extended tests/ignores/extras to support these features.

Reviewed changes

Copilot reviewed 15 out of 16 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
tests/utils/test_stats.py	Updates recorder stats formatting tests for backlog/buffer-capacity output.
tests/test_config.py	Adds unit tests for RecordingSettings.writegear_options() and fast-encoding behavior.
tests/services/test_multicam_controller.py	Adds test for recording-frame emission enable/disable behavior.
tests/gui/test_rec_manager.py	Updates multi-recorder summary aggregation tests and checks writer options propagation.
tests/gui/test_pose_overlay.py	Updates overlay-recording test to use the new recording-frame handler.
tests/conftest.py	Extends FakeVideoRecorder to accept new constructor args (buffer/writer options).
tests/cameras/backends/conftest.py	Updates Basler fake to include GrabStrategy_OneByOne.
pyproject.toml	Adds optional “profiling” dependency group.
dlclivegui/utils/stats.py	Adds recorder backlog/fill helpers and extends recorder stats formatting output.
dlclivegui/services/video_recorder.py	Adds writer_options, improves logging, and adjusts stats computation.
dlclivegui/services/multi_camera_controller.py	Adds recording_frame_ready signal and togglable emission during capture.
dlclivegui/gui/recording_manager.py	Centralizes writer option generation and expands stats aggregation/summaries.
dlclivegui/gui/main_window.py	Adds “fast encoding” UI toggle, connects new recording signal, and routes recording via lean handler.
dlclivegui/config.py	Adds fast_encoding to recording settings; changes timing-log defaults.
dlclivegui/cameras/backends/basler_backend.py	Switches Basler grab strategy and gates trigger debug logging behind a flag.
.gitignore	Ignores common profiling artifacts (scalene/profile outputs).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 18 out of 19 changed files in this pull request and generated 1 comment.

[deruyter92]

amazing start, great improvement. Good that you have the recording_frame_ready split and the Recorder stats seem very helpful for diagnosis.

I think the main concern worth discussing would be the unconditional OneByOne strategy (see comments below). For the rest no blocking issues.

[deruyter92]

shall we make this an env var that defaults to False? i.e. any scenario where it still would be useful?

[C-Achard]

Updated in d2f6125, I would keep it for debugging

[deruyter92]

This seems like a big unconditional change which we should give some consideration.

I understand that we want to prevent discarding intermediate frames (which is mostly useful for viewing, but not recording), but processing all frames in arrival order also has it's risks/downsides. Some remarks/questions:

• If the buffer is full (currently 10 -> ~100ms ?), does this mean that new incoming frames are lost?
• Does unconditionally adopting this new strategy mean that viewing (not recording) operates on a growing backlog of stale frames?
• Do we have proper diagnostics on the acquisition-side, not the recorder side? i.e. do we know the number of ready frames in the buffer, if frames are dropped, etc? or does the recorder never see this?

[deruyter92]

this seems slightly duplicated with writegear_options in config.py

[C-Achard]

It is, but it does leave VideoRecorder indendently usable, and I do like the overrides (renamed the param so it is clearer that it has precedence).
There is still overlap this way but it may be the simplest, best of both worlds approach.

Let me know what you think!

[deruyter92]

Good addition, nice win! Flagging here as a potential follow-up later: the multiple processes still compete for CPU, so hardware encoding might help a lot. But this would need their own option profile (e.g NVENC uses different presets).