Access logging and session replay: the two features left behind at the flip - #137

Open
corrin wants to merge 6 commits into
mainfrom
feat/restore-app-usage-logging
Open

Access logging and session replay: the two features left behind at the flip#137
corrin wants to merge 6 commits into
mainfrom
feat/restore-app-usage-logging

Conversation

@corrin

Copy link
Copy Markdown
Owner

Ports the last two features that were left behind at the flip: the per-request
access log, and session replay end to end. Both were recorded as deferred in
docs/rewrite-status.md, and access logging was the first item on the
post-cutover queue.

Access logging

One line per authenticated request on its own access logger, carrying the
X-Session-Replay-Id that joins a request to its recording.

It goes to the console, not v1's rotating access.log. config/settings.py
already states that journald is the sink, and journald gives the rotation and
retention v1 got from ConcurrentRotatingFileHandler — a file handler would
only put a second copy on disk for an operator to find and prune.

Two v1 constructs are deliberately not ported, both verified broken rather than
assumed:

  • v1 read request.user before calling the view. Under ninja auth — which
    sets request.user during operation dispatch, after every middleware — that
    logs nothing for any /api/** request, i.e. for the whole application, while
    a v1-shaped test still passes. v2 reads the principal after get_response.
  • DisallowedHostMiddleware never did anything.process_exception fires
    only for exceptions raised by the view, but DisallowedHost comes out of
    CommonMiddleware.process_request above it. Django was always returning that
    400; only the traceback was ever the complaint. apps/core/logging_filters.py
    keeps the record of the probe and strips its traceback. The test fails without
    the filter — I checked.

v1's JWT re-authentication block went with it (v2 is cookie-authenticated), and
with it a bare except Exception: pass.

Session replay

Chunk payloads go to a private disk root (SESSION_REPLAY_STORAGE_ROOT, which
instance.sh has been provisioning at 700 all along). The rows index the store
rather than being it, so browsing sessions stay out of every pg_dump.

The store is shared, not a second copy. Phone-call recordings already kept a
metadata row plus a payload on a private root, and v1 wrote that logic twice.
apps/core/file_store.py now owns the path-escape guard, the atomic write and
the refuse-to-overwrite, and phone_call_service uses it too (ADR 0039).

Access follows what the business actually had. A staff member may write to
their own recording and read none of them. v1 gated the API at office staff but
hung the page behind a superuser route; a replay is an unredacted video of
somebody's screen, so reads are superuser-only here.

Upload rules are about not losing a session, and are the part v1 iterated on:
a transient failure puts events back at the front of the buffer (behind later
events they would replay out of order), a 409 means the chunk already landed so
the sequence advances and recording continues, and only 401/403/404 discard the
recording.

Three things v1 lacked or got wrong:

  • CompanyDefaults.session_replay_enabled — an off-switch that is not a deploy.
    v1 had none outside a DEV-only E2E flag.
  • The purge is scheduled (01:30 NZT) and now deletes payloads, not just rows.
    Retention is this feature's only privacy control, so it ships with capture.
  • The scrubber truncates recordings. v1 blanked storage_path and sha256,
    which left the unscrubbed payloads on disk and left the purge unable to
    ever find them again.

Also fixed on the way past: src/api/client.ts carried a STUB comment where the
X-Session-Replay-Id header belonged, which is why every AppError persisted
since the port had a replay column it could never fill. And v1's admin page
"destroyed" its player with innerHTML = '', leaking the previous instance's
timers and listeners on every reselect.

Capture stays off under Playwright-over-ngrok — the E2E run is already
bottlenecked on that tunnel. The new spec clears that opt-out for itself.

Operational assets unblocked

Both blocked-by: rows in v1-disposition.md that named session replay are now
ported: the purge beat entry, and pull_prod_files.sh, whose only blocker was
the storage decision. It takes host and instance-user as required arguments
rather than defaulting to MSM production, and validates instance-user as a plain
unix account name — it is interpolated into the remote --rsync-path, the same
escalation pull_prod_backup.sh already guards.

Docs commit authored by another session

80b7d48 is the work of a concurrent session that exited before it could commit;
I reviewed and committed it rather than leave it in a shared worktree. It makes
rewrite-status.md forward-looking and turns cutover-checklist.md into a
record now the flip has run. I verified every redirected comment resolves to
content that really moved before committing it.

Verification

Commit and push tiers green (ruff, mypy strict, import-linter, find-duplicates,
deptry, exported schema, status table, code-quality, delta goldens, frontend
lint/format/boundary/type-check/audit, generated-client-current, server suites,
makemigrations). Python suite green; 613 frontend unit tests green, 6 of them new
and covering the upload rules above.

Not yet run: frontend/tests/e2e/admin/session-replay.spec.ts. It drives the
cross-layer path no unit test on either side can reach — browser capture, chunk
upload, gzip on disk, events endpoint, player mount — plus a second test toggling
session_replay_enabled off. Per CLAUDE.md this slice is not done until that
passes, so please treat this PR as unverified end-to-end until it does.

🤖 Generated with Claude Code

https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7

corrinand others added 5 commits September 2, 2026 17:27
One line per authenticated request on its own `access` logger, carrying the
X-Session-Replay-Id that joins a request to its recording. It goes to the
console, not v1's rotating access.log: journald already rotates, retains and
greps that stream.
Two v1 constructs are deliberately not ported. AccessLoggingMiddleware now
reads the principal AFTER get_response — v1 checked it on the way in and
returned early when anonymous, which under ninja auth (which sets request.user
during operation dispatch, after middleware) would log nothing for any /api/**
request while a v1-shaped test still passed. And DisallowedHostMiddleware never
did anything: process_exception fires only for view exceptions, while
DisallowedHost comes out of CommonMiddleware.process_request above it. Django
was always returning that 400; only the traceback was the complaint, so a
logging filter keeps the record of the probe and drops its traceback.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
Chunk payloads go to a private disk root (SESSION_REPLAY_STORAGE_ROOT, which
instance.sh has been provisioning at 700 all along); the rows index the store
rather than being it, so browsing sessions stay out of every pg_dump.
The store itself is shared, not a second copy. Phone-call recordings already
kept a metadata row plus a payload on a private root, and v1 wrote that logic
twice; apps/core/file_store.py now owns the path-escape guard, the atomic
write and the refuse-to-overwrite, and phone_call_service uses it too.
Access follows what the business actually had: a staff member may write to
their own recording and read none of them. v1 gated the API at office staff
but hung the page behind a superuser route, so reads are superuser-only here —
a replay is an unredacted video of somebody's screen.
Three things v1 lacked or got wrong:
- CompanyDefaults.session_replay_enabled, an off-switch that is not a deploy.
- The purge is scheduled and now deletes payloads, not just rows. Retention is
this feature's only privacy control, so it ships with capture.
- The scrubber truncates recordings. v1 blanked storage_path and sha256, which
left the unscrubbed payloads on disk AND left the purge unable to find them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
rrweb records every authenticated page and flushes batches every ten seconds.
The upload rules are the parts v1 iterated on and they are about not losing a
session: a transient failure puts the events back at the FRONT of the buffer
(behind later events they would replay out of order), a 409 means the chunk
already landed so the sequence advances and recording continues, and only
401/403/404 discard the recording.
X-Session-Replay-Id now leaves the client for real — src/api/client.ts had
carried a STUB comment where the header belonged, which is why every AppError
persisted since the port had a replay column it could never fill.
Capture stays off under Playwright-over-ngrok: the E2E run is already
bottlenecked on that tunnel and recording every spec would add chunk uploads
to it. The new spec clears that opt-out for itself, because the cross-layer
path — browser capture to gzip on disk to the events endpoint to the player —
is the one no unit test on either side can reach.
Two v1 defects not carried over: the admin page destroyed its player by
clearing innerHTML, leaking the previous instance's timers and listeners on
every reselect; and the wire's opaque JSON is now checked against rrweb's
event shape rather than trusted, so an unplayable recording fails visibly
instead of showing a blank frame.
The one docs line is the spec-count row the status-table gate regenerates
because this commit adds a spec file; the rest of that file is another
workstream's and is deliberately untouched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
Both rows named session replay as their blocker, and both blockers are now
answered, so the disposition rule applies: the code arrives with the feature.
pull_prod_files.sh is ported. Its blocker was the storage decision — payloads
live on disk under SESSION_REPLAY_STORAGE_ROOT, so all three destinations
exist and the script has somewhere to put them. Two changes from v1: host and
instance-user are required arguments rather than defaulting to MSM production,
and instance-user is validated as a plain unix account name, because it is
interpolated into the remote --rsync-path and is the same escalation
pull_prod_backup.sh already guards. Destinations are all resolved before the
first byte moves, since a missing root found halfway through leaves a partial
pull that looks complete on the directories it did reach.
purge_old_session_replays_daily is ported: scheduled 01:30 NZT, and deleting
chunk payloads and the recording directory rather than only rows.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
Authored by a concurrent session (update-rewrite-status-postrelease), which
exited before it could commit. I reviewed the diffs and committed them rather
than leave the work uncommitted in a shared worktree; I did not write them.
What it does: rewrite-status.md becomes forward-looking, holding tasks only,
with what shipped moved to rewrite-history.md; cutover-checklist.md becomes
cutover-record.md in all but filename, because the flip ran on 2026-08-29 and
a checklist nobody ticked on the night would lie in a new way; CLAUDE.md now
says work is tracked in both Jira and rewrite-status.md, and points at
release-process.md rather than the cutover file.
What I checked, since I did not author it: every redirected comment resolves
to content that really moved — the DRAFT/WIP divergence, the unweighted
billable_percentage and total_revenue quirks, the public-holiday working-day
split and the quote-event double count are all in rewrite-history.md now; the
two renamed doc targets (development_session.md, release-process.md) exist;
and the test docstring's "a client error IS an AppError" item is really in
rewrite-status.md. The five apps/accounting and two test/spec edits are
comment-only.
One line is mine: I deleted the "Re-dispose the two blocked-by: rows that
named session replays" task, which 7ff50da completed — that file only shrinks,
and finished work is deleted the moment it is finished.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
@coderabbitai

coderabbitaiBot commented Sep 2, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Team

Run ID: aa049e06-ce1d-4253-a6cc-fbf38800dff3


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Links rather than restates: the ticket is the authority, this file just says
enough for a session to pick it up cold and to know not to start it yet — it
overturns the layout CLAUDE.md documents, so its owner-ratified ADR gates
everything after it.
The scoping caution is worth carrying here because it changes how big the work
looks. The draft justified the epic as breaking seven ORM cycles; traversing
every concrete cross-app relation in the app registry finds exactly one
(accounts <-> job, via Staff.default_labour_subtype). The other six are
one-way, and purchasing -> job, accounting -> job and quoting -> company
already point where the target architecture wants them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@corrin
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Access logging and session replay: the two features left behind at the flip - #137

Open
corrin wants to merge 6 commits into
mainfrom
feat/restore-app-usage-logging
Open

Access logging and session replay: the two features left behind at the flip#137
corrin wants to merge 6 commits into
mainfrom
feat/restore-app-usage-logging

Conversation

@corrin

Copy link
Copy Markdown
Owner

Ports the last two features that were left behind at the flip: the per-request
access log, and session replay end to end. Both were recorded as deferred in
docs/rewrite-status.md, and access logging was the first item on the
post-cutover queue.

Access logging

One line per authenticated request on its own access logger, carrying the
X-Session-Replay-Id that joins a request to its recording.

It goes to the console, not v1's rotating access.log. config/settings.py
already states that journald is the sink, and journald gives the rotation and
retention v1 got from ConcurrentRotatingFileHandler — a file handler would
only put a second copy on disk for an operator to find and prune.

Two v1 constructs are deliberately not ported, both verified broken rather than
assumed:

  • v1 read request.user before calling the view. Under ninja auth — which
    sets request.user during operation dispatch, after every middleware — that
    logs nothing for any /api/** request, i.e. for the whole application, while
    a v1-shaped test still passes. v2 reads the principal after get_response.
  • DisallowedHostMiddleware never did anything.process_exception fires
    only for exceptions raised by the view, but DisallowedHost comes out of
    CommonMiddleware.process_request above it. Django was always returning that
    400; only the traceback was ever the complaint. apps/core/logging_filters.py
    keeps the record of the probe and strips its traceback. The test fails without
    the filter — I checked.

v1's JWT re-authentication block went with it (v2 is cookie-authenticated), and
with it a bare except Exception: pass.

Session replay

Chunk payloads go to a private disk root (SESSION_REPLAY_STORAGE_ROOT, which
instance.sh has been provisioning at 700 all along). The rows index the store
rather than being it, so browsing sessions stay out of every pg_dump.

The store is shared, not a second copy. Phone-call recordings already kept a
metadata row plus a payload on a private root, and v1 wrote that logic twice.
apps/core/file_store.py now owns the path-escape guard, the atomic write and
the refuse-to-overwrite, and phone_call_service uses it too (ADR 0039).

Access follows what the business actually had. A staff member may write to
their own recording and read none of them. v1 gated the API at office staff but
hung the page behind a superuser route; a replay is an unredacted video of
somebody's screen, so reads are superuser-only here.

Upload rules are about not losing a session, and are the part v1 iterated on:
a transient failure puts events back at the front of the buffer (behind later
events they would replay out of order), a 409 means the chunk already landed so
the sequence advances and recording continues, and only 401/403/404 discard the
recording.

Three things v1 lacked or got wrong:

  • CompanyDefaults.session_replay_enabled — an off-switch that is not a deploy.
    v1 had none outside a DEV-only E2E flag.
  • The purge is scheduled (01:30 NZT) and now deletes payloads, not just rows.
    Retention is this feature's only privacy control, so it ships with capture.
  • The scrubber truncates recordings. v1 blanked storage_path and sha256,
    which left the unscrubbed payloads on disk and left the purge unable to
    ever find them again.

Also fixed on the way past: src/api/client.ts carried a STUB comment where the
X-Session-Replay-Id header belonged, which is why every AppError persisted
since the port had a replay column it could never fill. And v1's admin page
"destroyed" its player with innerHTML = '', leaking the previous instance's
timers and listeners on every reselect.

Capture stays off under Playwright-over-ngrok — the E2E run is already
bottlenecked on that tunnel. The new spec clears that opt-out for itself.

Operational assets unblocked

Both blocked-by: rows in v1-disposition.md that named session replay are now
ported: the purge beat entry, and pull_prod_files.sh, whose only blocker was
the storage decision. It takes host and instance-user as required arguments
rather than defaulting to MSM production, and validates instance-user as a plain
unix account name — it is interpolated into the remote --rsync-path, the same
escalation pull_prod_backup.sh already guards.

Docs commit authored by another session

80b7d48 is the work of a concurrent session that exited before it could commit;
I reviewed and committed it rather than leave it in a shared worktree. It makes
rewrite-status.md forward-looking and turns cutover-checklist.md into a
record now the flip has run. I verified every redirected comment resolves to
content that really moved before committing it.

Verification

Commit and push tiers green (ruff, mypy strict, import-linter, find-duplicates,
deptry, exported schema, status table, code-quality, delta goldens, frontend
lint/format/boundary/type-check/audit, generated-client-current, server suites,
makemigrations). Python suite green; 613 frontend unit tests green, 6 of them new
and covering the upload rules above.

Not yet run: frontend/tests/e2e/admin/session-replay.spec.ts. It drives the
cross-layer path no unit test on either side can reach — browser capture, chunk
upload, gzip on disk, events endpoint, player mount — plus a second test toggling
session_replay_enabled off. Per CLAUDE.md this slice is not done until that
passes, so please treat this PR as unverified end-to-end until it does.

🤖 Generated with Claude Code

https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7

corrinand others added 5 commits September 2, 2026 17:27
One line per authenticated request on its own `access` logger, carrying the
X-Session-Replay-Id that joins a request to its recording. It goes to the
console, not v1's rotating access.log: journald already rotates, retains and
greps that stream.
Two v1 constructs are deliberately not ported. AccessLoggingMiddleware now
reads the principal AFTER get_response — v1 checked it on the way in and
returned early when anonymous, which under ninja auth (which sets request.user
during operation dispatch, after middleware) would log nothing for any /api/**
request while a v1-shaped test still passed. And DisallowedHostMiddleware never
did anything: process_exception fires only for view exceptions, while
DisallowedHost comes out of CommonMiddleware.process_request above it. Django
was always returning that 400; only the traceback was the complaint, so a
logging filter keeps the record of the probe and drops its traceback.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
Chunk payloads go to a private disk root (SESSION_REPLAY_STORAGE_ROOT, which
instance.sh has been provisioning at 700 all along); the rows index the store
rather than being it, so browsing sessions stay out of every pg_dump.
The store itself is shared, not a second copy. Phone-call recordings already
kept a metadata row plus a payload on a private root, and v1 wrote that logic
twice; apps/core/file_store.py now owns the path-escape guard, the atomic
write and the refuse-to-overwrite, and phone_call_service uses it too.
Access follows what the business actually had: a staff member may write to
their own recording and read none of them. v1 gated the API at office staff
but hung the page behind a superuser route, so reads are superuser-only here —
a replay is an unredacted video of somebody's screen.
Three things v1 lacked or got wrong:
- CompanyDefaults.session_replay_enabled, an off-switch that is not a deploy.
- The purge is scheduled and now deletes payloads, not just rows. Retention is
this feature's only privacy control, so it ships with capture.
- The scrubber truncates recordings. v1 blanked storage_path and sha256, which
left the unscrubbed payloads on disk AND left the purge unable to find them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
rrweb records every authenticated page and flushes batches every ten seconds.
The upload rules are the parts v1 iterated on and they are about not losing a
session: a transient failure puts the events back at the FRONT of the buffer
(behind later events they would replay out of order), a 409 means the chunk
already landed so the sequence advances and recording continues, and only
401/403/404 discard the recording.
X-Session-Replay-Id now leaves the client for real — src/api/client.ts had
carried a STUB comment where the header belonged, which is why every AppError
persisted since the port had a replay column it could never fill.
Capture stays off under Playwright-over-ngrok: the E2E run is already
bottlenecked on that tunnel and recording every spec would add chunk uploads
to it. The new spec clears that opt-out for itself, because the cross-layer
path — browser capture to gzip on disk to the events endpoint to the player —
is the one no unit test on either side can reach.
Two v1 defects not carried over: the admin page destroyed its player by
clearing innerHTML, leaking the previous instance's timers and listeners on
every reselect; and the wire's opaque JSON is now checked against rrweb's
event shape rather than trusted, so an unplayable recording fails visibly
instead of showing a blank frame.
The one docs line is the spec-count row the status-table gate regenerates
because this commit adds a spec file; the rest of that file is another
workstream's and is deliberately untouched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
Both rows named session replay as their blocker, and both blockers are now
answered, so the disposition rule applies: the code arrives with the feature.
pull_prod_files.sh is ported. Its blocker was the storage decision — payloads
live on disk under SESSION_REPLAY_STORAGE_ROOT, so all three destinations
exist and the script has somewhere to put them. Two changes from v1: host and
instance-user are required arguments rather than defaulting to MSM production,
and instance-user is validated as a plain unix account name, because it is
interpolated into the remote --rsync-path and is the same escalation
pull_prod_backup.sh already guards. Destinations are all resolved before the
first byte moves, since a missing root found halfway through leaves a partial
pull that looks complete on the directories it did reach.
purge_old_session_replays_daily is ported: scheduled 01:30 NZT, and deleting
chunk payloads and the recording directory rather than only rows.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
Authored by a concurrent session (update-rewrite-status-postrelease), which
exited before it could commit. I reviewed the diffs and committed them rather
than leave the work uncommitted in a shared worktree; I did not write them.
What it does: rewrite-status.md becomes forward-looking, holding tasks only,
with what shipped moved to rewrite-history.md; cutover-checklist.md becomes
cutover-record.md in all but filename, because the flip ran on 2026-08-29 and
a checklist nobody ticked on the night would lie in a new way; CLAUDE.md now
says work is tracked in both Jira and rewrite-status.md, and points at
release-process.md rather than the cutover file.
What I checked, since I did not author it: every redirected comment resolves
to content that really moved — the DRAFT/WIP divergence, the unweighted
billable_percentage and total_revenue quirks, the public-holiday working-day
split and the quote-event double count are all in rewrite-history.md now; the
two renamed doc targets (development_session.md, release-process.md) exist;
and the test docstring's "a client error IS an AppError" item is really in
rewrite-status.md. The five apps/accounting and two test/spec edits are
comment-only.
One line is mine: I deleted the "Re-dispose the two blocked-by: rows that
named session replays" task, which 7ff50da completed — that file only shrinks,
and finished work is deleted the moment it is finished.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
@coderabbitai

coderabbitaiBot commented Sep 2, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Team

Run ID: aa049e06-ce1d-4253-a6cc-fbf38800dff3


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Links rather than restates: the ticket is the authority, this file just says
enough for a session to pick it up cold and to know not to start it yet — it
overturns the layout CLAUDE.md documents, so its owner-ratified ADR gates
everything after it.
The scoping caution is worth carrying here because it changes how big the work
looks. The draft justified the epic as breaking seven ORM cycles; traversing
every concrete cross-app relation in the app registry finds exactly one
(accounts <-> job, via Staff.default_labour_subtype). The other six are
one-way, and purchasing -> job, accounting -> job and quoting -> company
already point where the target architecture wants them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@corrin
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Access logging and session replay: the two features left behind at the flip - #137

Open
corrin wants to merge 6 commits into
mainfrom
feat/restore-app-usage-logging
Open

Access logging and session replay: the two features left behind at the flip#137
corrin wants to merge 6 commits into
mainfrom
feat/restore-app-usage-logging

Conversation

@corrin

Copy link
Copy Markdown
Owner

Ports the last two features that were left behind at the flip: the per-request
access log, and session replay end to end. Both were recorded as deferred in
docs/rewrite-status.md, and access logging was the first item on the
post-cutover queue.

Access logging

One line per authenticated request on its own access logger, carrying the
X-Session-Replay-Id that joins a request to its recording.

It goes to the console, not v1's rotating access.log. config/settings.py
already states that journald is the sink, and journald gives the rotation and
retention v1 got from ConcurrentRotatingFileHandler — a file handler would
only put a second copy on disk for an operator to find and prune.

Two v1 constructs are deliberately not ported, both verified broken rather than
assumed:

  • v1 read request.user before calling the view. Under ninja auth — which
    sets request.user during operation dispatch, after every middleware — that
    logs nothing for any /api/** request, i.e. for the whole application, while
    a v1-shaped test still passes. v2 reads the principal after get_response.
  • DisallowedHostMiddleware never did anything.process_exception fires
    only for exceptions raised by the view, but DisallowedHost comes out of
    CommonMiddleware.process_request above it. Django was always returning that
    400; only the traceback was ever the complaint. apps/core/logging_filters.py
    keeps the record of the probe and strips its traceback. The test fails without
    the filter — I checked.

v1's JWT re-authentication block went with it (v2 is cookie-authenticated), and
with it a bare except Exception: pass.

Session replay

Chunk payloads go to a private disk root (SESSION_REPLAY_STORAGE_ROOT, which
instance.sh has been provisioning at 700 all along). The rows index the store
rather than being it, so browsing sessions stay out of every pg_dump.

The store is shared, not a second copy. Phone-call recordings already kept a
metadata row plus a payload on a private root, and v1 wrote that logic twice.
apps/core/file_store.py now owns the path-escape guard, the atomic write and
the refuse-to-overwrite, and phone_call_service uses it too (ADR 0039).

Access follows what the business actually had. A staff member may write to
their own recording and read none of them. v1 gated the API at office staff but
hung the page behind a superuser route; a replay is an unredacted video of
somebody's screen, so reads are superuser-only here.

Upload rules are about not losing a session, and are the part v1 iterated on:
a transient failure puts events back at the front of the buffer (behind later
events they would replay out of order), a 409 means the chunk already landed so
the sequence advances and recording continues, and only 401/403/404 discard the
recording.

Three things v1 lacked or got wrong:

  • CompanyDefaults.session_replay_enabled — an off-switch that is not a deploy.
    v1 had none outside a DEV-only E2E flag.
  • The purge is scheduled (01:30 NZT) and now deletes payloads, not just rows.
    Retention is this feature's only privacy control, so it ships with capture.
  • The scrubber truncates recordings. v1 blanked storage_path and sha256,
    which left the unscrubbed payloads on disk and left the purge unable to
    ever find them again.

Also fixed on the way past: src/api/client.ts carried a STUB comment where the
X-Session-Replay-Id header belonged, which is why every AppError persisted
since the port had a replay column it could never fill. And v1's admin page
"destroyed" its player with innerHTML = '', leaking the previous instance's
timers and listeners on every reselect.

Capture stays off under Playwright-over-ngrok — the E2E run is already
bottlenecked on that tunnel. The new spec clears that opt-out for itself.

Operational assets unblocked

Both blocked-by: rows in v1-disposition.md that named session replay are now
ported: the purge beat entry, and pull_prod_files.sh, whose only blocker was
the storage decision. It takes host and instance-user as required arguments
rather than defaulting to MSM production, and validates instance-user as a plain
unix account name — it is interpolated into the remote --rsync-path, the same
escalation pull_prod_backup.sh already guards.

Docs commit authored by another session

80b7d48 is the work of a concurrent session that exited before it could commit;
I reviewed and committed it rather than leave it in a shared worktree. It makes
rewrite-status.md forward-looking and turns cutover-checklist.md into a
record now the flip has run. I verified every redirected comment resolves to
content that really moved before committing it.

Verification

Commit and push tiers green (ruff, mypy strict, import-linter, find-duplicates,
deptry, exported schema, status table, code-quality, delta goldens, frontend
lint/format/boundary/type-check/audit, generated-client-current, server suites,
makemigrations). Python suite green; 613 frontend unit tests green, 6 of them new
and covering the upload rules above.

Not yet run: frontend/tests/e2e/admin/session-replay.spec.ts. It drives the
cross-layer path no unit test on either side can reach — browser capture, chunk
upload, gzip on disk, events endpoint, player mount — plus a second test toggling
session_replay_enabled off. Per CLAUDE.md this slice is not done until that
passes, so please treat this PR as unverified end-to-end until it does.

🤖 Generated with Claude Code

https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7

corrinand others added 5 commits September 2, 2026 17:27
One line per authenticated request on its own `access` logger, carrying the
X-Session-Replay-Id that joins a request to its recording. It goes to the
console, not v1's rotating access.log: journald already rotates, retains and
greps that stream.
Two v1 constructs are deliberately not ported. AccessLoggingMiddleware now
reads the principal AFTER get_response — v1 checked it on the way in and
returned early when anonymous, which under ninja auth (which sets request.user
during operation dispatch, after middleware) would log nothing for any /api/**
request while a v1-shaped test still passed. And DisallowedHostMiddleware never
did anything: process_exception fires only for view exceptions, while
DisallowedHost comes out of CommonMiddleware.process_request above it. Django
was always returning that 400; only the traceback was the complaint, so a
logging filter keeps the record of the probe and drops its traceback.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
Chunk payloads go to a private disk root (SESSION_REPLAY_STORAGE_ROOT, which
instance.sh has been provisioning at 700 all along); the rows index the store
rather than being it, so browsing sessions stay out of every pg_dump.
The store itself is shared, not a second copy. Phone-call recordings already
kept a metadata row plus a payload on a private root, and v1 wrote that logic
twice; apps/core/file_store.py now owns the path-escape guard, the atomic
write and the refuse-to-overwrite, and phone_call_service uses it too.
Access follows what the business actually had: a staff member may write to
their own recording and read none of them. v1 gated the API at office staff
but hung the page behind a superuser route, so reads are superuser-only here —
a replay is an unredacted video of somebody's screen.
Three things v1 lacked or got wrong:
- CompanyDefaults.session_replay_enabled, an off-switch that is not a deploy.
- The purge is scheduled and now deletes payloads, not just rows. Retention is
this feature's only privacy control, so it ships with capture.
- The scrubber truncates recordings. v1 blanked storage_path and sha256, which
left the unscrubbed payloads on disk AND left the purge unable to find them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
rrweb records every authenticated page and flushes batches every ten seconds.
The upload rules are the parts v1 iterated on and they are about not losing a
session: a transient failure puts the events back at the FRONT of the buffer
(behind later events they would replay out of order), a 409 means the chunk
already landed so the sequence advances and recording continues, and only
401/403/404 discard the recording.
X-Session-Replay-Id now leaves the client for real — src/api/client.ts had
carried a STUB comment where the header belonged, which is why every AppError
persisted since the port had a replay column it could never fill.
Capture stays off under Playwright-over-ngrok: the E2E run is already
bottlenecked on that tunnel and recording every spec would add chunk uploads
to it. The new spec clears that opt-out for itself, because the cross-layer
path — browser capture to gzip on disk to the events endpoint to the player —
is the one no unit test on either side can reach.
Two v1 defects not carried over: the admin page destroyed its player by
clearing innerHTML, leaking the previous instance's timers and listeners on
every reselect; and the wire's opaque JSON is now checked against rrweb's
event shape rather than trusted, so an unplayable recording fails visibly
instead of showing a blank frame.
The one docs line is the spec-count row the status-table gate regenerates
because this commit adds a spec file; the rest of that file is another
workstream's and is deliberately untouched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
Both rows named session replay as their blocker, and both blockers are now
answered, so the disposition rule applies: the code arrives with the feature.
pull_prod_files.sh is ported. Its blocker was the storage decision — payloads
live on disk under SESSION_REPLAY_STORAGE_ROOT, so all three destinations
exist and the script has somewhere to put them. Two changes from v1: host and
instance-user are required arguments rather than defaulting to MSM production,
and instance-user is validated as a plain unix account name, because it is
interpolated into the remote --rsync-path and is the same escalation
pull_prod_backup.sh already guards. Destinations are all resolved before the
first byte moves, since a missing root found halfway through leaves a partial
pull that looks complete on the directories it did reach.
purge_old_session_replays_daily is ported: scheduled 01:30 NZT, and deleting
chunk payloads and the recording directory rather than only rows.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
Authored by a concurrent session (update-rewrite-status-postrelease), which
exited before it could commit. I reviewed the diffs and committed them rather
than leave the work uncommitted in a shared worktree; I did not write them.
What it does: rewrite-status.md becomes forward-looking, holding tasks only,
with what shipped moved to rewrite-history.md; cutover-checklist.md becomes
cutover-record.md in all but filename, because the flip ran on 2026-08-29 and
a checklist nobody ticked on the night would lie in a new way; CLAUDE.md now
says work is tracked in both Jira and rewrite-status.md, and points at
release-process.md rather than the cutover file.
What I checked, since I did not author it: every redirected comment resolves
to content that really moved — the DRAFT/WIP divergence, the unweighted
billable_percentage and total_revenue quirks, the public-holiday working-day
split and the quote-event double count are all in rewrite-history.md now; the
two renamed doc targets (development_session.md, release-process.md) exist;
and the test docstring's "a client error IS an AppError" item is really in
rewrite-status.md. The five apps/accounting and two test/spec edits are
comment-only.
One line is mine: I deleted the "Re-dispose the two blocked-by: rows that
named session replays" task, which 7ff50da completed — that file only shrinks,
and finished work is deleted the moment it is finished.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
@coderabbitai

coderabbitaiBot commented Sep 2, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Team

Run ID: aa049e06-ce1d-4253-a6cc-fbf38800dff3


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Links rather than restates: the ticket is the authority, this file just says
enough for a session to pick it up cold and to know not to start it yet — it
overturns the layout CLAUDE.md documents, so its owner-ratified ADR gates
everything after it.
The scoping caution is worth carrying here because it changes how big the work
looks. The draft justified the epic as breaking seven ORM cycles; traversing
every concrete cross-app relation in the app registry finds exactly one
(accounts <-> job, via Staff.default_labour_subtype). The other six are
one-way, and purchasing -> job, accounting -> job and quoting -> company
already point where the target architecture wants them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@corrin
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Access logging and session replay: the two features left behind at the flip - #137

Open
corrin wants to merge 6 commits into
mainfrom
feat/restore-app-usage-logging
Open

Access logging and session replay: the two features left behind at the flip#137
corrin wants to merge 6 commits into
mainfrom
feat/restore-app-usage-logging

Conversation

@corrin

Copy link
Copy Markdown
Owner

Ports the last two features that were left behind at the flip: the per-request
access log, and session replay end to end. Both were recorded as deferred in
docs/rewrite-status.md, and access logging was the first item on the
post-cutover queue.

Access logging

One line per authenticated request on its own access logger, carrying the
X-Session-Replay-Id that joins a request to its recording.

It goes to the console, not v1's rotating access.log. config/settings.py
already states that journald is the sink, and journald gives the rotation and
retention v1 got from ConcurrentRotatingFileHandler — a file handler would
only put a second copy on disk for an operator to find and prune.

Two v1 constructs are deliberately not ported, both verified broken rather than
assumed:

  • v1 read request.user before calling the view. Under ninja auth — which
    sets request.user during operation dispatch, after every middleware — that
    logs nothing for any /api/** request, i.e. for the whole application, while
    a v1-shaped test still passes. v2 reads the principal after get_response.
  • DisallowedHostMiddleware never did anything.process_exception fires
    only for exceptions raised by the view, but DisallowedHost comes out of
    CommonMiddleware.process_request above it. Django was always returning that
    400; only the traceback was ever the complaint. apps/core/logging_filters.py
    keeps the record of the probe and strips its traceback. The test fails without
    the filter — I checked.

v1's JWT re-authentication block went with it (v2 is cookie-authenticated), and
with it a bare except Exception: pass.

Session replay

Chunk payloads go to a private disk root (SESSION_REPLAY_STORAGE_ROOT, which
instance.sh has been provisioning at 700 all along). The rows index the store
rather than being it, so browsing sessions stay out of every pg_dump.

The store is shared, not a second copy. Phone-call recordings already kept a
metadata row plus a payload on a private root, and v1 wrote that logic twice.
apps/core/file_store.py now owns the path-escape guard, the atomic write and
the refuse-to-overwrite, and phone_call_service uses it too (ADR 0039).

Access follows what the business actually had. A staff member may write to
their own recording and read none of them. v1 gated the API at office staff but
hung the page behind a superuser route; a replay is an unredacted video of
somebody's screen, so reads are superuser-only here.

Upload rules are about not losing a session, and are the part v1 iterated on:
a transient failure puts events back at the front of the buffer (behind later
events they would replay out of order), a 409 means the chunk already landed so
the sequence advances and recording continues, and only 401/403/404 discard the
recording.

Three things v1 lacked or got wrong:

  • CompanyDefaults.session_replay_enabled — an off-switch that is not a deploy.
    v1 had none outside a DEV-only E2E flag.
  • The purge is scheduled (01:30 NZT) and now deletes payloads, not just rows.
    Retention is this feature's only privacy control, so it ships with capture.
  • The scrubber truncates recordings. v1 blanked storage_path and sha256,
    which left the unscrubbed payloads on disk and left the purge unable to
    ever find them again.

Also fixed on the way past: src/api/client.ts carried a STUB comment where the
X-Session-Replay-Id header belonged, which is why every AppError persisted
since the port had a replay column it could never fill. And v1's admin page
"destroyed" its player with innerHTML = '', leaking the previous instance's
timers and listeners on every reselect.

Capture stays off under Playwright-over-ngrok — the E2E run is already
bottlenecked on that tunnel. The new spec clears that opt-out for itself.

Operational assets unblocked

Both blocked-by: rows in v1-disposition.md that named session replay are now
ported: the purge beat entry, and pull_prod_files.sh, whose only blocker was
the storage decision. It takes host and instance-user as required arguments
rather than defaulting to MSM production, and validates instance-user as a plain
unix account name — it is interpolated into the remote --rsync-path, the same
escalation pull_prod_backup.sh already guards.

Docs commit authored by another session

80b7d48 is the work of a concurrent session that exited before it could commit;
I reviewed and committed it rather than leave it in a shared worktree. It makes
rewrite-status.md forward-looking and turns cutover-checklist.md into a
record now the flip has run. I verified every redirected comment resolves to
content that really moved before committing it.

Verification

Commit and push tiers green (ruff, mypy strict, import-linter, find-duplicates,
deptry, exported schema, status table, code-quality, delta goldens, frontend
lint/format/boundary/type-check/audit, generated-client-current, server suites,
makemigrations). Python suite green; 613 frontend unit tests green, 6 of them new
and covering the upload rules above.

Not yet run: frontend/tests/e2e/admin/session-replay.spec.ts. It drives the
cross-layer path no unit test on either side can reach — browser capture, chunk
upload, gzip on disk, events endpoint, player mount — plus a second test toggling
session_replay_enabled off. Per CLAUDE.md this slice is not done until that
passes, so please treat this PR as unverified end-to-end until it does.

🤖 Generated with Claude Code

https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7

corrinand others added 5 commits September 2, 2026 17:27
One line per authenticated request on its own `access` logger, carrying the
X-Session-Replay-Id that joins a request to its recording. It goes to the
console, not v1's rotating access.log: journald already rotates, retains and
greps that stream.
Two v1 constructs are deliberately not ported. AccessLoggingMiddleware now
reads the principal AFTER get_response — v1 checked it on the way in and
returned early when anonymous, which under ninja auth (which sets request.user
during operation dispatch, after middleware) would log nothing for any /api/**
request while a v1-shaped test still passed. And DisallowedHostMiddleware never
did anything: process_exception fires only for view exceptions, while
DisallowedHost comes out of CommonMiddleware.process_request above it. Django
was always returning that 400; only the traceback was the complaint, so a
logging filter keeps the record of the probe and drops its traceback.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
Chunk payloads go to a private disk root (SESSION_REPLAY_STORAGE_ROOT, which
instance.sh has been provisioning at 700 all along); the rows index the store
rather than being it, so browsing sessions stay out of every pg_dump.
The store itself is shared, not a second copy. Phone-call recordings already
kept a metadata row plus a payload on a private root, and v1 wrote that logic
twice; apps/core/file_store.py now owns the path-escape guard, the atomic
write and the refuse-to-overwrite, and phone_call_service uses it too.
Access follows what the business actually had: a staff member may write to
their own recording and read none of them. v1 gated the API at office staff
but hung the page behind a superuser route, so reads are superuser-only here —
a replay is an unredacted video of somebody's screen.
Three things v1 lacked or got wrong:
- CompanyDefaults.session_replay_enabled, an off-switch that is not a deploy.
- The purge is scheduled and now deletes payloads, not just rows. Retention is
this feature's only privacy control, so it ships with capture.
- The scrubber truncates recordings. v1 blanked storage_path and sha256, which
left the unscrubbed payloads on disk AND left the purge unable to find them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
rrweb records every authenticated page and flushes batches every ten seconds.
The upload rules are the parts v1 iterated on and they are about not losing a
session: a transient failure puts the events back at the FRONT of the buffer
(behind later events they would replay out of order), a 409 means the chunk
already landed so the sequence advances and recording continues, and only
401/403/404 discard the recording.
X-Session-Replay-Id now leaves the client for real — src/api/client.ts had
carried a STUB comment where the header belonged, which is why every AppError
persisted since the port had a replay column it could never fill.
Capture stays off under Playwright-over-ngrok: the E2E run is already
bottlenecked on that tunnel and recording every spec would add chunk uploads
to it. The new spec clears that opt-out for itself, because the cross-layer
path — browser capture to gzip on disk to the events endpoint to the player —
is the one no unit test on either side can reach.
Two v1 defects not carried over: the admin page destroyed its player by
clearing innerHTML, leaking the previous instance's timers and listeners on
every reselect; and the wire's opaque JSON is now checked against rrweb's
event shape rather than trusted, so an unplayable recording fails visibly
instead of showing a blank frame.
The one docs line is the spec-count row the status-table gate regenerates
because this commit adds a spec file; the rest of that file is another
workstream's and is deliberately untouched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
Both rows named session replay as their blocker, and both blockers are now
answered, so the disposition rule applies: the code arrives with the feature.
pull_prod_files.sh is ported. Its blocker was the storage decision — payloads
live on disk under SESSION_REPLAY_STORAGE_ROOT, so all three destinations
exist and the script has somewhere to put them. Two changes from v1: host and
instance-user are required arguments rather than defaulting to MSM production,
and instance-user is validated as a plain unix account name, because it is
interpolated into the remote --rsync-path and is the same escalation
pull_prod_backup.sh already guards. Destinations are all resolved before the
first byte moves, since a missing root found halfway through leaves a partial
pull that looks complete on the directories it did reach.
purge_old_session_replays_daily is ported: scheduled 01:30 NZT, and deleting
chunk payloads and the recording directory rather than only rows.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
Authored by a concurrent session (update-rewrite-status-postrelease), which
exited before it could commit. I reviewed the diffs and committed them rather
than leave the work uncommitted in a shared worktree; I did not write them.
What it does: rewrite-status.md becomes forward-looking, holding tasks only,
with what shipped moved to rewrite-history.md; cutover-checklist.md becomes
cutover-record.md in all but filename, because the flip ran on 2026-08-29 and
a checklist nobody ticked on the night would lie in a new way; CLAUDE.md now
says work is tracked in both Jira and rewrite-status.md, and points at
release-process.md rather than the cutover file.
What I checked, since I did not author it: every redirected comment resolves
to content that really moved — the DRAFT/WIP divergence, the unweighted
billable_percentage and total_revenue quirks, the public-holiday working-day
split and the quote-event double count are all in rewrite-history.md now; the
two renamed doc targets (development_session.md, release-process.md) exist;
and the test docstring's "a client error IS an AppError" item is really in
rewrite-status.md. The five apps/accounting and two test/spec edits are
comment-only.
One line is mine: I deleted the "Re-dispose the two blocked-by: rows that
named session replays" task, which 7ff50da completed — that file only shrinks,
and finished work is deleted the moment it is finished.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
@coderabbitai

coderabbitaiBot commented Sep 2, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Team

Run ID: aa049e06-ce1d-4253-a6cc-fbf38800dff3


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Links rather than restates: the ticket is the authority, this file just says
enough for a session to pick it up cold and to know not to start it yet — it
overturns the layout CLAUDE.md documents, so its owner-ratified ADR gates
everything after it.
The scoping caution is worth carrying here because it changes how big the work
looks. The draft justified the epic as breaking seven ORM cycles; traversing
every concrete cross-app relation in the app registry finds exactly one
(accounts <-> job, via Staff.default_labour_subtype). The other six are
one-way, and purchasing -> job, accounting -> job and quoting -> company
already point where the target architecture wants them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@corrin
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Access logging and session replay: the two features left behind at the flip - #137

Open
corrin wants to merge 6 commits into
mainfrom
feat/restore-app-usage-logging
Open

Access logging and session replay: the two features left behind at the flip#137
corrin wants to merge 6 commits into
mainfrom
feat/restore-app-usage-logging

Conversation

@corrin

Copy link
Copy Markdown
Owner

Ports the last two features that were left behind at the flip: the per-request
access log, and session replay end to end. Both were recorded as deferred in
docs/rewrite-status.md, and access logging was the first item on the
post-cutover queue.

Access logging

One line per authenticated request on its own access logger, carrying the
X-Session-Replay-Id that joins a request to its recording.

It goes to the console, not v1's rotating access.log. config/settings.py
already states that journald is the sink, and journald gives the rotation and
retention v1 got from ConcurrentRotatingFileHandler — a file handler would
only put a second copy on disk for an operator to find and prune.

Two v1 constructs are deliberately not ported, both verified broken rather than
assumed:

  • v1 read request.user before calling the view. Under ninja auth — which
    sets request.user during operation dispatch, after every middleware — that
    logs nothing for any /api/** request, i.e. for the whole application, while
    a v1-shaped test still passes. v2 reads the principal after get_response.
  • DisallowedHostMiddleware never did anything.process_exception fires
    only for exceptions raised by the view, but DisallowedHost comes out of
    CommonMiddleware.process_request above it. Django was always returning that
    400; only the traceback was ever the complaint. apps/core/logging_filters.py
    keeps the record of the probe and strips its traceback. The test fails without
    the filter — I checked.

v1's JWT re-authentication block went with it (v2 is cookie-authenticated), and
with it a bare except Exception: pass.

Session replay

Chunk payloads go to a private disk root (SESSION_REPLAY_STORAGE_ROOT, which
instance.sh has been provisioning at 700 all along). The rows index the store
rather than being it, so browsing sessions stay out of every pg_dump.

The store is shared, not a second copy. Phone-call recordings already kept a
metadata row plus a payload on a private root, and v1 wrote that logic twice.
apps/core/file_store.py now owns the path-escape guard, the atomic write and
the refuse-to-overwrite, and phone_call_service uses it too (ADR 0039).

Access follows what the business actually had. A staff member may write to
their own recording and read none of them. v1 gated the API at office staff but
hung the page behind a superuser route; a replay is an unredacted video of
somebody's screen, so reads are superuser-only here.

Upload rules are about not losing a session, and are the part v1 iterated on:
a transient failure puts events back at the front of the buffer (behind later
events they would replay out of order), a 409 means the chunk already landed so
the sequence advances and recording continues, and only 401/403/404 discard the
recording.

Three things v1 lacked or got wrong:

  • CompanyDefaults.session_replay_enabled — an off-switch that is not a deploy.
    v1 had none outside a DEV-only E2E flag.
  • The purge is scheduled (01:30 NZT) and now deletes payloads, not just rows.
    Retention is this feature's only privacy control, so it ships with capture.
  • The scrubber truncates recordings. v1 blanked storage_path and sha256,
    which left the unscrubbed payloads on disk and left the purge unable to
    ever find them again.

Also fixed on the way past: src/api/client.ts carried a STUB comment where the
X-Session-Replay-Id header belonged, which is why every AppError persisted
since the port had a replay column it could never fill. And v1's admin page
"destroyed" its player with innerHTML = '', leaking the previous instance's
timers and listeners on every reselect.

Capture stays off under Playwright-over-ngrok — the E2E run is already
bottlenecked on that tunnel. The new spec clears that opt-out for itself.

Operational assets unblocked

Both blocked-by: rows in v1-disposition.md that named session replay are now
ported: the purge beat entry, and pull_prod_files.sh, whose only blocker was
the storage decision. It takes host and instance-user as required arguments
rather than defaulting to MSM production, and validates instance-user as a plain
unix account name — it is interpolated into the remote --rsync-path, the same
escalation pull_prod_backup.sh already guards.

Docs commit authored by another session

80b7d48 is the work of a concurrent session that exited before it could commit;
I reviewed and committed it rather than leave it in a shared worktree. It makes
rewrite-status.md forward-looking and turns cutover-checklist.md into a
record now the flip has run. I verified every redirected comment resolves to
content that really moved before committing it.

Verification

Commit and push tiers green (ruff, mypy strict, import-linter, find-duplicates,
deptry, exported schema, status table, code-quality, delta goldens, frontend
lint/format/boundary/type-check/audit, generated-client-current, server suites,
makemigrations). Python suite green; 613 frontend unit tests green, 6 of them new
and covering the upload rules above.

Not yet run: frontend/tests/e2e/admin/session-replay.spec.ts. It drives the
cross-layer path no unit test on either side can reach — browser capture, chunk
upload, gzip on disk, events endpoint, player mount — plus a second test toggling
session_replay_enabled off. Per CLAUDE.md this slice is not done until that
passes, so please treat this PR as unverified end-to-end until it does.

🤖 Generated with Claude Code

https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7

corrinand others added 5 commits September 2, 2026 17:27
One line per authenticated request on its own `access` logger, carrying the
X-Session-Replay-Id that joins a request to its recording. It goes to the
console, not v1's rotating access.log: journald already rotates, retains and
greps that stream.
Two v1 constructs are deliberately not ported. AccessLoggingMiddleware now
reads the principal AFTER get_response — v1 checked it on the way in and
returned early when anonymous, which under ninja auth (which sets request.user
during operation dispatch, after middleware) would log nothing for any /api/**
request while a v1-shaped test still passed. And DisallowedHostMiddleware never
did anything: process_exception fires only for view exceptions, while
DisallowedHost comes out of CommonMiddleware.process_request above it. Django
was always returning that 400; only the traceback was the complaint, so a
logging filter keeps the record of the probe and drops its traceback.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
Chunk payloads go to a private disk root (SESSION_REPLAY_STORAGE_ROOT, which
instance.sh has been provisioning at 700 all along); the rows index the store
rather than being it, so browsing sessions stay out of every pg_dump.
The store itself is shared, not a second copy. Phone-call recordings already
kept a metadata row plus a payload on a private root, and v1 wrote that logic
twice; apps/core/file_store.py now owns the path-escape guard, the atomic
write and the refuse-to-overwrite, and phone_call_service uses it too.
Access follows what the business actually had: a staff member may write to
their own recording and read none of them. v1 gated the API at office staff
but hung the page behind a superuser route, so reads are superuser-only here —
a replay is an unredacted video of somebody's screen.
Three things v1 lacked or got wrong:
- CompanyDefaults.session_replay_enabled, an off-switch that is not a deploy.
- The purge is scheduled and now deletes payloads, not just rows. Retention is
this feature's only privacy control, so it ships with capture.
- The scrubber truncates recordings. v1 blanked storage_path and sha256, which
left the unscrubbed payloads on disk AND left the purge unable to find them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
rrweb records every authenticated page and flushes batches every ten seconds.
The upload rules are the parts v1 iterated on and they are about not losing a
session: a transient failure puts the events back at the FRONT of the buffer
(behind later events they would replay out of order), a 409 means the chunk
already landed so the sequence advances and recording continues, and only
401/403/404 discard the recording.
X-Session-Replay-Id now leaves the client for real — src/api/client.ts had
carried a STUB comment where the header belonged, which is why every AppError
persisted since the port had a replay column it could never fill.
Capture stays off under Playwright-over-ngrok: the E2E run is already
bottlenecked on that tunnel and recording every spec would add chunk uploads
to it. The new spec clears that opt-out for itself, because the cross-layer
path — browser capture to gzip on disk to the events endpoint to the player —
is the one no unit test on either side can reach.
Two v1 defects not carried over: the admin page destroyed its player by
clearing innerHTML, leaking the previous instance's timers and listeners on
every reselect; and the wire's opaque JSON is now checked against rrweb's
event shape rather than trusted, so an unplayable recording fails visibly
instead of showing a blank frame.
The one docs line is the spec-count row the status-table gate regenerates
because this commit adds a spec file; the rest of that file is another
workstream's and is deliberately untouched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
Both rows named session replay as their blocker, and both blockers are now
answered, so the disposition rule applies: the code arrives with the feature.
pull_prod_files.sh is ported. Its blocker was the storage decision — payloads
live on disk under SESSION_REPLAY_STORAGE_ROOT, so all three destinations
exist and the script has somewhere to put them. Two changes from v1: host and
instance-user are required arguments rather than defaulting to MSM production,
and instance-user is validated as a plain unix account name, because it is
interpolated into the remote --rsync-path and is the same escalation
pull_prod_backup.sh already guards. Destinations are all resolved before the
first byte moves, since a missing root found halfway through leaves a partial
pull that looks complete on the directories it did reach.
purge_old_session_replays_daily is ported: scheduled 01:30 NZT, and deleting
chunk payloads and the recording directory rather than only rows.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
Authored by a concurrent session (update-rewrite-status-postrelease), which
exited before it could commit. I reviewed the diffs and committed them rather
than leave the work uncommitted in a shared worktree; I did not write them.
What it does: rewrite-status.md becomes forward-looking, holding tasks only,
with what shipped moved to rewrite-history.md; cutover-checklist.md becomes
cutover-record.md in all but filename, because the flip ran on 2026-08-29 and
a checklist nobody ticked on the night would lie in a new way; CLAUDE.md now
says work is tracked in both Jira and rewrite-status.md, and points at
release-process.md rather than the cutover file.
What I checked, since I did not author it: every redirected comment resolves
to content that really moved — the DRAFT/WIP divergence, the unweighted
billable_percentage and total_revenue quirks, the public-holiday working-day
split and the quote-event double count are all in rewrite-history.md now; the
two renamed doc targets (development_session.md, release-process.md) exist;
and the test docstring's "a client error IS an AppError" item is really in
rewrite-status.md. The five apps/accounting and two test/spec edits are
comment-only.
One line is mine: I deleted the "Re-dispose the two blocked-by: rows that
named session replays" task, which 7ff50da completed — that file only shrinks,
and finished work is deleted the moment it is finished.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
@coderabbitai

coderabbitaiBot commented Sep 2, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Team

Run ID: aa049e06-ce1d-4253-a6cc-fbf38800dff3


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Links rather than restates: the ticket is the authority, this file just says
enough for a session to pick it up cold and to know not to start it yet — it
overturns the layout CLAUDE.md documents, so its owner-ratified ADR gates
everything after it.
The scoping caution is worth carrying here because it changes how big the work
looks. The draft justified the epic as breaking seven ORM cycles; traversing
every concrete cross-app relation in the app registry finds exactly one
(accounts <-> job, via Staff.default_labour_subtype). The other six are
one-way, and purchasing -> job, accounting -> job and quoting -> company
already point where the target architecture wants them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@corrin
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Access logging and session replay: the two features left behind at the flip - #137

Open
corrin wants to merge 6 commits into
mainfrom
feat/restore-app-usage-logging
Open

Access logging and session replay: the two features left behind at the flip#137
corrin wants to merge 6 commits into
mainfrom
feat/restore-app-usage-logging

Conversation

@corrin

Copy link
Copy Markdown
Owner

Ports the last two features that were left behind at the flip: the per-request
access log, and session replay end to end. Both were recorded as deferred in
docs/rewrite-status.md, and access logging was the first item on the
post-cutover queue.

Access logging

One line per authenticated request on its own access logger, carrying the
X-Session-Replay-Id that joins a request to its recording.

It goes to the console, not v1's rotating access.log. config/settings.py
already states that journald is the sink, and journald gives the rotation and
retention v1 got from ConcurrentRotatingFileHandler — a file handler would
only put a second copy on disk for an operator to find and prune.

Two v1 constructs are deliberately not ported, both verified broken rather than
assumed:

  • v1 read request.user before calling the view. Under ninja auth — which
    sets request.user during operation dispatch, after every middleware — that
    logs nothing for any /api/** request, i.e. for the whole application, while
    a v1-shaped test still passes. v2 reads the principal after get_response.
  • DisallowedHostMiddleware never did anything.process_exception fires
    only for exceptions raised by the view, but DisallowedHost comes out of
    CommonMiddleware.process_request above it. Django was always returning that
    400; only the traceback was ever the complaint. apps/core/logging_filters.py
    keeps the record of the probe and strips its traceback. The test fails without
    the filter — I checked.

v1's JWT re-authentication block went with it (v2 is cookie-authenticated), and
with it a bare except Exception: pass.

Session replay

Chunk payloads go to a private disk root (SESSION_REPLAY_STORAGE_ROOT, which
instance.sh has been provisioning at 700 all along). The rows index the store
rather than being it, so browsing sessions stay out of every pg_dump.

The store is shared, not a second copy. Phone-call recordings already kept a
metadata row plus a payload on a private root, and v1 wrote that logic twice.
apps/core/file_store.py now owns the path-escape guard, the atomic write and
the refuse-to-overwrite, and phone_call_service uses it too (ADR 0039).

Access follows what the business actually had. A staff member may write to
their own recording and read none of them. v1 gated the API at office staff but
hung the page behind a superuser route; a replay is an unredacted video of
somebody's screen, so reads are superuser-only here.

Upload rules are about not losing a session, and are the part v1 iterated on:
a transient failure puts events back at the front of the buffer (behind later
events they would replay out of order), a 409 means the chunk already landed so
the sequence advances and recording continues, and only 401/403/404 discard the
recording.

Three things v1 lacked or got wrong:

  • CompanyDefaults.session_replay_enabled — an off-switch that is not a deploy.
    v1 had none outside a DEV-only E2E flag.
  • The purge is scheduled (01:30 NZT) and now deletes payloads, not just rows.
    Retention is this feature's only privacy control, so it ships with capture.
  • The scrubber truncates recordings. v1 blanked storage_path and sha256,
    which left the unscrubbed payloads on disk and left the purge unable to
    ever find them again.

Also fixed on the way past: src/api/client.ts carried a STUB comment where the
X-Session-Replay-Id header belonged, which is why every AppError persisted
since the port had a replay column it could never fill. And v1's admin page
"destroyed" its player with innerHTML = '', leaking the previous instance's
timers and listeners on every reselect.

Capture stays off under Playwright-over-ngrok — the E2E run is already
bottlenecked on that tunnel. The new spec clears that opt-out for itself.

Operational assets unblocked

Both blocked-by: rows in v1-disposition.md that named session replay are now
ported: the purge beat entry, and pull_prod_files.sh, whose only blocker was
the storage decision. It takes host and instance-user as required arguments
rather than defaulting to MSM production, and validates instance-user as a plain
unix account name — it is interpolated into the remote --rsync-path, the same
escalation pull_prod_backup.sh already guards.

Docs commit authored by another session

80b7d48 is the work of a concurrent session that exited before it could commit;
I reviewed and committed it rather than leave it in a shared worktree. It makes
rewrite-status.md forward-looking and turns cutover-checklist.md into a
record now the flip has run. I verified every redirected comment resolves to
content that really moved before committing it.

Verification

Commit and push tiers green (ruff, mypy strict, import-linter, find-duplicates,
deptry, exported schema, status table, code-quality, delta goldens, frontend
lint/format/boundary/type-check/audit, generated-client-current, server suites,
makemigrations). Python suite green; 613 frontend unit tests green, 6 of them new
and covering the upload rules above.

Not yet run: frontend/tests/e2e/admin/session-replay.spec.ts. It drives the
cross-layer path no unit test on either side can reach — browser capture, chunk
upload, gzip on disk, events endpoint, player mount — plus a second test toggling
session_replay_enabled off. Per CLAUDE.md this slice is not done until that
passes, so please treat this PR as unverified end-to-end until it does.

🤖 Generated with Claude Code

https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7

corrinand others added 5 commits September 2, 2026 17:27
One line per authenticated request on its own `access` logger, carrying the
X-Session-Replay-Id that joins a request to its recording. It goes to the
console, not v1's rotating access.log: journald already rotates, retains and
greps that stream.
Two v1 constructs are deliberately not ported. AccessLoggingMiddleware now
reads the principal AFTER get_response — v1 checked it on the way in and
returned early when anonymous, which under ninja auth (which sets request.user
during operation dispatch, after middleware) would log nothing for any /api/**
request while a v1-shaped test still passed. And DisallowedHostMiddleware never
did anything: process_exception fires only for view exceptions, while
DisallowedHost comes out of CommonMiddleware.process_request above it. Django
was always returning that 400; only the traceback was the complaint, so a
logging filter keeps the record of the probe and drops its traceback.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
Chunk payloads go to a private disk root (SESSION_REPLAY_STORAGE_ROOT, which
instance.sh has been provisioning at 700 all along); the rows index the store
rather than being it, so browsing sessions stay out of every pg_dump.
The store itself is shared, not a second copy. Phone-call recordings already
kept a metadata row plus a payload on a private root, and v1 wrote that logic
twice; apps/core/file_store.py now owns the path-escape guard, the atomic
write and the refuse-to-overwrite, and phone_call_service uses it too.
Access follows what the business actually had: a staff member may write to
their own recording and read none of them. v1 gated the API at office staff
but hung the page behind a superuser route, so reads are superuser-only here —
a replay is an unredacted video of somebody's screen.
Three things v1 lacked or got wrong:
- CompanyDefaults.session_replay_enabled, an off-switch that is not a deploy.
- The purge is scheduled and now deletes payloads, not just rows. Retention is
this feature's only privacy control, so it ships with capture.
- The scrubber truncates recordings. v1 blanked storage_path and sha256, which
left the unscrubbed payloads on disk AND left the purge unable to find them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
rrweb records every authenticated page and flushes batches every ten seconds.
The upload rules are the parts v1 iterated on and they are about not losing a
session: a transient failure puts the events back at the FRONT of the buffer
(behind later events they would replay out of order), a 409 means the chunk
already landed so the sequence advances and recording continues, and only
401/403/404 discard the recording.
X-Session-Replay-Id now leaves the client for real — src/api/client.ts had
carried a STUB comment where the header belonged, which is why every AppError
persisted since the port had a replay column it could never fill.
Capture stays off under Playwright-over-ngrok: the E2E run is already
bottlenecked on that tunnel and recording every spec would add chunk uploads
to it. The new spec clears that opt-out for itself, because the cross-layer
path — browser capture to gzip on disk to the events endpoint to the player —
is the one no unit test on either side can reach.
Two v1 defects not carried over: the admin page destroyed its player by
clearing innerHTML, leaking the previous instance's timers and listeners on
every reselect; and the wire's opaque JSON is now checked against rrweb's
event shape rather than trusted, so an unplayable recording fails visibly
instead of showing a blank frame.
The one docs line is the spec-count row the status-table gate regenerates
because this commit adds a spec file; the rest of that file is another
workstream's and is deliberately untouched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
Both rows named session replay as their blocker, and both blockers are now
answered, so the disposition rule applies: the code arrives with the feature.
pull_prod_files.sh is ported. Its blocker was the storage decision — payloads
live on disk under SESSION_REPLAY_STORAGE_ROOT, so all three destinations
exist and the script has somewhere to put them. Two changes from v1: host and
instance-user are required arguments rather than defaulting to MSM production,
and instance-user is validated as a plain unix account name, because it is
interpolated into the remote --rsync-path and is the same escalation
pull_prod_backup.sh already guards. Destinations are all resolved before the
first byte moves, since a missing root found halfway through leaves a partial
pull that looks complete on the directories it did reach.
purge_old_session_replays_daily is ported: scheduled 01:30 NZT, and deleting
chunk payloads and the recording directory rather than only rows.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
Authored by a concurrent session (update-rewrite-status-postrelease), which
exited before it could commit. I reviewed the diffs and committed them rather
than leave the work uncommitted in a shared worktree; I did not write them.
What it does: rewrite-status.md becomes forward-looking, holding tasks only,
with what shipped moved to rewrite-history.md; cutover-checklist.md becomes
cutover-record.md in all but filename, because the flip ran on 2026-08-29 and
a checklist nobody ticked on the night would lie in a new way; CLAUDE.md now
says work is tracked in both Jira and rewrite-status.md, and points at
release-process.md rather than the cutover file.
What I checked, since I did not author it: every redirected comment resolves
to content that really moved — the DRAFT/WIP divergence, the unweighted
billable_percentage and total_revenue quirks, the public-holiday working-day
split and the quote-event double count are all in rewrite-history.md now; the
two renamed doc targets (development_session.md, release-process.md) exist;
and the test docstring's "a client error IS an AppError" item is really in
rewrite-status.md. The five apps/accounting and two test/spec edits are
comment-only.
One line is mine: I deleted the "Re-dispose the two blocked-by: rows that
named session replays" task, which 7ff50da completed — that file only shrinks,
and finished work is deleted the moment it is finished.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
@coderabbitai

coderabbitaiBot commented Sep 2, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Team

Run ID: aa049e06-ce1d-4253-a6cc-fbf38800dff3


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Links rather than restates: the ticket is the authority, this file just says
enough for a session to pick it up cold and to know not to start it yet — it
overturns the layout CLAUDE.md documents, so its owner-ratified ADR gates
everything after it.
The scoping caution is worth carrying here because it changes how big the work
looks. The draft justified the epic as breaking seven ORM cycles; traversing
every concrete cross-app relation in the app registry finds exactly one
(accounts <-> job, via Staff.default_labour_subtype). The other six are
one-way, and purchasing -> job, accounting -> job and quoting -> company
already point where the target architecture wants them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@corrin
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Access logging and session replay: the two features left behind at the flip - #137

Open
corrin wants to merge 6 commits into
mainfrom
feat/restore-app-usage-logging
Open

Access logging and session replay: the two features left behind at the flip#137
corrin wants to merge 6 commits into
mainfrom
feat/restore-app-usage-logging

Conversation

@corrin

Copy link
Copy Markdown
Owner

Ports the last two features that were left behind at the flip: the per-request
access log, and session replay end to end. Both were recorded as deferred in
docs/rewrite-status.md, and access logging was the first item on the
post-cutover queue.

Access logging

One line per authenticated request on its own access logger, carrying the
X-Session-Replay-Id that joins a request to its recording.

It goes to the console, not v1's rotating access.log. config/settings.py
already states that journald is the sink, and journald gives the rotation and
retention v1 got from ConcurrentRotatingFileHandler — a file handler would
only put a second copy on disk for an operator to find and prune.

Two v1 constructs are deliberately not ported, both verified broken rather than
assumed:

  • v1 read request.user before calling the view. Under ninja auth — which
    sets request.user during operation dispatch, after every middleware — that
    logs nothing for any /api/** request, i.e. for the whole application, while
    a v1-shaped test still passes. v2 reads the principal after get_response.
  • DisallowedHostMiddleware never did anything.process_exception fires
    only for exceptions raised by the view, but DisallowedHost comes out of
    CommonMiddleware.process_request above it. Django was always returning that
    400; only the traceback was ever the complaint. apps/core/logging_filters.py
    keeps the record of the probe and strips its traceback. The test fails without
    the filter — I checked.

v1's JWT re-authentication block went with it (v2 is cookie-authenticated), and
with it a bare except Exception: pass.

Session replay

Chunk payloads go to a private disk root (SESSION_REPLAY_STORAGE_ROOT, which
instance.sh has been provisioning at 700 all along). The rows index the store
rather than being it, so browsing sessions stay out of every pg_dump.

The store is shared, not a second copy. Phone-call recordings already kept a
metadata row plus a payload on a private root, and v1 wrote that logic twice.
apps/core/file_store.py now owns the path-escape guard, the atomic write and
the refuse-to-overwrite, and phone_call_service uses it too (ADR 0039).

Access follows what the business actually had. A staff member may write to
their own recording and read none of them. v1 gated the API at office staff but
hung the page behind a superuser route; a replay is an unredacted video of
somebody's screen, so reads are superuser-only here.

Upload rules are about not losing a session, and are the part v1 iterated on:
a transient failure puts events back at the front of the buffer (behind later
events they would replay out of order), a 409 means the chunk already landed so
the sequence advances and recording continues, and only 401/403/404 discard the
recording.

Three things v1 lacked or got wrong:

  • CompanyDefaults.session_replay_enabled — an off-switch that is not a deploy.
    v1 had none outside a DEV-only E2E flag.
  • The purge is scheduled (01:30 NZT) and now deletes payloads, not just rows.
    Retention is this feature's only privacy control, so it ships with capture.
  • The scrubber truncates recordings. v1 blanked storage_path and sha256,
    which left the unscrubbed payloads on disk and left the purge unable to
    ever find them again.

Also fixed on the way past: src/api/client.ts carried a STUB comment where the
X-Session-Replay-Id header belonged, which is why every AppError persisted
since the port had a replay column it could never fill. And v1's admin page
"destroyed" its player with innerHTML = '', leaking the previous instance's
timers and listeners on every reselect.

Capture stays off under Playwright-over-ngrok — the E2E run is already
bottlenecked on that tunnel. The new spec clears that opt-out for itself.

Operational assets unblocked

Both blocked-by: rows in v1-disposition.md that named session replay are now
ported: the purge beat entry, and pull_prod_files.sh, whose only blocker was
the storage decision. It takes host and instance-user as required arguments
rather than defaulting to MSM production, and validates instance-user as a plain
unix account name — it is interpolated into the remote --rsync-path, the same
escalation pull_prod_backup.sh already guards.

Docs commit authored by another session

80b7d48 is the work of a concurrent session that exited before it could commit;
I reviewed and committed it rather than leave it in a shared worktree. It makes
rewrite-status.md forward-looking and turns cutover-checklist.md into a
record now the flip has run. I verified every redirected comment resolves to
content that really moved before committing it.

Verification

Commit and push tiers green (ruff, mypy strict, import-linter, find-duplicates,
deptry, exported schema, status table, code-quality, delta goldens, frontend
lint/format/boundary/type-check/audit, generated-client-current, server suites,
makemigrations). Python suite green; 613 frontend unit tests green, 6 of them new
and covering the upload rules above.

Not yet run: frontend/tests/e2e/admin/session-replay.spec.ts. It drives the
cross-layer path no unit test on either side can reach — browser capture, chunk
upload, gzip on disk, events endpoint, player mount — plus a second test toggling
session_replay_enabled off. Per CLAUDE.md this slice is not done until that
passes, so please treat this PR as unverified end-to-end until it does.

🤖 Generated with Claude Code

https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7

corrinand others added 5 commits September 2, 2026 17:27
One line per authenticated request on its own `access` logger, carrying the
X-Session-Replay-Id that joins a request to its recording. It goes to the
console, not v1's rotating access.log: journald already rotates, retains and
greps that stream.
Two v1 constructs are deliberately not ported. AccessLoggingMiddleware now
reads the principal AFTER get_response — v1 checked it on the way in and
returned early when anonymous, which under ninja auth (which sets request.user
during operation dispatch, after middleware) would log nothing for any /api/**
request while a v1-shaped test still passed. And DisallowedHostMiddleware never
did anything: process_exception fires only for view exceptions, while
DisallowedHost comes out of CommonMiddleware.process_request above it. Django
was always returning that 400; only the traceback was the complaint, so a
logging filter keeps the record of the probe and drops its traceback.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
Chunk payloads go to a private disk root (SESSION_REPLAY_STORAGE_ROOT, which
instance.sh has been provisioning at 700 all along); the rows index the store
rather than being it, so browsing sessions stay out of every pg_dump.
The store itself is shared, not a second copy. Phone-call recordings already
kept a metadata row plus a payload on a private root, and v1 wrote that logic
twice; apps/core/file_store.py now owns the path-escape guard, the atomic
write and the refuse-to-overwrite, and phone_call_service uses it too.
Access follows what the business actually had: a staff member may write to
their own recording and read none of them. v1 gated the API at office staff
but hung the page behind a superuser route, so reads are superuser-only here —
a replay is an unredacted video of somebody's screen.
Three things v1 lacked or got wrong:
- CompanyDefaults.session_replay_enabled, an off-switch that is not a deploy.
- The purge is scheduled and now deletes payloads, not just rows. Retention is
this feature's only privacy control, so it ships with capture.
- The scrubber truncates recordings. v1 blanked storage_path and sha256, which
left the unscrubbed payloads on disk AND left the purge unable to find them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
rrweb records every authenticated page and flushes batches every ten seconds.
The upload rules are the parts v1 iterated on and they are about not losing a
session: a transient failure puts the events back at the FRONT of the buffer
(behind later events they would replay out of order), a 409 means the chunk
already landed so the sequence advances and recording continues, and only
401/403/404 discard the recording.
X-Session-Replay-Id now leaves the client for real — src/api/client.ts had
carried a STUB comment where the header belonged, which is why every AppError
persisted since the port had a replay column it could never fill.
Capture stays off under Playwright-over-ngrok: the E2E run is already
bottlenecked on that tunnel and recording every spec would add chunk uploads
to it. The new spec clears that opt-out for itself, because the cross-layer
path — browser capture to gzip on disk to the events endpoint to the player —
is the one no unit test on either side can reach.
Two v1 defects not carried over: the admin page destroyed its player by
clearing innerHTML, leaking the previous instance's timers and listeners on
every reselect; and the wire's opaque JSON is now checked against rrweb's
event shape rather than trusted, so an unplayable recording fails visibly
instead of showing a blank frame.
The one docs line is the spec-count row the status-table gate regenerates
because this commit adds a spec file; the rest of that file is another
workstream's and is deliberately untouched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
Both rows named session replay as their blocker, and both blockers are now
answered, so the disposition rule applies: the code arrives with the feature.
pull_prod_files.sh is ported. Its blocker was the storage decision — payloads
live on disk under SESSION_REPLAY_STORAGE_ROOT, so all three destinations
exist and the script has somewhere to put them. Two changes from v1: host and
instance-user are required arguments rather than defaulting to MSM production,
and instance-user is validated as a plain unix account name, because it is
interpolated into the remote --rsync-path and is the same escalation
pull_prod_backup.sh already guards. Destinations are all resolved before the
first byte moves, since a missing root found halfway through leaves a partial
pull that looks complete on the directories it did reach.
purge_old_session_replays_daily is ported: scheduled 01:30 NZT, and deleting
chunk payloads and the recording directory rather than only rows.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
Authored by a concurrent session (update-rewrite-status-postrelease), which
exited before it could commit. I reviewed the diffs and committed them rather
than leave the work uncommitted in a shared worktree; I did not write them.
What it does: rewrite-status.md becomes forward-looking, holding tasks only,
with what shipped moved to rewrite-history.md; cutover-checklist.md becomes
cutover-record.md in all but filename, because the flip ran on 2026-08-29 and
a checklist nobody ticked on the night would lie in a new way; CLAUDE.md now
says work is tracked in both Jira and rewrite-status.md, and points at
release-process.md rather than the cutover file.
What I checked, since I did not author it: every redirected comment resolves
to content that really moved — the DRAFT/WIP divergence, the unweighted
billable_percentage and total_revenue quirks, the public-holiday working-day
split and the quote-event double count are all in rewrite-history.md now; the
two renamed doc targets (development_session.md, release-process.md) exist;
and the test docstring's "a client error IS an AppError" item is really in
rewrite-status.md. The five apps/accounting and two test/spec edits are
comment-only.
One line is mine: I deleted the "Re-dispose the two blocked-by: rows that
named session replays" task, which 7ff50da completed — that file only shrinks,
and finished work is deleted the moment it is finished.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
@coderabbitai

coderabbitaiBot commented Sep 2, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Team

Run ID: aa049e06-ce1d-4253-a6cc-fbf38800dff3


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Links rather than restates: the ticket is the authority, this file just says
enough for a session to pick it up cold and to know not to start it yet — it
overturns the layout CLAUDE.md documents, so its owner-ratified ADR gates
everything after it.
The scoping caution is worth carrying here because it changes how big the work
looks. The draft justified the epic as breaking seven ORM cycles; traversing
every concrete cross-app relation in the app registry finds exactly one
(accounts <-> job, via Staff.default_labour_subtype). The other six are
one-way, and purchasing -> job, accounting -> job and quoting -> company
already point where the target architecture wants them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@corrin
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Access logging and session replay: the two features left behind at the flip - #137

Open
corrin wants to merge 6 commits into
mainfrom
feat/restore-app-usage-logging
Open

Access logging and session replay: the two features left behind at the flip#137
corrin wants to merge 6 commits into
mainfrom
feat/restore-app-usage-logging

Conversation

@corrin

Copy link
Copy Markdown
Owner

Ports the last two features that were left behind at the flip: the per-request
access log, and session replay end to end. Both were recorded as deferred in
docs/rewrite-status.md, and access logging was the first item on the
post-cutover queue.

Access logging

One line per authenticated request on its own access logger, carrying the
X-Session-Replay-Id that joins a request to its recording.

It goes to the console, not v1's rotating access.log. config/settings.py
already states that journald is the sink, and journald gives the rotation and
retention v1 got from ConcurrentRotatingFileHandler — a file handler would
only put a second copy on disk for an operator to find and prune.

Two v1 constructs are deliberately not ported, both verified broken rather than
assumed:

  • v1 read request.user before calling the view. Under ninja auth — which
    sets request.user during operation dispatch, after every middleware — that
    logs nothing for any /api/** request, i.e. for the whole application, while
    a v1-shaped test still passes. v2 reads the principal after get_response.
  • DisallowedHostMiddleware never did anything.process_exception fires
    only for exceptions raised by the view, but DisallowedHost comes out of
    CommonMiddleware.process_request above it. Django was always returning that
    400; only the traceback was ever the complaint. apps/core/logging_filters.py
    keeps the record of the probe and strips its traceback. The test fails without
    the filter — I checked.

v1's JWT re-authentication block went with it (v2 is cookie-authenticated), and
with it a bare except Exception: pass.

Session replay

Chunk payloads go to a private disk root (SESSION_REPLAY_STORAGE_ROOT, which
instance.sh has been provisioning at 700 all along). The rows index the store
rather than being it, so browsing sessions stay out of every pg_dump.

The store is shared, not a second copy. Phone-call recordings already kept a
metadata row plus a payload on a private root, and v1 wrote that logic twice.
apps/core/file_store.py now owns the path-escape guard, the atomic write and
the refuse-to-overwrite, and phone_call_service uses it too (ADR 0039).

Access follows what the business actually had. A staff member may write to
their own recording and read none of them. v1 gated the API at office staff but
hung the page behind a superuser route; a replay is an unredacted video of
somebody's screen, so reads are superuser-only here.

Upload rules are about not losing a session, and are the part v1 iterated on:
a transient failure puts events back at the front of the buffer (behind later
events they would replay out of order), a 409 means the chunk already landed so
the sequence advances and recording continues, and only 401/403/404 discard the
recording.

Three things v1 lacked or got wrong:

  • CompanyDefaults.session_replay_enabled — an off-switch that is not a deploy.
    v1 had none outside a DEV-only E2E flag.
  • The purge is scheduled (01:30 NZT) and now deletes payloads, not just rows.
    Retention is this feature's only privacy control, so it ships with capture.
  • The scrubber truncates recordings. v1 blanked storage_path and sha256,
    which left the unscrubbed payloads on disk and left the purge unable to
    ever find them again.

Also fixed on the way past: src/api/client.ts carried a STUB comment where the
X-Session-Replay-Id header belonged, which is why every AppError persisted
since the port had a replay column it could never fill. And v1's admin page
"destroyed" its player with innerHTML = '', leaking the previous instance's
timers and listeners on every reselect.

Capture stays off under Playwright-over-ngrok — the E2E run is already
bottlenecked on that tunnel. The new spec clears that opt-out for itself.

Operational assets unblocked

Both blocked-by: rows in v1-disposition.md that named session replay are now
ported: the purge beat entry, and pull_prod_files.sh, whose only blocker was
the storage decision. It takes host and instance-user as required arguments
rather than defaulting to MSM production, and validates instance-user as a plain
unix account name — it is interpolated into the remote --rsync-path, the same
escalation pull_prod_backup.sh already guards.

Docs commit authored by another session

80b7d48 is the work of a concurrent session that exited before it could commit;
I reviewed and committed it rather than leave it in a shared worktree. It makes
rewrite-status.md forward-looking and turns cutover-checklist.md into a
record now the flip has run. I verified every redirected comment resolves to
content that really moved before committing it.

Verification

Commit and push tiers green (ruff, mypy strict, import-linter, find-duplicates,
deptry, exported schema, status table, code-quality, delta goldens, frontend
lint/format/boundary/type-check/audit, generated-client-current, server suites,
makemigrations). Python suite green; 613 frontend unit tests green, 6 of them new
and covering the upload rules above.

Not yet run: frontend/tests/e2e/admin/session-replay.spec.ts. It drives the
cross-layer path no unit test on either side can reach — browser capture, chunk
upload, gzip on disk, events endpoint, player mount — plus a second test toggling
session_replay_enabled off. Per CLAUDE.md this slice is not done until that
passes, so please treat this PR as unverified end-to-end until it does.

🤖 Generated with Claude Code

https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7

corrinand others added 5 commits September 2, 2026 17:27
One line per authenticated request on its own `access` logger, carrying the
X-Session-Replay-Id that joins a request to its recording. It goes to the
console, not v1's rotating access.log: journald already rotates, retains and
greps that stream.
Two v1 constructs are deliberately not ported. AccessLoggingMiddleware now
reads the principal AFTER get_response — v1 checked it on the way in and
returned early when anonymous, which under ninja auth (which sets request.user
during operation dispatch, after middleware) would log nothing for any /api/**
request while a v1-shaped test still passed. And DisallowedHostMiddleware never
did anything: process_exception fires only for view exceptions, while
DisallowedHost comes out of CommonMiddleware.process_request above it. Django
was always returning that 400; only the traceback was the complaint, so a
logging filter keeps the record of the probe and drops its traceback.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
Chunk payloads go to a private disk root (SESSION_REPLAY_STORAGE_ROOT, which
instance.sh has been provisioning at 700 all along); the rows index the store
rather than being it, so browsing sessions stay out of every pg_dump.
The store itself is shared, not a second copy. Phone-call recordings already
kept a metadata row plus a payload on a private root, and v1 wrote that logic
twice; apps/core/file_store.py now owns the path-escape guard, the atomic
write and the refuse-to-overwrite, and phone_call_service uses it too.
Access follows what the business actually had: a staff member may write to
their own recording and read none of them. v1 gated the API at office staff
but hung the page behind a superuser route, so reads are superuser-only here —
a replay is an unredacted video of somebody's screen.
Three things v1 lacked or got wrong:
- CompanyDefaults.session_replay_enabled, an off-switch that is not a deploy.
- The purge is scheduled and now deletes payloads, not just rows. Retention is
this feature's only privacy control, so it ships with capture.
- The scrubber truncates recordings. v1 blanked storage_path and sha256, which
left the unscrubbed payloads on disk AND left the purge unable to find them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
rrweb records every authenticated page and flushes batches every ten seconds.
The upload rules are the parts v1 iterated on and they are about not losing a
session: a transient failure puts the events back at the FRONT of the buffer
(behind later events they would replay out of order), a 409 means the chunk
already landed so the sequence advances and recording continues, and only
401/403/404 discard the recording.
X-Session-Replay-Id now leaves the client for real — src/api/client.ts had
carried a STUB comment where the header belonged, which is why every AppError
persisted since the port had a replay column it could never fill.
Capture stays off under Playwright-over-ngrok: the E2E run is already
bottlenecked on that tunnel and recording every spec would add chunk uploads
to it. The new spec clears that opt-out for itself, because the cross-layer
path — browser capture to gzip on disk to the events endpoint to the player —
is the one no unit test on either side can reach.
Two v1 defects not carried over: the admin page destroyed its player by
clearing innerHTML, leaking the previous instance's timers and listeners on
every reselect; and the wire's opaque JSON is now checked against rrweb's
event shape rather than trusted, so an unplayable recording fails visibly
instead of showing a blank frame.
The one docs line is the spec-count row the status-table gate regenerates
because this commit adds a spec file; the rest of that file is another
workstream's and is deliberately untouched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
Both rows named session replay as their blocker, and both blockers are now
answered, so the disposition rule applies: the code arrives with the feature.
pull_prod_files.sh is ported. Its blocker was the storage decision — payloads
live on disk under SESSION_REPLAY_STORAGE_ROOT, so all three destinations
exist and the script has somewhere to put them. Two changes from v1: host and
instance-user are required arguments rather than defaulting to MSM production,
and instance-user is validated as a plain unix account name, because it is
interpolated into the remote --rsync-path and is the same escalation
pull_prod_backup.sh already guards. Destinations are all resolved before the
first byte moves, since a missing root found halfway through leaves a partial
pull that looks complete on the directories it did reach.
purge_old_session_replays_daily is ported: scheduled 01:30 NZT, and deleting
chunk payloads and the recording directory rather than only rows.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
Authored by a concurrent session (update-rewrite-status-postrelease), which
exited before it could commit. I reviewed the diffs and committed them rather
than leave the work uncommitted in a shared worktree; I did not write them.
What it does: rewrite-status.md becomes forward-looking, holding tasks only,
with what shipped moved to rewrite-history.md; cutover-checklist.md becomes
cutover-record.md in all but filename, because the flip ran on 2026-08-29 and
a checklist nobody ticked on the night would lie in a new way; CLAUDE.md now
says work is tracked in both Jira and rewrite-status.md, and points at
release-process.md rather than the cutover file.
What I checked, since I did not author it: every redirected comment resolves
to content that really moved — the DRAFT/WIP divergence, the unweighted
billable_percentage and total_revenue quirks, the public-holiday working-day
split and the quote-event double count are all in rewrite-history.md now; the
two renamed doc targets (development_session.md, release-process.md) exist;
and the test docstring's "a client error IS an AppError" item is really in
rewrite-status.md. The five apps/accounting and two test/spec edits are
comment-only.
One line is mine: I deleted the "Re-dispose the two blocked-by: rows that
named session replays" task, which 7ff50da completed — that file only shrinks,
and finished work is deleted the moment it is finished.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
@coderabbitai

coderabbitaiBot commented Sep 2, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Team

Run ID: aa049e06-ce1d-4253-a6cc-fbf38800dff3


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Links rather than restates: the ticket is the authority, this file just says
enough for a session to pick it up cold and to know not to start it yet — it
overturns the layout CLAUDE.md documents, so its owner-ratified ADR gates
everything after it.
The scoping caution is worth carrying here because it changes how big the work
looks. The draft justified the epic as breaking seven ORM cycles; traversing
every concrete cross-app relation in the app registry finds exactly one
(accounts <-> job, via Staff.default_labour_subtype). The other six are
one-way, and purchasing -> job, accounting -> job and quoting -> company
already point where the target architecture wants them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012KsKjSzg9umSt7ad4NHEg7
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@corrin