Skip to content

Guard the atomic custom_fields merge/delete against a NULL column - #675

Merged
ddon merged 1 commit into
BeamLabEU:mainfrom
timujinne:fix-custom-fields-atomic-merge
Jul 31, 2026
Merged

Guard the atomic custom_fields merge/delete against a NULL column#675
ddon merged 1 commit into
BeamLabEU:mainfrom
timujinne:fix-custom-fields-atomic-merge

Conversation

@timujinne

Copy link
Copy Markdown
Contributor

The custom_fields JSONB column is nullable (V18), and in Postgres both
NULL || jsonb and NULL - key evaluate to NULL. merge_user_custom_fields/3
and delete_user_custom_field/3 build exactly those fragments, so on a user row
whose custom_fields is still NULL the merge silently swallows the additions
(no error, {:ok, user}, nothing written) — the same gap the V30 migration
already guards against with COALESCE for its own atomic merge.

Changes

  • merge_user_custom_fields/3: COALESCE(?, {}::jsonb) || ?
  • delete_user_custom_field/3: COALESCE(?, {}::jsonb) - ?::text — also
    preserves the old read-modify-write path's side effect of normalizing a NULL
    column to {} on any delete
  • Docs: the locale path now writes through the atomic single-key primitives
    (merge_user_custom_fields/3 / delete_user_custom_field/3) with
    ensure_definitions: false, which the moduledoc did not say; and
    set_user_custom_field/2 returns {:error, :not_found} (not a changeset
    error) when the row was deleted concurrently, same contract as the merge it
    delegates to.

Tests

Two new cases in test/integration/users/profile_test.exs: merging into a NULL
column keeps the additions, deleting from a NULL column normalizes it to %{}.
Full file green (37 tests); the suite otherwise matches main.

…s; docs for the locale path's new primitives and set_user_custom_field's :not_found

@timujinnetimujinne left a comment

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GLM Review — PR #675 (COALESCE the atomic custom_fields merge/delete)

Model: glm-5.2 via the z.ai endpoint, reviewer persona (two-stage: spec compliance, then code quality). Read-only pass over git diff upstream/main...fix-custom-fields-atomic-merge, with the SQL semantics checked against a live Postgres.


Stage 1: Spec Compliance

lib/phoenix_kit/users/auth.ex:1704PASS: The merge fragment is correctly rewritten as COALESCE(u.custom_fields, '{}'::jsonb) || type(^additions, :map). COALESCE wraps only the nullable column operand; the ^additions parameter (always a map) is correctly left outside. Verified empirically: NULL::jsonb || '{"survives":"yes"}'NULL (the bug), and COALESCE(NULL,'{}') || '{"survives":"yes"}'{"survives":"yes"} (the fix).

lib/phoenix_kit/users/auth.ex:2080PASS: The delete fragment is correctly rewritten as COALESCE(u.custom_fields, '{}'::jsonb) - ?::text. Verified: NULL::jsonb - 'x'NULL (bug), COALESCE(NULL,'{}') - 'x'{} (fix).

Non-NULL path unchanged — CONFIRMED. Empirically '{"existing":1}'::jsonb || additions and COALESCE('{"existing":1}', '{}') || additions return byte-identical results. The updated_at: ^UtilsDate.utc_now() lines in both clauses are unchanged context in the diff — this PR does not touch them, so the upstream-added timestamp bump is preserved exactly.

lib/phoenix_kit/users/auth.ex:1569-1575 (locale doc) — PASS: Matches code at auth.ex:1597-1603 — set goes through merge_user_custom_fields/3 with ensure_definitions: false; clear goes through delete_user_custom_field/3 (which never registered definitions, so the ensure_definitions opt is irrelevant there). "The first write no longer auto-registers a field definition" is accurate.

lib/phoenix_kit/users/auth.ex:2041-2043 (set_user_custom_field doc) — PASS: Matches code at auth.ex:2050; it delegates to merge_user_custom_fields/3, which returns {:error, :not_found} on {0, _} at auth.ex:1719-1720.

test/integration/users/profile_test.exs:191-207 and :256-272PASS, and the tests prove the claimed behavior (they do not pass for the wrong reason): each forces the column to NULL via update_all(set: [custom_fields: nil]) (Ecto :map serializes nil → SQL NULL, and the column is genuinely nullable per V18 add :custom_fields, :map, null: true). The merge assertion merged.custom_fields == %{"survives" => "yes"} would yield nil (FAIL) without COALESCE, since update_all(... select: u) uses RETURNING on Postgres. Same for the delete assertion == %{}. The update: [...]-in-query + []-updates call shape mirrors the production function at auth.ex:1694-1711.

No other NULL-jsonb write gaps left unfixed. The only application-code atomic jsonb mutations on custom_fields in all of lib/ are the two sites this PR fixes. The remaining custom_fields || %{} hits (auth.ex:1791,1973,1978,1987,2005, custom_fields.ex:382) are Elixir-level map reads with nil-coalescing — not SQL. digest_worker.ex:147 is a read-side ? existence check (NULL → false, the desired filter behavior). Migration V76's custom_fields - 'avatar_file_id' is guarded by WHERE custom_fields ? 'avatar_file_id' (NULL rows excluded) and is a one-time migration anyway.

Spec Verdict: PASS


Stage 2: Code Quality

NITPICK: New tests duplicate the NULL-column setup, and the delete path subtly changes NULL semantics vs. the immediate upstream state

File: test/integration/users/profile_test.exs:191-207 and 256-272; lib/phoenix_kit/users/auth.ex:2077-2080
Problem: Two observations, neither a defect:

  1. Both new tests inline an identical import Ecto.Query + update_all(set: [custom_fields: nil]) block. A private force_custom_fields_nil/1 helper would remove the duplication — though the file's adjacent updated_at tests (lines 175-189, 240-254) inline their own update_all setup the same way, so this matches local convention.
  2. Worth a conscious reviewer sign-off: on the immediately-preceding upstream/main atomic implementation, delete_user_custom_field on a NULL row left it NULL (NULL - key = NULL). This PR changes that to normalize NULL → {}. The choice is deliberate (comment at auth.ex:2078-2079, test at :268-271), restores the pre-atomic-refactor Elixir behavior, and matches the column's declared default: %{} — so it is the right call — but it is a behavior change on the NULL path, not purely a no-op fix.
    Suggestion: Optional: extract the setup into a small helper for DRY. No action needed on item 2 beyond awareness.
    Rationale: %{} is the correct neutral value here (matches column default, matches documented historical behavior, and the merge path can't produce NULL either, so it removes nil-vs-empty ambiguity for readers). Leaving NULL on delete would be more "minimal" but would preserve the very ambiguity the fix targets.

Quality Summary: 0 critical, 0 major, 0 minor, 1 nitpick
Quality Verdict: Ship


Overall Verdict: PASS

The fix is correct, minimal, and empirically verified: COALESCE on both the || merge and the - delete operators resolves the silent NULL-swallow, leaves the non-NULL path byte-identical (so the updated_at bump upstream added in the same clauses is unaffected), the two new tests genuinely fail without the fix, the doc edits accurately describe the code, and no other application-code atomic jsonb write on custom_fields was left unfixed. No action required before merge; the two nitpick observations are informational.

@ddon
ddon merged commit 1e31b8b into BeamLabEU:mainJul 31, 2026
ddon pushed a commit that referenced this pull request Jul 31, 2026
Stop Notifications.Prefs from replacing the whole custom_fields column: it
rebuilt the map from the %User{} the caller held since mount/3, so a Telegram
connect or a locale switch landing mid-session was silently reverted by the
next preferences save. It now writes only its own key through the atomic merge,
which is what ChannelConfig's per-channel key layout always assumed.
Stop the media browser and canvas viewer registering internal UI preferences as
admin-visible custom fields — ensure_definitions_exist/1 registers every key in
the map it is handed, and those five sites passed the whole column.
Add a @SPEC for update_user_locale_preference/2 documenting both error shapes,
correct the Prefs specs to {:error, :not_found}, and add tests for the NULL
column contract, set_user_custom_field/3's :not_found, and sibling-key survival
across a Prefs.merge/2.
Review: dev_docs/pull_requests/2026/675-fix-custom-fields-atomic-merge/CLAUDE_REVIEW.md
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@timujinne
timujinne deleted the fix-custom-fields-atomic-merge branch August 6, 2026 05:55
timujinne added a commit to timujinne/phoenix_kit that referenced this pull request Aug 10, 2026
The GLM reviewer pass on PR BeamLabEU#675 (COALESCE guard for the atomic
custom_fields merge/delete) was left untracked in a second, branch-name
mismatched directory. Filed under the PR's existing directory, whose slug
matches the head branch fix-custom-fields-atomic-merge, per the
one-directory-per-PR convention in CLAUDE.md. That puts it next to the
PR's CLAUDE_REVIEW.md; the two earlier GLM reviews (BeamLabEU#668, BeamLabEU#680) instead
sit in their own slug directories, which is drift worth not repeating.
.pi-subagents/ holds mission JSON and run transcripts written by the
external subagent harness; its reports land outside the repo, so the
directory is scratch and is now ignored rather than committed.
ddon added a commit that referenced this pull request Aug 10, 2026
Add PR #675 GLM review, ignore harness scratch
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@timujinne@ddon