Skip to content

Fix prefixed migrations + tooling for low-privilege multi-schema installs - #629

Closed
mdon wants to merge 5 commits into
BeamLabEU:mainfrom
mdon:prefix-hardening
Closed

Fix prefixed migrations + tooling for low-privilege multi-schema installs#629
mdon wants to merge 5 commits into
BeamLabEU:mainfrom
mdon:prefix-hardening

Conversation

@mdon

@mdonmdon commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

Follow-up to #628, driven by a field report from a hardened multi-schema install (schema pre-created by a DBA and owned by the app role, no CREATE on the database, PG 15+ non-writable public, PgBouncer). The report confirmed #628's index/existence-check fixes and surfaced five more bug families — all reproduced against source and fixed here, plus three adjacent bugs found while verifying.

Migration chain

  • CREATE EXTENSION / CREATE SCHEMA fail on privilege before the IF-NOT-EXISTS short-circuit. All five extension sites (V01 citext, V26/V40 pgcrypto, V111 pg_trgm, UUIDRepair) and V01's schema creation now check pg_extension / information_schema.schemata first and only attempt creation when the object is genuinely missing. A missing extension that the role cannot create raises an operator-facing message listing the three required extensions (also documented in PhoenixKit.Migration's moduledoc for DBAs to pre-provision).
  • V27 dropped the caller's create_schema flag at the Oban handoffOban.Migration.up(prefix: prefix) re-defaults it to true for any non-public prefix and executed the failing CREATE SCHEMA mid-chain (the report's "dies around v27–v31"; queued SQL surfaces at a later flush). V27 now passes create_schema: false — by V27 the schema always exists, V01 owns creation.
  • uuid_generate_v7() was created unqualified — it landed wherever search_path pointed, polluting public on prefixed installs and failing outright where public isn't writable. The function is now created inside the install's schema (all 4 creation sites + UUIDRepair) and all ~89 call sites across 35 files (DEFAULTs, backfill UPDATEs, fragment/1) are schema-qualified. For upgrade chains starting ≥ V40 (which skip the creation sites), Postgres.up/1 re-ensures the function at the prefix before running steps. Existing installs are unaffected: existing columns keep their OID-pinned defaults; the ensure step only creates the function where missing (never REPLACEs a function owned by someone else).
  • The prefix is now validated at the up/down entry points ([a-z_][a-z0-9_]*) — it's interpolated into hundreds of statements, mostly unquoted, so anything needing quoting only ever half-worked.
  • New PhoenixKit.Migrations.Postgres.Helpers centralizes these patterns (qualify_table/2, uuid_v7_call/1, ensure_extension!/1-2, ensure_uuid_v7_function/1-2, validate_prefix!/1) — the per-version hand-rolled helpers were exactly why Fix prefixed migrations: schema-qualified index names + cross-schema existence checks #628's index bug existed in four independent places. Migrating old versions to it is opportunistic; new code should use it.

Tooling

  • mix phoenix_kit.install --prefix X now persists config :phoenix_kit, prefix: "X" (new PhoenixKit.Install.PrefixConfig) — the migration docs always told hosts to add it, but nothing did, so update/status looked at public and reported a prefixed install as "Not installed".
  • update/status/gen.migration resolve the prefix as --prefixconfig :phoenix_kit, :prefix"public".
  • Install.Common no longer fabricates {:current_version, 1} from the mere existence of migration files when the DB has no marker at the prefix — that fallback made mix phoenix_kit.update generate a from-scratch v01→v142 migration into the wrong schema (the report caught it by eye before running it). The not-installed notice now names the resolved prefix and hints at --prefix/the config key.
  • Update-migration generators always emit create_schema: false — updating implies the schema exists. Also fixed: MigrationStrategy hardcoded "public" when version-checking an existing install, and migration_opts/2 silently dropped --create-schema=false (omitting the key let the chain re-default it to true).
  • Repo auto-detection refuses to silently pick among multiple detected Ecto repos (the report's project had a migration-only CoreRepo that got wired into config :phoenix_kit, repo:); it now lists them and requires --repo.

Verification

  • test/integration/prefix_migration_test.exs extended: asserts uuid_generate_v7 lives in the prefixed schema and uuid DEFAULTs are pinned to it. Full chain into a scratch schema passes.
  • Fresh public install from a dropped DB: full v01→v142 clean (extension-creation path exercised).
  • The report's exact repro (section D) now passes end-to-end: role with no superuser and no database CREATE, pre-created schema, pre-provisioned extensions → full v01→v142 chain, marker 142, function + Oban tables in the prefix, zero objects created in public.
  • New unit tests for Helpers (prefix validation, qualification) and PrefixConfig.resolve_prefix/1.
  • mix precommit clean; full suite failures identical to the pre-existing multi-session/sitemap/settings set (verified by stash-rerun on the same base).

Notes for the reviewer

  • The report flags two correct qualified-name sites — uuid_fk_columns.exdrop_uuid_fk_index/4 and V56 drop_uuid_unique_indexes (DROP INDEX schema.name is valid) — deliberately untouched.
  • Known gap, out of scope: runtime code never reads config :phoenix_kit, :prefix — Ecto queries run unprefixed, so prefixed installs still need the DB role's search_path to include the schema (the report's setup relied on this). Documented in AGENTS.md.
  • This branch also carries two earlier local commits: the daisyUI advisory-warning approach (vendored-plugin custody replaced by install/update/doctor version warnings) and an AGENTS.md refresh.

mdon added 5 commits July 12, 2026 20:32
… docs
- Database section gains the prefix-safe migration rules (bare index
names on CREATE, schema-anchored existence checks, late flush()
failures) with the new integration test as the oracle
- New Permissions section covering the allowlist model and the 1.7.182
sub-permissions (dotted keys, sub-implies-base, edit protection,
advisory-lock serialization) - pointer-style, moduledoc is the source
of truth
- Tests section no longer claims test/support/postgres/migrations/
exists (retired 2026-05-05 for ensure_current/2)
- Component-coverage TODO drops checkbox from the remaining gaps
(checkbox_test.exs exists)
…d daisyUI
daisyUI 5.0.x reserves the modal scrollbar gutter unconditionally while a
modal/drawer is open. Core countered it with an unlayered
:root:has(.modal-open, ...) { scrollbar-gutter: auto } rule (admin
LayoutWrapper + layouts/root.html.heex) plus an inline PkDialog override —
killing the phantom right-edge strip on non-scrolling pages, but defeating
daisyUI's anti-jump reservation on pages that DO scroll: classic-scrollbar
users saw content reflow ~15px and the scrollbar pop in/out around every
modal open/close (user-reported as 'clicked cancel and a scroll bar showed
up' on the dashboards create modal).
daisyUI >= 5.1 reserves the gutter only when the page really has a scrollbar
(rootscrollgutter.css), which handles both page types correctly — so the
compensations are removed rather than made smarter:
- both CSS counter-rules deleted
- PkDialog's inline scrollbar-gutter override and its open-refcount
machinery (window._PkDialogOpenCount) deleted from phoenix_kit.js
The vendored daisyUI stays HOST-OWNED (assets/vendor/daisyui.js, scaffolded
by phx.new) — core does not manage or replace it. Instead the new
PhoenixKit.Install.DaisyUI declares the designed-for minimum (5.6.0; verified
against 5.6.17) and warns when a host is behind:
- mix phoenix_kit.install: Igniter warning with upgrade instructions
- mix phoenix_kit.update: shell warning, next to its CSS/JS refresh steps
- mix phoenix_kit.doctor: new 'daisyUI Version' check
(pass / outdated / unversioned / missing states)
Hosts still on daisyUI < 5.1 get daisyUI's stock old modal behavior (the
gutter strip while a modal is open on non-scrolling pages) plus the warning;
the cure is updating the two vendored files, not core CSS. Do not re-add
scrollbar-gutter overrides in layouts, PkDialog, or modules.
Verified in phoenix_kit_parent (vendored daisyUI updated to 5.6.17) with
native classic scrollbars: on the scrolling dashboards list, body width and
container position stay pixel-identical through the whole open -> cancel
cycle; on the viewport-locked builder, no gutter is reserved and the dialog
covers the full viewport width. Unit tests cover the version parsing and
comparison.
…ns, guarded DDL
Field report (2026-07-12): a hardened multi-schema install — schema
pre-created by a DBA and owned by the app role, no CREATE on the
database, PG15+ non-writable public — failed at three points the
superuser-driven test suite never exercises:
- CREATE EXTENSION IF NOT EXISTS and CREATE SCHEMA IF NOT EXISTS check
the CREATE privilege BEFORE the IF-NOT-EXISTS short-circuit, so both
fail for low-privilege roles even when the object already exists.
V01/V26/V40/V111/UUIDRepair now check pg_extension /
information_schema.schemata first and only attempt creation when the
object is genuinely missing (raising an operator-facing message
listing citext/pgcrypto/pg_trgm when missing and uncreatable).
- V27 dropped the caller's create_schema flag at the Oban handoff:
Oban.Migration.up defaults it to true for any non-public prefix and
executed the failing CREATE SCHEMA mid-chain. By V27 the schema always
exists (V01 owns creation), so it now passes create_schema: false.
- uuid_generate_v7() was created unqualified — it landed wherever
search_path pointed, polluting public on prefixed installs and
failing outright where public isn't writable. The function is now
created inside the install's schema and all 89 call sites (DEFAULTs,
backfill UPDATEs, fragments) are schema-qualified. Postgres.up/1
re-ensures the function at the prefix for upgrade chains starting
>= V40, which skip the V40/V56/V61/V63 creation sites.
New PhoenixKit.Migrations.Postgres.Helpers centralizes these patterns
(qualify_table/2, uuid_v7_call/1, ensure_extension!/1-2,
ensure_uuid_v7_function/1-2) plus validate_prefix!/1 — the prefix is
interpolated into SQL mostly unquoted, so up/down now reject anything
outside [a-z_][a-z0-9_]*.
The prefix integration test additionally asserts the function lives in
the prefixed schema and that uuid DEFAULTs are pinned to it. Verified
end-to-end against the report's repro recipe: full v01->v142 chain as a
role with no superuser and no database CREATE into a pre-created
schema — zero objects created in public.
Same field report, tooling half — a prefixed install was invisible to
every mix task because nothing persisted or resolved the prefix:
- mix phoenix_kit.install --prefix X now writes
config :phoenix_kit, prefix: "X" into config.exs/test.exs (as the
migration docs already instructed hosts to do by hand), via the new
PhoenixKit.Install.PrefixConfig.
- update/status/gen.migration resolve the prefix as
--prefix option -> config :phoenix_kit, :prefix -> "public"
(PrefixConfig.resolve_prefix/1), so version detection stops silently
looking at public and reporting a prefixed install as missing.
- Install.Common no longer fabricates {:current_version, 1} from the
mere existence of migration FILES when the DB has no marker at the
prefix — that fallback made mix phoenix_kit.update generate a
from-scratch v01->vN migration into the wrong schema. The updater's
not-installed notice now names the resolved prefix and hints at
--prefix / the config key.
- Update-migration generators (update task + gen.migration) always emit
create_schema: false — updating implies the schema exists, and the
flag previously re-defaulted to true for prefixed installs.
- MigrationStrategy no longer hardcodes "public" when checking an
existing install's version, and migration_opts/2 emits create_schema
explicitly for non-public prefixes (omitting it silently discarded
--create-schema=false through the chain's put_new default).
- Repo auto-detection refuses to pick among multiple detected Ecto
repos (it silently wired a migration-only repo in the field report);
it now lists them and asks for --repo.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@mdon