From 3cd45094e3dc18612fef06d7e0c05c518d8a29e9 Mon Sep 17 00:00:00 2001 From: Lee Hannigan Date: Wed, 19 Aug 2026 18:21:17 +0000 Subject: [PATCH 1/6] docs(docker): publish a Docker Hub Overview, and a step to keep it current The Docker Hub repository has 446 pulls and a blank Overview, and a blank short description with it. Neither field is set by anything: `docker push` and the `skopeo copy` that promote-image uses both write images, never repository metadata, and nothing in CI touches the Hub API. GHCR looks documented for the same image because it derives its package description from the OCI labels the Dockerfile already sets (`org.opencontainers.image.title` / `.description`). Docker Hub ignores those labels for the Overview entirely, which is why one registry reads as maintained and the other reads as abandoned. `docker/README.md` was not reusable as-is. It is contributor-facing: how to generate notices, build with `--build-arg`, run the four-role Compose stack, run the smoke test. Someone pulling the image wants the tag policy, the runtime contract, the fact that PostgreSQL is not bundled, and how to verify the signature. So this adds a consumer-facing file rather than pointing the Hub at the existing one. Content is sourced from `docker/dockerhub-overview.md` so the published Overview is reviewed like any other content and cannot drift from the repository. It carries the trademark notice from the README verbatim, since the Hub page is a public front door and previously carried no notice at all, and links to extenddb.org. The workflow is dispatch-only and gated on the `dockerhub` environment, because it uses the same credential that can push images. Deliberately not wired into promote-image: the Overview is version-neutral, so a wording fix should not require a release and a release should not silently rewrite it. It also keeps this clear of #283, which is already changing that file. It verifies rather than assumes: after the PATCH it re-reads the repository anonymously and diffs the published Overview against the file, so a silent no-op or a partial write fails the run instead of looking green. Verification: - YAML parses; the `jq` payload construction was exercised locally without credentials (4945 characters, against Docker Hub's 25000 cap; short description 73 characters against its 100 cap, both asserted in the job). - Every link in the Overview was fetched: extenddb.org, the repository, the releases page, and the four linked docs all return 200. - Runtime facts were taken from the Dockerfile rather than restated from memory: port 18443, state at /var/lib/extenddb, UID/GID 10001:10001, tini entrypoint, and a healthcheck that is liveness only. The Hub API path itself cannot be exercised from a PR, since the credential exists only inside the gated job. The read-back assertion is there so the first dispatch proves it end to end rather than reporting success blindly. --- .github/workflows/dockerhub-description.yml | 122 ++++++++++++++++++++ docker/dockerhub-overview.md | 93 +++++++++++++++ 2 files changed, 215 insertions(+) create mode 100644 .github/workflows/dockerhub-description.yml create mode 100644 docker/dockerhub-overview.md diff --git a/.github/workflows/dockerhub-description.yml b/.github/workflows/dockerhub-description.yml new file mode 100644 index 00000000..5ac47296 --- /dev/null +++ b/.github/workflows/dockerhub-description.yml @@ -0,0 +1,122 @@ +# Copyright 2026 ExtendDB contributors +# SPDX-License-Identifier: Apache-2.0 +# +# Publish the Docker Hub repository Overview from a file in this repository. +# +# Neither `docker push` nor the `skopeo copy` that promote-image uses touches +# repository metadata: the Overview lives on the Docker Hub repository object, +# not on any image. GHCR derives its description from the OCI labels the +# Dockerfile already sets, so the same image reads as documented there and blank +# on Docker Hub. That is why this exists as its own step rather than falling out +# of a push. +# +# Sourced from `docker/dockerhub-overview.md` so the published Overview is +# reviewed like any other content and cannot drift from the repository. +# +# Dispatch-only and gated on the `dockerhub` environment, because it uses the +# same credential that can push images. Deliberately NOT wired into +# promote-image: the Overview is version-neutral, so a wording fix should not +# require a release, and a release should not silently rewrite it. + +name: Docker Hub description + +on: + workflow_dispatch: + +permissions: + contents: read + +concurrency: + group: dockerhub-description-extenddb-postgres + cancel-in-progress: false + +env: + HUB_NAMESPACE: extenddb + HUB_REPO: extenddb-postgres + OVERVIEW_FILE: docker/dockerhub-overview.md + # Docker Hub caps the short description at 100 characters. + SHORT_DESCRIPTION: DynamoDB-compatible API adapter backed by your own PostgreSQL 14+ database + +jobs: + publish-description: + runs-on: ubuntu-latest + timeout-minutes: 10 + environment: dockerhub + steps: + - name: Check out main + uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 + + - name: The Overview file must exist and be non-trivial + run: | + set -euo pipefail + if [[ ! -s "$OVERVIEW_FILE" ]]; then + echo "::error::$OVERVIEW_FILE is missing or empty" + exit 1 + fi + BYTES=$(wc -c < "$OVERVIEW_FILE") + # Docker Hub caps full_description at 25000 characters. + if (( BYTES > 25000 )); then + echo "::error::$OVERVIEW_FILE is ${BYTES} bytes; Docker Hub caps the Overview at 25000" + exit 1 + fi + if (( ${#SHORT_DESCRIPTION} > 100 )); then + echo "::error::SHORT_DESCRIPTION is ${#SHORT_DESCRIPTION} characters; Docker Hub caps it at 100" + exit 1 + fi + echo "::notice::Overview is ${BYTES} bytes" + + - name: Publish the Overview + env: + DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }} + DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }} + run: | + set -euo pipefail + + # Hub API v2 wants a session token, then a PATCH on the repository. + TOKEN=$(jq -n --arg u "$DOCKERHUB_USERNAME" --arg p "$DOCKERHUB_TOKEN" \ + '{username:$u, password:$p}' \ + | curl -sS --fail-with-body -X POST \ + -H 'Content-Type: application/json' \ + --data @- \ + https://hub.docker.com/v2/users/login/ \ + | jq -r '.token') + if [[ -z "$TOKEN" || "$TOKEN" == "null" ]]; then + echo "::error::could not obtain a Docker Hub session token" + exit 1 + fi + + jq -n \ + --rawfile full "$OVERVIEW_FILE" \ + --arg short "$SHORT_DESCRIPTION" \ + '{full_description:$full, description:$short}' \ + | curl -sS --fail-with-body -X PATCH \ + -H "Authorization: JWT ${TOKEN}" \ + -H 'Content-Type: application/json' \ + --data @- \ + "https://hub.docker.com/v2/repositories/${HUB_NAMESPACE}/${HUB_REPO}/" \ + > /dev/null + + - name: Verify it took effect + run: | + set -euo pipefail + # Read back anonymously: proves the Overview is what an unauthenticated + # visitor now sees, rather than trusting the write's response. + GOT=$(curl -sS --fail-with-body \ + "https://hub.docker.com/v2/repositories/${HUB_NAMESPACE}/${HUB_REPO}/") + GOT_FULL=$(jq -r '.full_description // ""' <<< "$GOT") + GOT_SHORT=$(jq -r '.description // ""' <<< "$GOT") + + if [[ -z "$GOT_FULL" ]]; then + echo "::error::Overview is still empty after the PATCH" + exit 1 + fi + if ! diff -q <(printf '%s' "$GOT_FULL") <(cat "$OVERVIEW_FILE") >/dev/null; then + echo "::error::published Overview does not match ${OVERVIEW_FILE}" + diff <(printf '%s' "$GOT_FULL") "$OVERVIEW_FILE" | head -20 || true + exit 1 + fi + if [[ "$GOT_SHORT" != "$SHORT_DESCRIPTION" ]]; then + echo "::error::short description is '${GOT_SHORT}', expected '${SHORT_DESCRIPTION}'" + exit 1 + fi + echo "::notice::Overview published and verified anonymously" diff --git a/docker/dockerhub-overview.md b/docker/dockerhub-overview.md new file mode 100644 index 00000000..16b320fc --- /dev/null +++ b/docker/dockerhub-overview.md @@ -0,0 +1,93 @@ +# ExtendDB (PostgreSQL backend) + +**The DynamoDB API, everywhere you run code.** — [extenddb.org](https://extenddb.org) + +> ExtendDB is an independent open source project managed by Amazon Web Services. It is not Amazon DynamoDB and does not contain any DynamoDB source code. "DynamoDB" is a trademark of Amazon.com, Inc. ExtendDB is a clean-room implementation that speaks the DynamoDB wire protocol. Behavioral differences from the real service are documented in [Differences from DynamoDB](https://github.com/ExtendDB/extenddb/blob/main/docs/differences-from-dynamodb.md). + +ExtendDB speaks the DynamoDB wire protocol, so any AWS SDK, CLI, or tool that works with DynamoDB works with ExtendDB unchanged. Point your client at the ExtendDB endpoint and nothing else about your code changes. + +This image is an **adapter, not a database**. It contains the ExtendDB server compiled with the PostgreSQL backend. It does **not** contain a PostgreSQL server or data directory: bring your own PostgreSQL 14 or newer, whether that is RDS, Aurora PostgreSQL, or self-managed. + +## Tags + +| Tag | What it is | +| --- | --- | +| `X.Y.Z` | A released version. Immutable: a version tag is never overwritten. | +| `latest` | The highest released version. Only ever moves forward. | +| `sha-<40-char-commit>` | A build candidate, published before promotion. Not a release; pin a version instead. | + +Both `linux/amd64` and `linux/arm64` are published as a single multi-architecture index, so `docker pull` selects the right platform automatically. + +For production, pin `X.Y.Z` or a digest rather than `latest`. + +## Quick start + +For local development the fastest path is the Compose stack in the repository, which brings up PostgreSQL alongside ExtendDB and runs the one-time bootstrap for you. See [Local development with Docker Compose](https://github.com/ExtendDB/extenddb/blob/main/docs/local-dev-docker-compose.md). + +Once running, use any DynamoDB client against the endpoint: + +```bash +aws dynamodb list-tables \ + --endpoint-url https://127.0.0.1:18443 \ + --region us-east-1 +``` + +## Running it yourself + +The image serves three explicit operations. Do not use the Compose bootstrap behaviour as a production control plane. + +**1. Bootstrap once**, as a single Job holding the PostgreSQL bootstrap credential. Inject `EXTENDDB_PG_PASSWORD` and `EXTENDDB_APP_PASSWORD` from your orchestrator's secret store rather than passing them as arguments: + +```bash +extenddb init \ + --config /var/lib/extenddb/extenddb.toml \ + --backend postgres \ + --pg-host --pg-port 5432 \ + --pg-user --extenddb-user \ + --bind-addr 0.0.0.0 --tls-san +``` + +**2. On upgrade**, back up PostgreSQL, scale down incompatible serving versions, then run one migration Job: + +```bash +extenddb migrate --config /var/lib/extenddb/extenddb.toml --yes --pg-user +``` + +The generated config stores the application connection but not the bootstrap password, so the migration Job receives that elevated credential separately. The serving container never gets it. + +**3. Steady state** is the image's default command: + +```bash +extenddb serve --config /var/lib/extenddb/extenddb.toml --foreground +``` + +## Runtime contract + +| | | +| --- | --- | +| Port | `18443` (HTTPS) | +| State volume | `/var/lib/extenddb` — must be writable; holds config and TLS material | +| User | `10001:10001`, non-root | +| Entrypoint | `tini`, so `SIGTERM` reaches the server and shutdown is graceful | +| Healthcheck | `extenddb healthcheck` — liveness only, it does not probe PostgreSQL readiness | + +Supports a read-only root filesystem. Drop all capabilities, keep the runtime's default seccomp profile, and set CPU and memory limits in your orchestrator. + +TLS is on by default with a generated self-signed certificate, replaceable with a CA-signed one. The Compose stack's certificate and passwords are development defaults and must not be used anywhere shared. + +## Verifying the image + +Every published image is signed with ExtendDB's release key, an ECDSA P-256 key held in AWS KMS. The public key is attached to each [GitHub release](https://github.com/ExtendDB/extenddb/releases) as `extenddb-signing.pub.pem`, and each release's notes carry the exact `cosign verify` invocation for that release. Verify against the digest you are deploying. + +## Also available on GHCR + +The same images, by digest, are mirrored to `ghcr.io/extenddb/extenddb-postgres`. + +## Links + +- Project site: [extenddb.org](https://extenddb.org) +- Source, issues and discussions: [github.com/ExtendDB/extenddb](https://github.com/ExtendDB/extenddb) +- [Getting started](https://github.com/ExtendDB/extenddb/blob/main/docs/getting-started.md) +- [Differences from DynamoDB](https://github.com/ExtendDB/extenddb/blob/main/docs/differences-from-dynamodb.md) +- [Troubleshooting](https://github.com/ExtendDB/extenddb/blob/main/docs/troubleshooting.md) +- Licensed under Apache-2.0 From a656d68fc703e762c1b8ff5ad22a82c6909d350f Mon Sep 17 00:00:00 2001 From: Lee Hannigan Date: Wed, 19 Aug 2026 18:28:47 +0000 Subject: [PATCH 2/6] docs(docker): add the dev image Overview, publish either by dispatch Covers `extenddb/extenddb-dev` as well as the Postgres image, and turns the workflow into a per-image dispatch rather than one hardcoded repository. The dev image needs a different page, not a trimmed one. Its whole value is that it is zero-config, so the Overview leads with the one-line `docker run` and the SDK call against it, then the two storage modes. It also has to carry warnings the production page does not: plain HTTP, open authorization, loopback-only publishing, and no real data. Those are stated up front rather than in a limitations footnote, because someone skimming a registry page and copying the run command is the exact person who needs them. Both use the documented example credential verbatim, since the server seeds it and prints it, and secret scanners recognise it as an example. One image per dispatch rather than a matrix over both: a Docker Hub repository that does not exist yet should fail loudly for that image alone, not fail a run that also had valid work to do for the other. Hence the explicit repository-exists gate, which returns a clear "create the repository first" error instead of an opaque PATCH failure. Ordering, worth stating because one link depends on it: the dev Overview links `docs/dev-image.md`, which currently 404s on main because it lands with #281. That is safe rather than a broken link in production, because the dev Overview cannot be published until `extenddb/extenddb-dev` exists on Docker Hub, and that repository is created as part of shipping #281. The repository-exists gate enforces the ordering rather than relying on anyone remembering it. Verification: - YAML parses; the choice input resolves to both images. - Size caps asserted in the job and checked here: Postgres Overview 4945 characters and Dev 4742, against the 25000 cap; short descriptions 74 and 89 against the 100 cap. - Every link in the dev Overview was fetched. All 200 except `docs/dev-image.md`, explained above. - Dev facts were read from `Dockerfile.dev` and `docs/dev-image.md` on the #281 branch rather than restated: uid 65532, distroless cc-debian12 nonroot with no shell, port 18443 over plain HTTP, state at /var/lib/extenddb, `EXTENDDB__STORAGE__SQLITE__PATH=:memory:` for the ephemeral mode, and a built-in healthcheck. --- .github/workflows/dockerhub-description.yml | 115 +++++++++++++++----- docker/dockerhub-overview-dev.md | 89 +++++++++++++++ 2 files changed, 176 insertions(+), 28 deletions(-) create mode 100644 docker/dockerhub-overview-dev.md diff --git a/.github/workflows/dockerhub-description.yml b/.github/workflows/dockerhub-description.yml index 5ac47296..1fd9befb 100644 --- a/.github/workflows/dockerhub-description.yml +++ b/.github/workflows/dockerhub-description.yml @@ -1,41 +1,49 @@ # Copyright 2026 ExtendDB contributors # SPDX-License-Identifier: Apache-2.0 # -# Publish the Docker Hub repository Overview from a file in this repository. +# Publish a Docker Hub repository Overview from a file in this repository. # # Neither `docker push` nor the `skopeo copy` that promote-image uses touches # repository metadata: the Overview lives on the Docker Hub repository object, # not on any image. GHCR derives its description from the OCI labels the -# Dockerfile already sets, so the same image reads as documented there and blank +# Dockerfiles already set, so the same image reads as documented there and blank # on Docker Hub. That is why this exists as its own step rather than falling out # of a push. # -# Sourced from `docker/dockerhub-overview.md` so the published Overview is +# Sourced from `docker/dockerhub-overview*.md` so the published Overview is # reviewed like any other content and cannot drift from the repository. # # Dispatch-only and gated on the `dockerhub` environment, because it uses the # same credential that can push images. Deliberately NOT wired into # promote-image: the Overview is version-neutral, so a wording fix should not # require a release, and a release should not silently rewrite it. +# +# One image per dispatch rather than a matrix over both, because a Docker Hub +# repository that does not exist yet must fail loudly for that image alone +# instead of failing a run that also had work to do for the other. name: Docker Hub description on: workflow_dispatch: + inputs: + image: + description: 'Which Docker Hub repository to publish the Overview for' + required: true + type: choice + options: + - extenddb-postgres + - extenddb-dev permissions: contents: read concurrency: - group: dockerhub-description-extenddb-postgres + group: dockerhub-description-${{ inputs.image }} cancel-in-progress: false env: HUB_NAMESPACE: extenddb - HUB_REPO: extenddb-postgres - OVERVIEW_FILE: docker/dockerhub-overview.md - # Docker Hub caps the short description at 100 characters. - SHORT_DESCRIPTION: DynamoDB-compatible API adapter backed by your own PostgreSQL 14+ database jobs: publish-description: @@ -46,27 +54,76 @@ jobs: - name: Check out main uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 - - name: The Overview file must exist and be non-trivial + - name: Resolve the Overview source for this image + id: src + env: + IMAGE: ${{ inputs.image }} + run: | + set -euo pipefail + # Docker Hub caps the short description at 100 characters; asserted below. + case "$IMAGE" in + extenddb-postgres) + FILE='docker/dockerhub-overview.md' + SHORT='DynamoDB-compatible API adapter backed by your own PostgreSQL 14+ database' + ;; + extenddb-dev) + FILE='docker/dockerhub-overview-dev.md' + SHORT='DynamoDB-compatible API over SQLite for local dev and CI. Plain HTTP, not for production.' + ;; + *) + echo "::error::unknown image '$IMAGE'" + exit 1 + ;; + esac + echo "file=$FILE" >> "$GITHUB_OUTPUT" + echo "short=$SHORT" >> "$GITHUB_OUTPUT" + echo "::notice::publishing $FILE to ${IMAGE}" + + - name: The Overview file must exist and be within Docker Hub's limits + env: + FILE: ${{ steps.src.outputs.file }} + SHORT: ${{ steps.src.outputs.short }} run: | set -euo pipefail - if [[ ! -s "$OVERVIEW_FILE" ]]; then - echo "::error::$OVERVIEW_FILE is missing or empty" + if [[ ! -s "$FILE" ]]; then + echo "::error::$FILE is missing or empty" exit 1 fi - BYTES=$(wc -c < "$OVERVIEW_FILE") + BYTES=$(wc -c < "$FILE") # Docker Hub caps full_description at 25000 characters. if (( BYTES > 25000 )); then - echo "::error::$OVERVIEW_FILE is ${BYTES} bytes; Docker Hub caps the Overview at 25000" + echo "::error::$FILE is ${BYTES} bytes; Docker Hub caps the Overview at 25000" exit 1 fi - if (( ${#SHORT_DESCRIPTION} > 100 )); then - echo "::error::SHORT_DESCRIPTION is ${#SHORT_DESCRIPTION} characters; Docker Hub caps it at 100" + if (( ${#SHORT} > 100 )); then + echo "::error::short description is ${#SHORT} characters; Docker Hub caps it at 100" + exit 1 + fi + echo "::notice::Overview is ${BYTES} bytes, short description ${#SHORT} characters" + + - name: The Docker Hub repository must already exist + env: + IMAGE: ${{ inputs.image }} + run: | + set -euo pipefail + # A 404 here means the repository has not been created yet. Say so + # plainly rather than letting the PATCH fail with an opaque error. + CODE=$(curl -sS -o /dev/null -w '%{http_code}' \ + "https://hub.docker.com/v2/repositories/${HUB_NAMESPACE}/${IMAGE}/") + if [[ "$CODE" == "404" ]]; then + echo "::error::docker.io/${HUB_NAMESPACE}/${IMAGE} does not exist. Create the repository on Docker Hub first." + exit 1 + fi + if [[ "$CODE" != "200" ]]; then + echo "::error::unexpected HTTP ${CODE} querying docker.io/${HUB_NAMESPACE}/${IMAGE}" exit 1 fi - echo "::notice::Overview is ${BYTES} bytes" - name: Publish the Overview env: + IMAGE: ${{ inputs.image }} + FILE: ${{ steps.src.outputs.file }} + SHORT: ${{ steps.src.outputs.short }} DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }} DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }} run: | @@ -85,24 +142,26 @@ jobs: exit 1 fi - jq -n \ - --rawfile full "$OVERVIEW_FILE" \ - --arg short "$SHORT_DESCRIPTION" \ + jq -n --rawfile full "$FILE" --arg short "$SHORT" \ '{full_description:$full, description:$short}' \ | curl -sS --fail-with-body -X PATCH \ -H "Authorization: JWT ${TOKEN}" \ -H 'Content-Type: application/json' \ --data @- \ - "https://hub.docker.com/v2/repositories/${HUB_NAMESPACE}/${HUB_REPO}/" \ + "https://hub.docker.com/v2/repositories/${HUB_NAMESPACE}/${IMAGE}/" \ > /dev/null - name: Verify it took effect + env: + IMAGE: ${{ inputs.image }} + FILE: ${{ steps.src.outputs.file }} + SHORT: ${{ steps.src.outputs.short }} run: | set -euo pipefail # Read back anonymously: proves the Overview is what an unauthenticated - # visitor now sees, rather than trusting the write's response. + # visitor now sees, rather than trusting the write's own response. GOT=$(curl -sS --fail-with-body \ - "https://hub.docker.com/v2/repositories/${HUB_NAMESPACE}/${HUB_REPO}/") + "https://hub.docker.com/v2/repositories/${HUB_NAMESPACE}/${IMAGE}/") GOT_FULL=$(jq -r '.full_description // ""' <<< "$GOT") GOT_SHORT=$(jq -r '.description // ""' <<< "$GOT") @@ -110,13 +169,13 @@ jobs: echo "::error::Overview is still empty after the PATCH" exit 1 fi - if ! diff -q <(printf '%s' "$GOT_FULL") <(cat "$OVERVIEW_FILE") >/dev/null; then - echo "::error::published Overview does not match ${OVERVIEW_FILE}" - diff <(printf '%s' "$GOT_FULL") "$OVERVIEW_FILE" | head -20 || true + if ! diff -q <(printf '%s' "$GOT_FULL") <(cat "$FILE") >/dev/null; then + echo "::error::published Overview does not match ${FILE}" + diff <(printf '%s' "$GOT_FULL") "$FILE" | head -20 || true exit 1 fi - if [[ "$GOT_SHORT" != "$SHORT_DESCRIPTION" ]]; then - echo "::error::short description is '${GOT_SHORT}', expected '${SHORT_DESCRIPTION}'" + if [[ "$GOT_SHORT" != "$SHORT" ]]; then + echo "::error::short description is '${GOT_SHORT}', expected '${SHORT}'" exit 1 fi - echo "::notice::Overview published and verified anonymously" + echo "::notice::Overview for ${IMAGE} published and verified anonymously" diff --git a/docker/dockerhub-overview-dev.md b/docker/dockerhub-overview-dev.md new file mode 100644 index 00000000..5161a172 --- /dev/null +++ b/docker/dockerhub-overview-dev.md @@ -0,0 +1,89 @@ +# ExtendDB dev (SQLite) + +**The DynamoDB API, everywhere you run code.** — [extenddb.org](https://extenddb.org) + +> ExtendDB is an independent open source project managed by Amazon Web Services. It is not Amazon DynamoDB and does not contain any DynamoDB source code. "DynamoDB" is a trademark of Amazon.com, Inc. ExtendDB is a clean-room implementation that speaks the DynamoDB wire protocol. Behavioral differences from the real service are documented in [Differences from DynamoDB](https://github.com/ExtendDB/extenddb/blob/main/docs/differences-from-dynamodb.md). + +A single container that speaks the DynamoDB wire protocol over SQLite, for **local development and CI**. No external database, no init step, no certificate to trust. + +> **Not for production, and not for shared networks.** This image serves plain HTTP with open authorization: whoever holds the credential can do everything. Always publish it to loopback only. For a durable, TLS-terminated deployment use [`extenddb/extenddb-postgres`](https://hub.docker.com/r/extenddb/extenddb-postgres). + +## Quick start + +```bash +docker run -d -p 127.0.0.1:18443:18443 -v extenddb:/var/lib/extenddb \ + extenddb/extenddb-dev +``` + +Then point any AWS SDK, CLI, or tool at it, unchanged: + +```bash +AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE \ +AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY \ +aws dynamodb list-tables \ + --region us-east-1 --endpoint-url http://127.0.0.1:18443 +``` + +That is AWS's documented example key pair. The server seeds it at startup and prints it in the logs. Secret scanners recognise it as an example credential, so it is safe in test fixtures. Pass `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` to the container to use a different one. + +## Storage modes + +One image, mode chosen at run time. + +**File-backed (default).** The database is `/var/lib/extenddb/extenddb.sqlite`. Mount a volume there and data survives restarts and upgrades: + +```bash +docker run -d -p 127.0.0.1:18443:18443 -v extenddb:/var/lib/extenddb \ + extenddb/extenddb-dev +``` + +**In-memory.** Everything lives in process memory and vanishes when the container stops, which is what you want for a suite that needs a pristine database per run: + +```bash +docker run -d -p 127.0.0.1:18443:18443 \ + -e EXTENDDB__STORAGE__SQLITE__PATH=:memory: \ + extenddb/extenddb-dev +``` + +## Why loopback matters + +The container binds `0.0.0.0` internally, which is required for port publishing to work at all. Containment is therefore the publish flag, not the bind address: keep the host side on `127.0.0.1`, as every example above does. Do not publish this port on a routable or shared interface, and do not put real data in it. + +## Tags + +| Tag | What it is | +| --- | --- | +| `X.Y.Z` | A released version. Immutable: a version tag is never overwritten. | +| `latest` | The highest released version. Only ever moves forward. | + +Both `linux/amd64` and `linux/arm64` are published as a single multi-architecture index, so `docker pull` selects the right platform automatically. + +## Runtime contract + +| | | +| --- | --- | +| Endpoint | `http://127.0.0.1:18443` — plain HTTP, no TLS | +| State volume | `/var/lib/extenddb` (file mode); omit it for a throwaway container | +| User | `65532:65532`, non-root | +| Healthcheck | built in, so `docker ps` and Compose `depends_on: condition: service_healthy` work with no extra wiring | + +The runtime base is distroless (`gcr.io/distroless/cc-debian12:nonroot`): no shell and no package manager, so `docker exec` into it is not possible by design. Licence notices for this image's exact dependency set ship at `/usr/share/doc/extenddb/SOFTWARE-LICENSE-NOTICES.html`. + +## Limitations + +- Single node, single process. Throughput settings are accepted and tracked but not enforced as capacity. +- No TLS and no access control, as above. + +## Verifying the image + +Signed with the same ExtendDB release key as the production image, an ECDSA P-256 key held in AWS KMS. The public key is attached to each [GitHub release](https://github.com/ExtendDB/extenddb/releases) as `extenddb-signing.pub.pem`, and each release's notes carry the exact `cosign verify` invocation for that release. + +## Links + +- Project site: [extenddb.org](https://extenddb.org) +- Source, issues and discussions: [github.com/ExtendDB/extenddb](https://github.com/ExtendDB/extenddb) +- [The `extenddb-dev` image](https://github.com/ExtendDB/extenddb/blob/main/docs/dev-image.md) +- [Getting started](https://github.com/ExtendDB/extenddb/blob/main/docs/getting-started.md) +- [Differences from DynamoDB](https://github.com/ExtendDB/extenddb/blob/main/docs/differences-from-dynamodb.md) +- Production image: [`extenddb/extenddb-postgres`](https://hub.docker.com/r/extenddb/extenddb-postgres) +- Licensed under Apache-2.0 From db1435b4fa920cbf164a38951f0e7ed92e10db34 Mon Sep 17 00:00:00 2001 From: Lee Hannigan Date: Wed, 19 Aug 2026 18:47:16 +0000 Subject: [PATCH 3/6] docs(docker): rewrite the Overviews as product pages, gate on link health MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The first drafts read like runbooks: tag-policy tables, a runtime-contract table, verification instructions. That is reference material, and it belongs in the docs these pages link to, not on a registry front page. Reworked both to the shape a reader of a Docker Hub page actually gets value from: what it is in one sentence, why you would use it, then one command that works. Structure now mirrors the convention such pages follow — definition, benefits, getting started on Docker, where to learn more, closing note. Benefits are stated as benefits rather than as properties: works with your existing DynamoDB API calls, no internet connection needed, no provisioned throughput, storage or data transfer costs. Postgres went from 641 to 456 words, dev from 609 to 347. Both are shorter than what they replace and say more of what a reader came for. The tables are gone; tag policy and signature verification survive as two short paragraphs because pinning and provenance are decisions a reader makes on this page. Two things stay that the pages we are modelling do not need. The trademark notice, which is now a closing Note rather than a banner above the fold: it reads as a standard legal footer there instead of as the first thing the page says about itself, while remaining complete and unmissable. And on the dev page, the plain-HTTP and open-authorization warning, kept in the Note with the loopback instruction, because someone copying a run command off a registry page is precisely who needs it. It is no longer three warnings before the reader has learned what the image is. Also drops a link the first draft would have shipped broken: the Postgres page pointed at `hub.docker.com/r/extenddb/extenddb-dev`, which does not exist yet. The Postgres Overview is publishable today, so that would have been a dead link on a live page. Which is why the workflow now fails on any link in the Overview that does not return 200, loopback examples excluded. A dead link on a public front door is worse than a blank page, and this class of error should not depend on someone re-checking by hand. It has the intended effect immediately: the Postgres page passes, and the dev page fails on `docs/dev-image.md`, which lands with #281. That is correct rather than inconvenient, since the dev Overview cannot be published until `extenddb/extenddb-dev` exists, and that repository is created as part of shipping #281. Verification: - YAML parses; seven steps in order. - The link check was run exactly as the job runs it, per file: Postgres all 200, dev failing only on the #281 doc. - Sizes re-checked against Docker Hub's caps: 3462 and 2675 characters against 25000; short descriptions unchanged at 74 and 89 against 100. --- .github/workflows/dockerhub-description.yml | 26 ++++++ docker/dockerhub-overview-dev.md | 88 +++++-------------- docker/dockerhub-overview.md | 94 ++++----------------- 3 files changed, 63 insertions(+), 145 deletions(-) diff --git a/.github/workflows/dockerhub-description.yml b/.github/workflows/dockerhub-description.yml index 1fd9befb..4b73f9c4 100644 --- a/.github/workflows/dockerhub-description.yml +++ b/.github/workflows/dockerhub-description.yml @@ -101,6 +101,32 @@ jobs: fi echo "::notice::Overview is ${BYTES} bytes, short description ${#SHORT} characters" + - name: Every link in the Overview must resolve + env: + FILE: ${{ steps.src.outputs.file }} + run: | + set -euo pipefail + # This page is a public front door, so a dead link is worse than a + # blank page. Loopback URLs are examples inside code blocks, not links. + FAILED=0 + while read -r url; do + [[ -z "$url" ]] && continue + CODE=$(curl -sS -o /dev/null -w '%{http_code}' -L --max-time 20 "$url" || echo 000) + if [[ "$CODE" != "200" ]]; then + echo "::error::${CODE} ${url}" + FAILED=1 + else + echo " 200 ${url}" + fi + done < <(grep -oE 'https://[^)"[:space:]]+' "$FILE" \ + | sed 's/[),.]*$//' \ + | grep -v '127\.0\.0\.1' \ + | sort -u) + if (( FAILED )); then + echo "::error::${FILE} contains links that do not resolve" + exit 1 + fi + - name: The Docker Hub repository must already exist env: IMAGE: ${{ inputs.image }} diff --git a/docker/dockerhub-overview-dev.md b/docker/dockerhub-overview-dev.md index 5161a172..95053700 100644 --- a/docker/dockerhub-overview-dev.md +++ b/docker/dockerhub-overview-dev.md @@ -1,89 +1,43 @@ -# ExtendDB dev (SQLite) +# ExtendDB dev -**The DynamoDB API, everywhere you run code.** — [extenddb.org](https://extenddb.org) +ExtendDB dev is a single-container, DynamoDB-compatible endpoint that enables developers to develop and test applications against the DynamoDB API in their own development environment. -> ExtendDB is an independent open source project managed by Amazon Web Services. It is not Amazon DynamoDB and does not contain any DynamoDB source code. "DynamoDB" is a trademark of Amazon.com, Inc. ExtendDB is a clean-room implementation that speaks the DynamoDB wire protocol. Behavioral differences from the real service are documented in [Differences from DynamoDB](https://github.com/ExtendDB/extenddb/blob/main/docs/differences-from-dynamodb.md). +## Benefits of using ExtendDB dev -A single container that speaks the DynamoDB wire protocol over SQLite, for **local development and CI**. No external database, no init step, no certificate to trust. +The image has everything built in: the API, its storage, and a seeded credential. There is no external database to run, no init step, no bootstrap sidecar, and no certificate to trust, so it drops into a containerized build or a continuous integration suite as one service. -> **Not for production, and not for shared networks.** This image serves plain HTTP with open authorization: whoever holds the credential can do everything. Always publish it to loopback only. For a durable, TLS-terminated deployment use [`extenddb/extenddb-postgres`](https://hub.docker.com/r/extenddb/extenddb-postgres). +ExtendDB dev works with your existing DynamoDB API calls. Any AWS SDK, CLI, or tool that talks to DynamoDB talks to it unchanged, so you point your client at a different endpoint and nothing else about your code changes. -## Quick start +It needs no internet connection, and there are no provisioned throughput, data storage, or data transfer costs. -```bash -docker run -d -p 127.0.0.1:18443:18443 -v extenddb:/var/lib/extenddb \ - extenddb/extenddb-dev -``` - -Then point any AWS SDK, CLI, or tool at it, unchanged: - -```bash -AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE \ -AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY \ -aws dynamodb list-tables \ - --region us-east-1 --endpoint-url http://127.0.0.1:18443 -``` - -That is AWS's documented example key pair. The server seeds it at startup and prints it in the logs. Secret scanners recognise it as an example credential, so it is safe in test fixtures. Pass `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` to the container to use a different one. - -## Storage modes +Data is durable across restarts by default, or entirely in memory when you want a pristine database per test run. -One image, mode chosen at run time. +## Getting started with ExtendDB dev on Docker -**File-backed (default).** The database is `/var/lib/extenddb/extenddb.sqlite`. Mount a volume there and data survives restarts and upgrades: +Run: ```bash -docker run -d -p 127.0.0.1:18443:18443 -v extenddb:/var/lib/extenddb \ - extenddb/extenddb-dev +docker run -d -p 127.0.0.1:18443:18443 -v extenddb:/var/lib/extenddb extenddb/extenddb-dev ``` -**In-memory.** Everything lives in process memory and vanishes when the container stops, which is what you want for a suite that needs a pristine database per run: +Then use it like DynamoDB. The server seeds AWS's documented example credential and prints it at startup: ```bash -docker run -d -p 127.0.0.1:18443:18443 \ - -e EXTENDDB__STORAGE__SQLITE__PATH=:memory: \ - extenddb/extenddb-dev +AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE \ +AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY \ +aws dynamodb list-tables --region us-east-1 --endpoint-url http://127.0.0.1:18443 ``` -## Why loopback matters - -The container binds `0.0.0.0` internally, which is required for port publishing to work at all. Containment is therefore the publish flag, not the bind address: keep the host side on `127.0.0.1`, as every example above does. Do not publish this port on a routable or shared interface, and do not put real data in it. - -## Tags - -| Tag | What it is | -| --- | --- | -| `X.Y.Z` | A released version. Immutable: a version tag is never overwritten. | -| `latest` | The highest released version. Only ever moves forward. | - -Both `linux/amd64` and `linux/arm64` are published as a single multi-architecture index, so `docker pull` selects the right platform automatically. - -## Runtime contract - -| | | -| --- | --- | -| Endpoint | `http://127.0.0.1:18443` — plain HTTP, no TLS | -| State volume | `/var/lib/extenddb` (file mode); omit it for a throwaway container | -| User | `65532:65532`, non-root | -| Healthcheck | built in, so `docker ps` and Compose `depends_on: condition: service_healthy` work with no extra wiring | - -The runtime base is distroless (`gcr.io/distroless/cc-debian12:nonroot`): no shell and no package manager, so `docker exec` into it is not possible by design. Licence notices for this image's exact dependency set ship at `/usr/share/doc/extenddb/SOFTWARE-LICENSE-NOTICES.html`. +For an in-memory database that vanishes with the container, drop the volume and add `-e EXTENDDB__STORAGE__SQLITE__PATH=:memory:`. -## Limitations +To learn how to configure ExtendDB dev, see [the `extenddb-dev` image](https://github.com/ExtendDB/extenddb/blob/main/docs/dev-image.md). -- Single node, single process. Throughput settings are accepted and tracked but not enforced as capacity. -- No TLS and no access control, as above. +For a durable, TLS-terminated deployment, use [`extenddb/extenddb-postgres`](https://hub.docker.com/r/extenddb/extenddb-postgres). -## Verifying the image +## Note -Signed with the same ExtendDB release key as the production image, an ECDSA P-256 key held in AWS KMS. The public key is attached to each [GitHub release](https://github.com/ExtendDB/extenddb/releases) as `extenddb-signing.pub.pem`, and each release's notes carry the exact `cosign verify` invocation for that release. +This image is built for local development and CI. It serves plain HTTP and its authorization is open, so whoever holds the credential can do anything. Publish it to loopback only, as the command above does, and do not keep real data in it. -## Links +ExtendDB is an independent open source project managed by Amazon Web Services. It is not Amazon DynamoDB and does not contain any DynamoDB source code. "DynamoDB" is a trademark of Amazon.com, Inc. ExtendDB is a clean-room implementation that speaks the DynamoDB wire protocol; behavioral differences from the service are documented in [Differences from DynamoDB](https://github.com/ExtendDB/extenddb/blob/main/docs/differences-from-dynamodb.md). -- Project site: [extenddb.org](https://extenddb.org) -- Source, issues and discussions: [github.com/ExtendDB/extenddb](https://github.com/ExtendDB/extenddb) -- [The `extenddb-dev` image](https://github.com/ExtendDB/extenddb/blob/main/docs/dev-image.md) -- [Getting started](https://github.com/ExtendDB/extenddb/blob/main/docs/getting-started.md) -- [Differences from DynamoDB](https://github.com/ExtendDB/extenddb/blob/main/docs/differences-from-dynamodb.md) -- Production image: [`extenddb/extenddb-postgres`](https://hub.docker.com/r/extenddb/extenddb-postgres) -- Licensed under Apache-2.0 +More at [extenddb.org](https://extenddb.org) and [github.com/ExtendDB/extenddb](https://github.com/ExtendDB/extenddb). Licensed under Apache-2.0. diff --git a/docker/dockerhub-overview.md b/docker/dockerhub-overview.md index 16b320fc..5f019d7c 100644 --- a/docker/dockerhub-overview.md +++ b/docker/dockerhub-overview.md @@ -1,93 +1,31 @@ -# ExtendDB (PostgreSQL backend) +# ExtendDB -**The DynamoDB API, everywhere you run code.** — [extenddb.org](https://extenddb.org) +ExtendDB is a DynamoDB-compatible API adapter that enables teams to run DynamoDB workloads on their own infrastructure, backed by a PostgreSQL database they operate. -> ExtendDB is an independent open source project managed by Amazon Web Services. It is not Amazon DynamoDB and does not contain any DynamoDB source code. "DynamoDB" is a trademark of Amazon.com, Inc. ExtendDB is a clean-room implementation that speaks the DynamoDB wire protocol. Behavioral differences from the real service are documented in [Differences from DynamoDB](https://github.com/ExtendDB/extenddb/blob/main/docs/differences-from-dynamodb.md). +## Benefits of using ExtendDB -ExtendDB speaks the DynamoDB wire protocol, so any AWS SDK, CLI, or tool that works with DynamoDB works with ExtendDB unchanged. Point your client at the ExtendDB endpoint and nothing else about your code changes. +ExtendDB works with your existing DynamoDB API calls. Any AWS SDK, CLI, or tool that talks to DynamoDB talks to ExtendDB unchanged, so you point your client at a different endpoint and nothing else about your code changes. The full surface is there: CRUD, Query, Scan, Batch, Transactions, Streams, TTL, Import and Export, plus SigV4 with IAM-compatible users, roles and policies. -This image is an **adapter, not a database**. It contains the ExtendDB server compiled with the PostgreSQL backend. It does **not** contain a PostgreSQL server or data directory: bring your own PostgreSQL 14 or newer, whether that is RDS, Aurora PostgreSQL, or self-managed. +It is an adapter, not a database. This image contains the ExtendDB server compiled with the PostgreSQL backend; you supply PostgreSQL 14 or newer, whether that is RDS, Aurora PostgreSQL, or self-managed. Your data therefore lives somewhere you already know how to back up, replicate, monitor and restore, using tools you already run. -## Tags +Because it runs on your infrastructure, ExtendDB needs no internet connection, and there are no provisioned throughput, data storage, or data transfer costs. That makes it a fit for self-hosted and air-gapped deployments, for DynamoDB semantics on any cloud that runs PostgreSQL, and for continuous integration against a durable backend rather than an ephemeral one. -| Tag | What it is | -| --- | --- | -| `X.Y.Z` | A released version. Immutable: a version tag is never overwritten. | -| `latest` | The highest released version. Only ever moves forward. | -| `sha-<40-char-commit>` | A build candidate, published before promotion. Not a release; pin a version instead. | +## Getting started with ExtendDB on Docker -Both `linux/amd64` and `linux/arm64` are published as a single multi-architecture index, so `docker pull` selects the right platform automatically. +For local development, the Compose stack in the repository starts PostgreSQL alongside ExtendDB and runs the one-time bootstrap for you. See [local development with Docker Compose](https://github.com/ExtendDB/extenddb/blob/main/docs/local-dev-docker-compose.md). -For production, pin `X.Y.Z` or a digest rather than `latest`. +For a real deployment the image serves three explicit operations: `extenddb init` once to bootstrap the database, `extenddb migrate` when upgrading, and `extenddb serve` in steady state, which is the default command. It listens on port 18443 over TLS, keeps configuration and certificates under `/var/lib/extenddb`, runs as a non-root user, and supports a read-only root filesystem. -## Quick start +To learn how to configure and operate it, see [getting started](https://github.com/ExtendDB/extenddb/blob/main/docs/getting-started.md) and the [container notes](https://github.com/ExtendDB/extenddb/blob/main/docker/README.md). -For local development the fastest path is the Compose stack in the repository, which brings up PostgreSQL alongside ExtendDB and runs the one-time bootstrap for you. See [Local development with Docker Compose](https://github.com/ExtendDB/extenddb/blob/main/docs/local-dev-docker-compose.md). +## Tags and verification -Once running, use any DynamoDB client against the endpoint: +Pin `X.Y.Z` or a digest for production; a version tag is never overwritten. `latest` tracks the highest release and only moves forward. Tags of the form `sha-` are unpromoted build candidates, not releases. Both `linux/amd64` and `linux/arm64` ship as one multi-architecture index. -```bash -aws dynamodb list-tables \ - --endpoint-url https://127.0.0.1:18443 \ - --region us-east-1 -``` +Every published image is signed with ExtendDB's release key, an ECDSA P-256 key held in AWS KMS. Each [release](https://github.com/ExtendDB/extenddb/releases) attaches the public key and gives the exact `cosign verify` command for that release. The same images are mirrored by digest to `ghcr.io/extenddb/extenddb-postgres`. -## Running it yourself +## Note -The image serves three explicit operations. Do not use the Compose bootstrap behaviour as a production control plane. +ExtendDB is an independent open source project managed by Amazon Web Services. It is not Amazon DynamoDB and does not contain any DynamoDB source code. "DynamoDB" is a trademark of Amazon.com, Inc. ExtendDB is a clean-room implementation that speaks the DynamoDB wire protocol; behavioral differences from the service are documented in [Differences from DynamoDB](https://github.com/ExtendDB/extenddb/blob/main/docs/differences-from-dynamodb.md). -**1. Bootstrap once**, as a single Job holding the PostgreSQL bootstrap credential. Inject `EXTENDDB_PG_PASSWORD` and `EXTENDDB_APP_PASSWORD` from your orchestrator's secret store rather than passing them as arguments: - -```bash -extenddb init \ - --config /var/lib/extenddb/extenddb.toml \ - --backend postgres \ - --pg-host --pg-port 5432 \ - --pg-user --extenddb-user \ - --bind-addr 0.0.0.0 --tls-san -``` - -**2. On upgrade**, back up PostgreSQL, scale down incompatible serving versions, then run one migration Job: - -```bash -extenddb migrate --config /var/lib/extenddb/extenddb.toml --yes --pg-user -``` - -The generated config stores the application connection but not the bootstrap password, so the migration Job receives that elevated credential separately. The serving container never gets it. - -**3. Steady state** is the image's default command: - -```bash -extenddb serve --config /var/lib/extenddb/extenddb.toml --foreground -``` - -## Runtime contract - -| | | -| --- | --- | -| Port | `18443` (HTTPS) | -| State volume | `/var/lib/extenddb` — must be writable; holds config and TLS material | -| User | `10001:10001`, non-root | -| Entrypoint | `tini`, so `SIGTERM` reaches the server and shutdown is graceful | -| Healthcheck | `extenddb healthcheck` — liveness only, it does not probe PostgreSQL readiness | - -Supports a read-only root filesystem. Drop all capabilities, keep the runtime's default seccomp profile, and set CPU and memory limits in your orchestrator. - -TLS is on by default with a generated self-signed certificate, replaceable with a CA-signed one. The Compose stack's certificate and passwords are development defaults and must not be used anywhere shared. - -## Verifying the image - -Every published image is signed with ExtendDB's release key, an ECDSA P-256 key held in AWS KMS. The public key is attached to each [GitHub release](https://github.com/ExtendDB/extenddb/releases) as `extenddb-signing.pub.pem`, and each release's notes carry the exact `cosign verify` invocation for that release. Verify against the digest you are deploying. - -## Also available on GHCR - -The same images, by digest, are mirrored to `ghcr.io/extenddb/extenddb-postgres`. - -## Links - -- Project site: [extenddb.org](https://extenddb.org) -- Source, issues and discussions: [github.com/ExtendDB/extenddb](https://github.com/ExtendDB/extenddb) -- [Getting started](https://github.com/ExtendDB/extenddb/blob/main/docs/getting-started.md) -- [Differences from DynamoDB](https://github.com/ExtendDB/extenddb/blob/main/docs/differences-from-dynamodb.md) -- [Troubleshooting](https://github.com/ExtendDB/extenddb/blob/main/docs/troubleshooting.md) -- Licensed under Apache-2.0 +More at [extenddb.org](https://extenddb.org) and [github.com/ExtendDB/extenddb](https://github.com/ExtendDB/extenddb). Licensed under Apache-2.0. From a098dccf45f98ccccba5914afae852ee65f9b61e Mon Sep 17 00:00:00 2001 From: Lee Hannigan Date: Thu, 20 Aug 2026 00:01:46 +0000 Subject: [PATCH 4/6] docs(docker): correct the dev Overview quickstart to port 18080 The published extenddb/extenddb-dev image exposes 18080/tcp and sets EXTENDDB__SERVER__PORT=18080 (verified on the pulled 0.1.7 digest). This Overview was written before that flip landed in #281, so both quickstart commands pointed at 18443 and neither worked as printed. Both corrected commands were run verbatim against the published image: the container reaches Docker healthy and list-tables returns. The memory-mode instruction was also verified, with a durable control: the volume-backed container keeps its table across a full container recreate while the :memory: container loses it on restart. The Postgres Overview is unchanged; 18443 is correct there, that image does serve TLS on it. --- docker/dockerhub-overview-dev.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docker/dockerhub-overview-dev.md b/docker/dockerhub-overview-dev.md index 95053700..a0318079 100644 --- a/docker/dockerhub-overview-dev.md +++ b/docker/dockerhub-overview-dev.md @@ -17,7 +17,7 @@ Data is durable across restarts by default, or entirely in memory when you want Run: ```bash -docker run -d -p 127.0.0.1:18443:18443 -v extenddb:/var/lib/extenddb extenddb/extenddb-dev +docker run -d -p 127.0.0.1:18080:18080 -v extenddb:/var/lib/extenddb extenddb/extenddb-dev ``` Then use it like DynamoDB. The server seeds AWS's documented example credential and prints it at startup: @@ -25,7 +25,7 @@ Then use it like DynamoDB. The server seeds AWS's documented example credential ```bash AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE \ AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY \ -aws dynamodb list-tables --region us-east-1 --endpoint-url http://127.0.0.1:18443 +aws dynamodb list-tables --region us-east-1 --endpoint-url http://127.0.0.1:18080 ``` For an in-memory database that vanishes with the container, drop the volume and add `-e EXTENDDB__STORAGE__SQLITE__PATH=:memory:`. From 319cc0e8134fd5a6640c5e006b299c905281f1ee Mon Sep 17 00:00:00 2001 From: Lee Hannigan Date: Thu, 20 Aug 2026 10:55:25 +0000 Subject: [PATCH 5/6] docs(docker): point verification at the in-repo key, list all three registries Two claims in the postgres Overview had drifted from reality. The verification paragraph said each GitHub release attaches the public key and gives the exact cosign verify command. That was true of v0.1.6 and is false of v0.1.7, which carries no assets and no verify instructions. The durable home for the key is now the repository itself: #283 commits extenddb-signing.pub.pem at the root (merged 2026-08-20, link verified 200). Pointing the Overview there survives any release whose notes are thin, and per-release command specifics stay in release notes where tlog differences between pre- and post-Rekor releases belong. Restoring the v0.1.7 release assets is tracked separately. The registry list said images are mirrored to GHCR. ECR Public already serves 0.1.5, 0.1.6, latest and both signature artifacts (verified live), and #283 makes it a first-class promotion target, so the Overview now names both mirrors and notes the signatures travel too. The dev Overview needed nothing: its rewrite carries no per-release verification claim. All seven links in the postgres Overview return 200, including the new in-repo key path; 3,571 characters against Docker Hub's 25,000 cap. --- docker/dockerhub-overview.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docker/dockerhub-overview.md b/docker/dockerhub-overview.md index 5f019d7c..ea36b526 100644 --- a/docker/dockerhub-overview.md +++ b/docker/dockerhub-overview.md @@ -22,7 +22,7 @@ To learn how to configure and operate it, see [getting started](https://github.c Pin `X.Y.Z` or a digest for production; a version tag is never overwritten. `latest` tracks the highest release and only moves forward. Tags of the form `sha-` are unpromoted build candidates, not releases. Both `linux/amd64` and `linux/arm64` ship as one multi-architecture index. -Every published image is signed with ExtendDB's release key, an ECDSA P-256 key held in AWS KMS. Each [release](https://github.com/ExtendDB/extenddb/releases) attaches the public key and gives the exact `cosign verify` command for that release. The same images are mirrored by digest to `ghcr.io/extenddb/extenddb-postgres`. +Every published image is signed with ExtendDB's release key, an ECDSA P-256 key held in AWS KMS. The public key is committed in the repository as [`extenddb-signing.pub.pem`](https://github.com/ExtendDB/extenddb/blob/main/extenddb-signing.pub.pem), for use with `cosign verify --key`. The same images and their signatures are mirrored by digest to `ghcr.io/extenddb/extenddb-postgres` and `public.ecr.aws/extenddb/extenddb-postgres`. ## Note From f6e99285d7910c86a423c1e49f72d5e93b6bde4e Mon Sep 17 00:00:00 2001 From: Lee Hannigan Date: Thu, 20 Aug 2026 12:18:34 +0000 Subject: [PATCH 6/6] ci(docs): fix the false-failing verify comparison; validate before the gate Review round on #289 (robinnsc), all four items. The blocker, reproduced before fixing: the verify step compared $(jq -r ...) output - trailing newline stripped by command substitution - against raw `cat` output, which keeps the newline every .md file ends with. diff therefore reported a difference on every successful publish: the gate could never pass, and the first dispatch would have reported failure after publishing correctly. Both sides of the comparison are now command substitutions, so both are stripped and the comparison is apples-to-apples. Verified against the real overview file: the old comparison fails, the new one passes. The should-fix: all four pre-publish checks (file exists, size caps, link health, repository exists) are credential-free but lived inside the environment-gated job, so a dead link burned a reviewer approval to discover - the same shape flagged on sign-image's dev option. They now run in an ungated `validate` job with no secrets; `publish` needs it and carries the gate, so the reviewer approves content that has already passed validation. Nits: the link checker and the repo-exists probe retry transient failures (--retry 2), which post-split cost nothing. The PR body's stale "extenddb/extenddb-dev does not exist yet (404)" claim is corrected in the PR description alongside this commit; the repo was created with the 0.1.7 release. --- .github/workflows/dockerhub-description.yml | 51 +++++++++++++++------ 1 file changed, 37 insertions(+), 14 deletions(-) diff --git a/.github/workflows/dockerhub-description.yml b/.github/workflows/dockerhub-description.yml index 4b73f9c4..64183957 100644 --- a/.github/workflows/dockerhub-description.yml +++ b/.github/workflows/dockerhub-description.yml @@ -13,10 +13,13 @@ # Sourced from `docker/dockerhub-overview*.md` so the published Overview is # reviewed like any other content and cannot drift from the repository. # -# Dispatch-only and gated on the `dockerhub` environment, because it uses the -# same credential that can push images. Deliberately NOT wired into -# promote-image: the Overview is version-neutral, so a wording fix should not -# require a release, and a release should not silently rewrite it. +# Two jobs: `validate` is credential-free and ungated, so a dead link, an +# oversize file, or a missing Docker Hub repository is discovered on dispatch +# without spending a reviewer approval. `publish` needs validate and carries +# the `dockerhub` environment gate, because it uses the same credential that +# can push images. Deliberately NOT wired into promote-image: the Overview is +# version-neutral, so a wording fix should not require a release, and a +# release should not silently rewrite it. # # One image per dispatch rather than a matrix over both, because a Docker Hub # repository that does not exist yet must fail loudly for that image alone @@ -46,10 +49,14 @@ env: HUB_NAMESPACE: extenddb jobs: - publish-description: + # Credential-free, no environment gate: everything that can be proven + # without a secret is proven before a reviewer is asked to approve. + validate: runs-on: ubuntu-latest timeout-minutes: 10 - environment: dockerhub + outputs: + file: ${{ steps.src.outputs.file }} + short: ${{ steps.src.outputs.short }} steps: - name: Check out main uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 @@ -108,10 +115,12 @@ jobs: set -euo pipefail # This page is a public front door, so a dead link is worse than a # blank page. Loopback URLs are examples inside code blocks, not links. + # --retry so a transient blip does not fail the run. FAILED=0 while read -r url; do [[ -z "$url" ]] && continue - CODE=$(curl -sS -o /dev/null -w '%{http_code}' -L --max-time 20 "$url" || echo 000) + CODE=$(curl -sS -o /dev/null -w '%{http_code}' -L --max-time 20 \ + --retry 2 --retry-delay 3 "$url" || echo 000) if [[ "$CODE" != "200" ]]; then echo "::error::${CODE} ${url}" FAILED=1 @@ -134,7 +143,7 @@ jobs: set -euo pipefail # A 404 here means the repository has not been created yet. Say so # plainly rather than letting the PATCH fail with an opaque error. - CODE=$(curl -sS -o /dev/null -w '%{http_code}' \ + CODE=$(curl -sS -o /dev/null -w '%{http_code}' --retry 2 --retry-delay 3 \ "https://hub.docker.com/v2/repositories/${HUB_NAMESPACE}/${IMAGE}/") if [[ "$CODE" == "404" ]]; then echo "::error::docker.io/${HUB_NAMESPACE}/${IMAGE} does not exist. Create the repository on Docker Hub first." @@ -145,11 +154,23 @@ jobs: exit 1 fi + # The only job with the credential, behind the dockerhub environment gate. + # By the time a reviewer approves, the content has already passed validation. + publish: + needs: validate + runs-on: ubuntu-latest + timeout-minutes: 10 + environment: dockerhub + env: + FILE: ${{ needs.validate.outputs.file }} + SHORT: ${{ needs.validate.outputs.short }} + steps: + - name: Check out main + uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 + - name: Publish the Overview env: IMAGE: ${{ inputs.image }} - FILE: ${{ steps.src.outputs.file }} - SHORT: ${{ steps.src.outputs.short }} DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }} DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }} run: | @@ -180,8 +201,6 @@ jobs: - name: Verify it took effect env: IMAGE: ${{ inputs.image }} - FILE: ${{ steps.src.outputs.file }} - SHORT: ${{ steps.src.outputs.short }} run: | set -euo pipefail # Read back anonymously: proves the Overview is what an unauthenticated @@ -195,9 +214,13 @@ jobs: echo "::error::Overview is still empty after the PATCH" exit 1 fi - if ! diff -q <(printf '%s' "$GOT_FULL") <(cat "$FILE") >/dev/null; then + # Both sides are command substitutions, so both have trailing + # newlines stripped: apples-to-apples. Comparing a stripped + # substitution against raw `cat` output false-fails on the + # trailing newline every .md file ends with. + if [[ "$GOT_FULL" != "$(cat "$FILE")" ]]; then echo "::error::published Overview does not match ${FILE}" - diff <(printf '%s' "$GOT_FULL") "$FILE" | head -20 || true + diff <(printf '%s\n' "$GOT_FULL") "$FILE" | head -20 || true exit 1 fi if [[ "$GOT_SHORT" != "$SHORT" ]]; then