Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
230 changes: 230 additions & 0 deletions .github/workflows/dockerhub-description.yml
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,230 @@
# Copyright 2026 ExtendDB contributors
# SPDX-License-Identifier: Apache-2.0
#
# Publish a Docker Hub repository Overview from a file in this repository.
#
# Neither `docker push` nor the `skopeo copy` that promote-image uses touches
# repository metadata: the Overview lives on the Docker Hub repository object,
# not on any image. GHCR derives its description from the OCI labels the
# Dockerfiles already set, so the same image reads as documented there and blank
# on Docker Hub. That is why this exists as its own step rather than falling out
# of a push.
#
# Sourced from `docker/dockerhub-overview*.md` so the published Overview is
# reviewed like any other content and cannot drift from the repository.
#
# Two jobs: `validate` is credential-free and ungated, so a dead link, an
# oversize file, or a missing Docker Hub repository is discovered on dispatch
# without spending a reviewer approval. `publish` needs validate and carries
# the `dockerhub` environment gate, because it uses the same credential that
# can push images. Deliberately NOT wired into promote-image: the Overview is
# version-neutral, so a wording fix should not require a release, and a
# release should not silently rewrite it.
#
# One image per dispatch rather than a matrix over both, because a Docker Hub
# repository that does not exist yet must fail loudly for that image alone
# instead of failing a run that also had work to do for the other.

name: Docker Hub description

on:
workflow_dispatch:
inputs:
image:
description: 'Which Docker Hub repository to publish the Overview for'
required: true
type: choice
options:
- extenddb-postgres
- extenddb-dev

permissions:
contents: read

concurrency:
group: dockerhub-description-${{ inputs.image }}
cancel-in-progress: false

env:
HUB_NAMESPACE: extenddb

jobs:
# Credential-free, no environment gate: everything that can be proven
# without a secret is proven before a reviewer is asked to approve.
validate:
runs-on: ubuntu-latest
timeout-minutes: 10
outputs:
file: ${{ steps.src.outputs.file }}
short: ${{ steps.src.outputs.short }}
steps:
- name: Check out main
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0

- name: Resolve the Overview source for this image
id: src
env:
IMAGE: ${{ inputs.image }}
run: |
set -euo pipefail
# Docker Hub caps the short description at 100 characters; asserted below.
case "$IMAGE" in
extenddb-postgres)
FILE='docker/dockerhub-overview.md'
SHORT='DynamoDB-compatible API adapter backed by your own PostgreSQL 14+ database'
;;
extenddb-dev)
FILE='docker/dockerhub-overview-dev.md'
SHORT='DynamoDB-compatible API over SQLite for local dev and CI. Plain HTTP, not for production.'
;;
*)
echo "::error::unknown image '$IMAGE'"
exit 1
;;
esac
echo "file=$FILE" >> "$GITHUB_OUTPUT"
echo "short=$SHORT" >> "$GITHUB_OUTPUT"
echo "::notice::publishing $FILE to ${IMAGE}"

- name: The Overview file must exist and be within Docker Hub's limits
env:
FILE: ${{ steps.src.outputs.file }}
SHORT: ${{ steps.src.outputs.short }}
run: |
set -euo pipefail
if [[ ! -s "$FILE" ]]; then
echo "::error::$FILE is missing or empty"
exit 1
fi
BYTES=$(wc -c < "$FILE")
# Docker Hub caps full_description at 25000 characters.
if (( BYTES > 25000 )); then
echo "::error::$FILE is ${BYTES} bytes; Docker Hub caps the Overview at 25000"
exit 1
fi
if (( ${#SHORT} > 100 )); then
echo "::error::short description is ${#SHORT} characters; Docker Hub caps it at 100"
exit 1
fi
echo "::notice::Overview is ${BYTES} bytes, short description ${#SHORT} characters"

- name: Every link in the Overview must resolve
env:
FILE: ${{ steps.src.outputs.file }}
run: |
set -euo pipefail
# This page is a public front door, so a dead link is worse than a
# blank page. Loopback URLs are examples inside code blocks, not links.
# --retry so a transient blip does not fail the run.
FAILED=0
while read -r url; do
[[ -z "$url" ]] && continue
CODE=$(curl -sS -o /dev/null -w '%{http_code}' -L --max-time 20 \
--retry 2 --retry-delay 3 "$url" || echo 000)
if [[ "$CODE" != "200" ]]; then
echo "::error::${CODE} ${url}"
FAILED=1
else
echo " 200 ${url}"
fi
done < <(grep -oE 'https://[^)"[:space:]]+' "$FILE" \
| sed 's/[),.]*$//' \
| grep -v '127\.0\.0\.1' \
| sort -u)
if (( FAILED )); then
echo "::error::${FILE} contains links that do not resolve"
exit 1
fi

- name: The Docker Hub repository must already exist
env:
IMAGE: ${{ inputs.image }}
run: |
set -euo pipefail
# A 404 here means the repository has not been created yet. Say so
# plainly rather than letting the PATCH fail with an opaque error.
CODE=$(curl -sS -o /dev/null -w '%{http_code}' --retry 2 --retry-delay 3 \
"https://hub.docker.com/v2/repositories/${HUB_NAMESPACE}/${IMAGE}/")
if [[ "$CODE" == "404" ]]; then
echo "::error::docker.io/${HUB_NAMESPACE}/${IMAGE} does not exist. Create the repository on Docker Hub first."
exit 1
fi
if [[ "$CODE" != "200" ]]; then
echo "::error::unexpected HTTP ${CODE} querying docker.io/${HUB_NAMESPACE}/${IMAGE}"
exit 1
fi

# The only job with the credential, behind the dockerhub environment gate.
# By the time a reviewer approves, the content has already passed validation.
publish:
needs: validate
runs-on: ubuntu-latest
timeout-minutes: 10
environment: dockerhub
env:
FILE: ${{ needs.validate.outputs.file }}
SHORT: ${{ needs.validate.outputs.short }}
steps:
- name: Check out main
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0

- name: Publish the Overview
env:
IMAGE: ${{ inputs.image }}
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
run: |
set -euo pipefail

# Hub API v2 wants a session token, then a PATCH on the repository.
TOKEN=$(jq -n --arg u "$DOCKERHUB_USERNAME" --arg p "$DOCKERHUB_TOKEN" \
'{username:$u, password:$p}' \
| curl -sS --fail-with-body -X POST \
-H 'Content-Type: application/json' \
--data @- \
https://hub.docker.com/v2/users/login/ \
| jq -r '.token')
if [[ -z "$TOKEN" || "$TOKEN" == "null" ]]; then
echo "::error::could not obtain a Docker Hub session token"
exit 1
fi

jq -n --rawfile full "$FILE" --arg short "$SHORT" \
'{full_description:$full, description:$short}' \
| curl -sS --fail-with-body -X PATCH \
-H "Authorization: JWT ${TOKEN}" \
-H 'Content-Type: application/json' \
--data @- \
"https://hub.docker.com/v2/repositories/${HUB_NAMESPACE}/${IMAGE}/" \
> /dev/null

- name: Verify it took effect
env:
IMAGE: ${{ inputs.image }}
run: |
set -euo pipefail
# Read back anonymously: proves the Overview is what an unauthenticated
# visitor now sees, rather than trusting the write's own response.
GOT=$(curl -sS --fail-with-body \
"https://hub.docker.com/v2/repositories/${HUB_NAMESPACE}/${IMAGE}/")
GOT_FULL=$(jq -r '.full_description // ""' <<< "$GOT")
GOT_SHORT=$(jq -r '.description // ""' <<< "$GOT")

if [[ -z "$GOT_FULL" ]]; then
echo "::error::Overview is still empty after the PATCH"
exit 1
fi
# Both sides are command substitutions, so both have trailing
# newlines stripped: apples-to-apples. Comparing a stripped
# substitution against raw `cat` output false-fails on the
# trailing newline every .md file ends with.
if [[ "$GOT_FULL" != "$(cat "$FILE")" ]]; then
echo "::error::published Overview does not match ${FILE}"
diff <(printf '%s\n' "$GOT_FULL") "$FILE" | head -20 || true
exit 1
fi
if [[ "$GOT_SHORT" != "$SHORT" ]]; then
echo "::error::short description is '${GOT_SHORT}', expected '${SHORT}'"
exit 1
fi
echo "::notice::Overview for ${IMAGE} published and verified anonymously"
43 changes: 43 additions & 0 deletions docker/dockerhub-overview-dev.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
# ExtendDB dev

ExtendDB dev is a single-container, DynamoDB-compatible endpoint that enables developers to develop and test applications against the DynamoDB API in their own development environment.

## Benefits of using ExtendDB dev

The image has everything built in: the API, its storage, and a seeded credential. There is no external database to run, no init step, no bootstrap sidecar, and no certificate to trust, so it drops into a containerized build or a continuous integration suite as one service.

ExtendDB dev works with your existing DynamoDB API calls. Any AWS SDK, CLI, or tool that talks to DynamoDB talks to it unchanged, so you point your client at a different endpoint and nothing else about your code changes.

It needs no internet connection, and there are no provisioned throughput, data storage, or data transfer costs.

Data is durable across restarts by default, or entirely in memory when you want a pristine database per test run.

## Getting started with ExtendDB dev on Docker

Run:

```bash
docker run -d -p 127.0.0.1:18080:18080 -v extenddb:/var/lib/extenddb extenddb/extenddb-dev
```

Then use it like DynamoDB. The server seeds AWS's documented example credential and prints it at startup:

```bash
AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE \
AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY \
aws dynamodb list-tables --region us-east-1 --endpoint-url http://127.0.0.1:18080
```

For an in-memory database that vanishes with the container, drop the volume and add `-e EXTENDDB__STORAGE__SQLITE__PATH=:memory:`.

To learn how to configure ExtendDB dev, see [the `extenddb-dev` image](https://github.com/ExtendDB/extenddb/blob/main/docs/dev-image.md).

For a durable, TLS-terminated deployment, use [`extenddb/extenddb-postgres`](https://hub.docker.com/r/extenddb/extenddb-postgres).

## Note

This image is built for local development and CI. It serves plain HTTP and its authorization is open, so whoever holds the credential can do anything. Publish it to loopback only, as the command above does, and do not keep real data in it.

ExtendDB is an independent open source project managed by Amazon Web Services. It is not Amazon DynamoDB and does not contain any DynamoDB source code. "DynamoDB" is a trademark of Amazon.com, Inc. ExtendDB is a clean-room implementation that speaks the DynamoDB wire protocol; behavioral differences from the service are documented in [Differences from DynamoDB](https://github.com/ExtendDB/extenddb/blob/main/docs/differences-from-dynamodb.md).

More at [extenddb.org](https://extenddb.org) and [github.com/ExtendDB/extenddb](https://github.com/ExtendDB/extenddb). Licensed under Apache-2.0.
31 changes: 31 additions & 0 deletions docker/dockerhub-overview.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
# ExtendDB

ExtendDB is a DynamoDB-compatible API adapter that enables teams to run DynamoDB workloads on their own infrastructure, backed by a PostgreSQL database they operate.

## Benefits of using ExtendDB

ExtendDB works with your existing DynamoDB API calls. Any AWS SDK, CLI, or tool that talks to DynamoDB talks to ExtendDB unchanged, so you point your client at a different endpoint and nothing else about your code changes. The full surface is there: CRUD, Query, Scan, Batch, Transactions, Streams, TTL, Import and Export, plus SigV4 with IAM-compatible users, roles and policies.

It is an adapter, not a database. This image contains the ExtendDB server compiled with the PostgreSQL backend; you supply PostgreSQL 14 or newer, whether that is RDS, Aurora PostgreSQL, or self-managed. Your data therefore lives somewhere you already know how to back up, replicate, monitor and restore, using tools you already run.

Because it runs on your infrastructure, ExtendDB needs no internet connection, and there are no provisioned throughput, data storage, or data transfer costs. That makes it a fit for self-hosted and air-gapped deployments, for DynamoDB semantics on any cloud that runs PostgreSQL, and for continuous integration against a durable backend rather than an ephemeral one.

## Getting started with ExtendDB on Docker

For local development, the Compose stack in the repository starts PostgreSQL alongside ExtendDB and runs the one-time bootstrap for you. See [local development with Docker Compose](https://github.com/ExtendDB/extenddb/blob/main/docs/local-dev-docker-compose.md).

For a real deployment the image serves three explicit operations: `extenddb init` once to bootstrap the database, `extenddb migrate` when upgrading, and `extenddb serve` in steady state, which is the default command. It listens on port 18443 over TLS, keeps configuration and certificates under `/var/lib/extenddb`, runs as a non-root user, and supports a read-only root filesystem.

To learn how to configure and operate it, see [getting started](https://github.com/ExtendDB/extenddb/blob/main/docs/getting-started.md) and the [container notes](https://github.com/ExtendDB/extenddb/blob/main/docker/README.md).

## Tags and verification

Pin `X.Y.Z` or a digest for production; a version tag is never overwritten. `latest` tracks the highest release and only moves forward. Tags of the form `sha-<commit>` are unpromoted build candidates, not releases. Both `linux/amd64` and `linux/arm64` ship as one multi-architecture index.

Every published image is signed with ExtendDB's release key, an ECDSA P-256 key held in AWS KMS. The public key is committed in the repository as [`extenddb-signing.pub.pem`](https://github.com/ExtendDB/extenddb/blob/main/extenddb-signing.pub.pem), for use with `cosign verify --key`. The same images and their signatures are mirrored by digest to `ghcr.io/extenddb/extenddb-postgres` and `public.ecr.aws/extenddb/extenddb-postgres`.

## Note

ExtendDB is an independent open source project managed by Amazon Web Services. It is not Amazon DynamoDB and does not contain any DynamoDB source code. "DynamoDB" is a trademark of Amazon.com, Inc. ExtendDB is a clean-room implementation that speaks the DynamoDB wire protocol; behavioral differences from the service are documented in [Differences from DynamoDB](https://github.com/ExtendDB/extenddb/blob/main/docs/differences-from-dynamodb.md).

More at [extenddb.org](https://extenddb.org) and [github.com/ExtendDB/extenddb](https://github.com/ExtendDB/extenddb). Licensed under Apache-2.0.
Loading