mirror of
https://github.com/jjbrunton/Sauci.git
synced 2026-10-31 10:19:43 +00:00
feat(backend): add standalone data plane
Replace the Supabase application data plane with the Node/PostgreSQL API, migrate mobile, admin, web, and MCP callers, add controlled cutover tooling, and define hardened Dokploy infrastructure.
This commit is contained in:
1 parent
01bb8225ae
commit
006761d3b0
294 files changed
+16259
-6138
No files matched your search
@@ -0,0 +1,8 @@
|
||||
# Local development only. Docker Compose already supplies these defaults.
|
||||
# Do not reuse these credentials for a deployed environment.
|
||||
# Database name/user/password are deliberately fixed by compose.yaml so shell
|
||||
# variables from another environment cannot accidentally change this stack.
|
||||
SAUCI_POSTGRES_PORT=54320
|
||||
|
||||
# The API runs on the host during development, outside Docker Compose.
|
||||
SAUCI_API_PORT=3003
|
||||
@@ -0,0 +1,32 @@
|
||||
# Copy these names into Dokploy's environment configuration. Do not commit values.
|
||||
POSTGRES_DB=sauci
|
||||
POSTGRES_USER=sauci
|
||||
POSTGRES_PASSWORD=generate-a-long-random-password
|
||||
DATABASE_URL=postgresql://sauci:URL_ENCODED_PASSWORD@postgres:5432/sauci
|
||||
|
||||
# Hosted Supabase Auth only. The service-role key is used solely by the
|
||||
# server-side Auth Admin endpoint for account deletion and admin identity
|
||||
# enrichment. It belongs on the API only, never the worker, MCP, or a client.
|
||||
SUPABASE_AUTH_URL=https://your-project.supabase.co
|
||||
SUPABASE_AUTH_AUDIENCE=authenticated
|
||||
SUPABASE_AUTH_SERVICE_ROLE_KEY=set-a-server-only-auth-admin-key
|
||||
|
||||
# Opaque standalone admin API credential. Configure both values together.
|
||||
# The UUID must identify an active least-privilege admin_users row. MCP receives
|
||||
# the same token as SAUCI_ADMIN_API_TOKEN, not this user ID and no Supabase key.
|
||||
ADMIN_API_SERVICE_TOKEN=generate-at-least-32-random-characters
|
||||
ADMIN_API_SERVICE_USER_ID=set-to-an-active-migrated-admin-profile-uuid
|
||||
REVENUECAT_API_KEY=set-the-secret-revenuecat-api-key
|
||||
REVENUECAT_ENTITLEMENT_ID=Sauci Pro
|
||||
REVENUECAT_WEBHOOK_SECRET=generate-a-long-random-webhook-secret
|
||||
MEDIA_SIGNING_SECRET=generate-at-least-32-random-characters
|
||||
MEDIA_PUBLIC_BASE_URL=https://api.example.com
|
||||
|
||||
# Operations worker. Discord is optional.
|
||||
CLASSIFIER_ENABLED=true
|
||||
OPENROUTER_API_KEY=set-the-secret-openrouter-key
|
||||
CLASSIFIER_MODEL=openai/gpt-4o
|
||||
ADMIN_PRIVATE_KEY_JWK=set-the-private-jwk-json-used-for-message-moderation
|
||||
DISCORD_WEBHOOK_URL=
|
||||
WORKER_POLL_INTERVAL_MS=30000
|
||||
WORKER_BATCH_SIZE=25
|
||||
@@ -0,0 +1,115 @@
|
||||
# Standalone backend infrastructure
|
||||
|
||||
This Compose project is the local data plane for the standalone Sauci API. It
|
||||
does not run Supabase services. Supabase remains the hosted identity provider;
|
||||
the API verifies its access tokens and owns all application data access.
|
||||
|
||||
Use `scripts/dev-local.sh` rather than invoking Compose directly. PostgreSQL is
|
||||
bound to loopback and persists in the `sauci-local_sauci-postgres-data` volume.
|
||||
The committed credentials are deliberately local-only and must not be reused by
|
||||
a deployed environment.
|
||||
|
||||
Private media is stored on the API's `media-data` volume. Mobile clients upload
|
||||
with bearer authentication, persist opaque `media:<uuid>` references, and ask
|
||||
the API for short-lived signed read URLs. The public content route accepts only
|
||||
a valid unexpired HMAC capability; it never exposes directory paths.
|
||||
|
||||
## Dokploy deployment contract
|
||||
|
||||
`compose.production.yaml` is a deployment-ready baseline containing the Node
|
||||
API and worker, PostgreSQL, and a one-shot migration service. Configure Dokploy to build the
|
||||
Compose file from the repository root and route the public API hostname to the
|
||||
`api` service on port `3003`. Do not publish PostgreSQL.
|
||||
|
||||
Copy the variable names from `.env.production.example` into Dokploy's secret
|
||||
configuration. `POSTGRES_PASSWORD` and the password embedded (URL encoded) in
|
||||
`DATABASE_URL` must match. Generate them for the deployment; never reuse the
|
||||
committed local password. The API needs only the hosted Supabase Auth URL to
|
||||
discover its issuer and JWKS endpoint. The server-only service-role key is used
|
||||
solely by the hosted Auth Admin deletion endpoint; it must never reach a client
|
||||
or be used for application data or Storage access. The backend must not receive
|
||||
a Supabase anon key, database password, or direct Supabase Storage connection.
|
||||
|
||||
Administrative automation uses a separate opaque credential pair. Configure
|
||||
`ADMIN_API_SERVICE_TOKEN` and `ADMIN_API_SERVICE_USER_ID` on the API together;
|
||||
the UUID must resolve to an active, least-privilege `admin_users` row imported or
|
||||
created in the standalone database. Configure MCP with the same token under the
|
||||
different name `SAUCI_ADMIN_API_TOKEN` plus its independent caller-facing
|
||||
`SAUCI_MCP_API_KEY`. The token does not confer Supabase authority: the API maps
|
||||
it to the configured administrator and applies normal permissions and auditing.
|
||||
Do not give either admin token to mobile, admin browser, or worker containers.
|
||||
|
||||
Secret ownership is intentionally split:
|
||||
|
||||
| Secret | API | Worker | MCP |
|
||||
|---|:---:|:---:|:---:|
|
||||
| Hosted Auth service-role key | yes, Auth Admin only | no | no |
|
||||
| RevenueCat API/webhook secrets | yes | no | no |
|
||||
| Admin API service token | yes | no | yes, as `SAUCI_ADMIN_API_TOKEN` |
|
||||
| OpenRouter/Discord worker credentials | no | yes | no |
|
||||
| PostgreSQL/media signing configuration | yes | yes | no |
|
||||
|
||||
On each deployment, `migrate` waits for PostgreSQL, takes a PostgreSQL advisory
|
||||
lock, applies the committed `apps/api/drizzle` migrations, and exits. The `api`
|
||||
service starts only after migration succeeds. A failed migration therefore
|
||||
fails closed instead of starting new application code against an old schema.
|
||||
The image also exposes the same operation as:
|
||||
|
||||
```sh
|
||||
node dist/db/migrate.js
|
||||
```
|
||||
|
||||
The container runs as the unprivileged `node` user with a read-only filesystem,
|
||||
dropped Linux capabilities, an init process, and bounded graceful shutdown.
|
||||
Docker checks `/health/live`; Dokploy should use `/health/ready` for routing so
|
||||
traffic is admitted only when PostgreSQL is reachable.
|
||||
|
||||
Before the first deployment, provide:
|
||||
|
||||
1. The API hostname and TLS route.
|
||||
2. Persistent Dokploy volumes for `postgres-data` and `media-data`, with one
|
||||
coordinated backup/restore policy covering both.
|
||||
3. The hosted Supabase project URL and expected JWT audience.
|
||||
4. Fresh PostgreSQL credentials stored only in Dokploy secrets.
|
||||
5. A rollback and restore point before any migrated production data is imported.
|
||||
6. A separately generated `MEDIA_SIGNING_SECRET` (at least 32 characters) and
|
||||
the HTTPS API origin as `MEDIA_PUBLIC_BASE_URL`.
|
||||
7. A non-production hosted Auth project and disposable, onboarding-complete test
|
||||
identities mapped into the staging database for authenticated mobile acceptance.
|
||||
|
||||
Blob deletion is retryable. Database cascades and avatar replacement enqueue
|
||||
the server-generated storage key in `media_deletion_queue`; the operations
|
||||
worker removes the file and acknowledges the queue row only after a
|
||||
successful deletion (missing files are treated as already removed). Failed
|
||||
metadata transactions remove the newly written file immediately. Operators
|
||||
should alert on an old/growing deletion queue and include it in database
|
||||
backups; never delete files by constructing paths outside this lifecycle.
|
||||
|
||||
The worker also leases notification, moderation, and optional Discord jobs from
|
||||
`operations_outbox`. Jobs are retried with bounded exponential backoff and are
|
||||
not selected again after five attempts. Alert on rows that remain unsent after
|
||||
five attempts; investigate and replay them explicitly rather than deleting the
|
||||
idempotency record or blindly retrying provider calls.
|
||||
|
||||
Validate configuration without starting anything:
|
||||
|
||||
```sh
|
||||
docker compose --env-file infra/backend/.env.production.local \
|
||||
-f infra/backend/compose.production.yaml config
|
||||
```
|
||||
|
||||
Run the repeatable disposable container check before changing the Dokploy
|
||||
application. It uses hard-coded test-only credentials, creates a unique Compose
|
||||
project, verifies migration/startup/security/shutdown behaviour, and removes its
|
||||
volume on exit:
|
||||
|
||||
```sh
|
||||
infra/backend/verify-production.sh
|
||||
```
|
||||
|
||||
Do not point this Compose project at an existing production database during
|
||||
local validation. Validate against a disposable local volume first. Production
|
||||
cutover additionally requires a stopped-source final sync, row/count checks,
|
||||
legacy-media-reference checks, and authenticated mobile behaviour checks against
|
||||
the non-production fixture; container health alone is not proof of a successful
|
||||
migration.
|
||||
@@ -0,0 +1,125 @@
|
||||
name: sauci-backend
|
||||
|
||||
x-common-environment: &common-environment
|
||||
NODE_ENV: production
|
||||
DATABASE_URL: ${DATABASE_URL:?set DATABASE_URL}
|
||||
MEDIA_ROOT: /data/media
|
||||
MEDIA_SIGNING_SECRET: ${MEDIA_SIGNING_SECRET:?set MEDIA_SIGNING_SECRET}
|
||||
MEDIA_PUBLIC_BASE_URL: ${MEDIA_PUBLIC_BASE_URL:?set MEDIA_PUBLIC_BASE_URL}
|
||||
|
||||
x-api-environment: &api-environment
|
||||
<<: *common-environment
|
||||
PORT: 3003
|
||||
SUPABASE_AUTH_URL: ${SUPABASE_AUTH_URL:?set SUPABASE_AUTH_URL}
|
||||
SUPABASE_AUTH_AUDIENCE: ${SUPABASE_AUTH_AUDIENCE:-authenticated}
|
||||
SUPABASE_AUTH_SERVICE_ROLE_KEY: ${SUPABASE_AUTH_SERVICE_ROLE_KEY:?set SUPABASE_AUTH_SERVICE_ROLE_KEY}
|
||||
REVENUECAT_API_KEY: ${REVENUECAT_API_KEY:?set REVENUECAT_API_KEY}
|
||||
REVENUECAT_ENTITLEMENT_ID: ${REVENUECAT_ENTITLEMENT_ID:-Sauci Pro}
|
||||
REVENUECAT_WEBHOOK_SECRET: ${REVENUECAT_WEBHOOK_SECRET:?set REVENUECAT_WEBHOOK_SECRET}
|
||||
ADMIN_API_SERVICE_TOKEN: ${ADMIN_API_SERVICE_TOKEN:?set ADMIN_API_SERVICE_TOKEN}
|
||||
ADMIN_API_SERVICE_USER_ID: ${ADMIN_API_SERVICE_USER_ID:?set ADMIN_API_SERVICE_USER_ID}
|
||||
ADMIN_PRIVATE_KEY_JWK: ${ADMIN_PRIVATE_KEY_JWK:-}
|
||||
|
||||
x-worker-environment: &worker-environment
|
||||
<<: *common-environment
|
||||
OPENROUTER_API_KEY: ${OPENROUTER_API_KEY:-}
|
||||
CLASSIFIER_ENABLED: ${CLASSIFIER_ENABLED:-true}
|
||||
CLASSIFIER_MODEL: ${CLASSIFIER_MODEL:-openai/gpt-4o}
|
||||
ADMIN_PRIVATE_KEY_JWK: ${ADMIN_PRIVATE_KEY_JWK:-}
|
||||
DISCORD_WEBHOOK_URL: ${DISCORD_WEBHOOK_URL:-}
|
||||
WORKER_POLL_INTERVAL_MS: ${WORKER_POLL_INTERVAL_MS:-30000}
|
||||
WORKER_BATCH_SIZE: ${WORKER_BATCH_SIZE:-25}
|
||||
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:17-alpine
|
||||
environment:
|
||||
POSTGRES_DB: ${POSTGRES_DB:-sauci}
|
||||
POSTGRES_USER: ${POSTGRES_USER:-sauci}
|
||||
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?set POSTGRES_PASSWORD}
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U $${POSTGRES_USER} -d $${POSTGRES_DB}"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 20
|
||||
start_period: 10s
|
||||
volumes:
|
||||
- postgres-data:/var/lib/postgresql/data
|
||||
restart: unless-stopped
|
||||
stop_grace_period: 60s
|
||||
security_opt:
|
||||
- no-new-privileges:true
|
||||
|
||||
migrate:
|
||||
build:
|
||||
context: ../..
|
||||
dockerfile: apps/api/Dockerfile
|
||||
command: ["node", "dist/db/migrate.js"]
|
||||
environment:
|
||||
NODE_ENV: production
|
||||
DATABASE_URL: ${DATABASE_URL:?set DATABASE_URL}
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
restart: "no"
|
||||
init: true
|
||||
read_only: true
|
||||
tmpfs:
|
||||
- /tmp
|
||||
cap_drop:
|
||||
- ALL
|
||||
security_opt:
|
||||
- no-new-privileges:true
|
||||
|
||||
api:
|
||||
build:
|
||||
context: ../..
|
||||
dockerfile: apps/api/Dockerfile
|
||||
environment: *api-environment
|
||||
expose:
|
||||
- "3003"
|
||||
depends_on:
|
||||
migrate:
|
||||
condition: service_completed_successfully
|
||||
restart: unless-stopped
|
||||
init: true
|
||||
read_only: true
|
||||
volumes:
|
||||
- media-data:/data/media
|
||||
tmpfs:
|
||||
- /tmp
|
||||
cap_drop:
|
||||
- ALL
|
||||
security_opt:
|
||||
- no-new-privileges:true
|
||||
stop_grace_period: 15s
|
||||
|
||||
worker:
|
||||
build:
|
||||
context: ../..
|
||||
dockerfile: apps/api/Dockerfile
|
||||
command: ["node", "dist/workers/index.js"]
|
||||
environment: *worker-environment
|
||||
depends_on:
|
||||
migrate:
|
||||
condition: service_completed_successfully
|
||||
restart: unless-stopped
|
||||
# The image health check probes the HTTP API. This process deliberately has
|
||||
# no listener; restart-on-exit is its liveness contract.
|
||||
healthcheck:
|
||||
disable: true
|
||||
init: true
|
||||
read_only: true
|
||||
volumes:
|
||||
- media-data:/data/media
|
||||
tmpfs:
|
||||
- /tmp
|
||||
cap_drop:
|
||||
- ALL
|
||||
security_opt:
|
||||
- no-new-privileges:true
|
||||
stop_grace_period: 30s
|
||||
|
||||
volumes:
|
||||
postgres-data:
|
||||
media-data:
|
||||
@@ -0,0 +1,23 @@
|
||||
name: sauci-local
|
||||
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:17-alpine
|
||||
environment:
|
||||
POSTGRES_DB: sauci
|
||||
POSTGRES_USER: sauci
|
||||
POSTGRES_PASSWORD: sauci_local_only
|
||||
ports:
|
||||
- "127.0.0.1:${SAUCI_POSTGRES_PORT:-54320}:5432"
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U $${POSTGRES_USER} -d $${POSTGRES_DB}"]
|
||||
interval: 2s
|
||||
timeout: 3s
|
||||
retries: 20
|
||||
start_period: 5s
|
||||
volumes:
|
||||
- sauci-postgres-data:/var/lib/postgresql/data
|
||||
restart: unless-stopped
|
||||
|
||||
volumes:
|
||||
sauci-postgres-data:
|
||||
Executable
+49
@@ -0,0 +1,49 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
compose_file="$repo_root/infra/backend/compose.production.yaml"
|
||||
project_name="sauci-deploy-verify-$$"
|
||||
|
||||
# Deliberately fixed disposable values: this check must never consume deployment
|
||||
# secrets or connect to an externally configured database.
|
||||
export POSTGRES_PASSWORD='sauci-container-test-only'
|
||||
export DATABASE_URL='postgresql://sauci:sauci-container-test-only@postgres:5432/sauci'
|
||||
export SUPABASE_AUTH_URL='https://example.supabase.co'
|
||||
export SUPABASE_AUTH_AUDIENCE='authenticated'
|
||||
export SUPABASE_AUTH_SERVICE_ROLE_KEY='container-test-auth-admin-only'
|
||||
export ADMIN_API_SERVICE_TOKEN='container-test-admin-service-token-123456789'
|
||||
export ADMIN_API_SERVICE_USER_ID='00000000-0000-4000-8000-000000000002'
|
||||
export REVENUECAT_API_KEY='container-test-revenuecat-only'
|
||||
export REVENUECAT_WEBHOOK_SECRET='container-test-webhook-only'
|
||||
export MEDIA_SIGNING_SECRET='sauci-media-container-test-only-secret'
|
||||
export MEDIA_PUBLIC_BASE_URL='https://api.sauci.test'
|
||||
export CLASSIFIER_ENABLED='false'
|
||||
|
||||
cleanup() {
|
||||
docker compose -p "$project_name" -f "$compose_file" down --volumes --remove-orphans >/dev/null 2>&1 || true
|
||||
}
|
||||
trap cleanup EXIT
|
||||
|
||||
docker compose -p "$project_name" -f "$compose_file" config >/dev/null
|
||||
docker compose -p "$project_name" -f "$compose_file" up -d --build --wait
|
||||
|
||||
docker compose -p "$project_name" -f "$compose_file" exec -T api node -e \
|
||||
"Promise.all(['/health/live','/health/ready','/v1/me'].map(async p=>[p,(await fetch('http://127.0.0.1:3003'+p)).status])).then(results=>{const expected=[200,200,401];if(results.some((result,index)=>result[1]!==expected[index])){console.error(results);process.exit(1)};console.log(results)})"
|
||||
|
||||
docker compose -p "$project_name" -f "$compose_file" run --rm migrate
|
||||
docker compose -p "$project_name" -f "$compose_file" exec -T api sh -c 'test "$(id -u)" != 0'
|
||||
docker compose -p "$project_name" -f "$compose_file" exec -T worker sh -c 'test "$(id -u)" != 0'
|
||||
docker compose -p "$project_name" -f "$compose_file" exec -T api sh -c 'touch /data/media/write-check && rm /data/media/write-check'
|
||||
|
||||
if docker compose -p "$project_name" -f "$compose_file" exec -T api sh -c 'touch /app/should-not-write'; then
|
||||
echo 'API filesystem unexpectedly allowed writes' >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
docker compose -p "$project_name" -f "$compose_file" stop api
|
||||
docker compose -p "$project_name" -f "$compose_file" logs --no-color api | grep -q 'Received SIGTERM; shutting down'
|
||||
docker compose -p "$project_name" -f "$compose_file" stop worker
|
||||
docker compose -p "$project_name" -f "$compose_file" logs --no-color worker | grep -q 'stopping operations worker'
|
||||
|
||||
echo 'Production container verification passed'
|
||||
Reference in new issue
Block a user