edr-platform/DEPLOYMENT.md

# Deployment Runbook

This document explains how deployments work for the EDR platform using Docker, GitHub Actions, and self-hosted runners.

## Overview

- Monorepo contains 6 deployable services:
  - `freight-api`
  - `freight-portal`
  - `freight-backoffice`
  - `passenger-api`
  - `passenger-portal`
  - `passenger-backoffice`
- Deployments run through one workflow: `.github/workflows/deploy.yml`
- Each service is built/deployed independently in parallel (matrix jobs).
- Docker Compose project names are branch-aware to avoid environment collisions on the same host.

## Prerequisites

- Docker Engine with Compose plugin on the self-hosted runner.
- GitHub self-hosted runner registered for this repository.
- Repository secret configured:
  - `NPM_TOKEN` (for private `@tria-plc/*` package install during Docker build)
- Server-side env files created for each branch/environment.

## Server Environment Files

`sync-env-from-server.sh` reads env files from:

`/home/<DEPLOY_USER>/environment/edr/<branch-slug>/<project>/`

Where:

- `<DEPLOY_USER>` defaults to `tria` (overridable by `DEPLOY_USER`)
- `<branch-slug>` is derived from Git branch (lowercase, non-alphanumeric replaced with `-`)
- `<project>` is `edr-freight` or `edr-passenger`

### Required files per project

For `edr-freight`:

- `freight-api.env`
- `freight-portal.env`
- `freight-backoffice.env`
- optional: `freight-web.build.env`

For `edr-passenger`:

- `passenger-api.env`
- `passenger-portal.env`
- `passenger-backoffice.env`
- optional: `passenger-web.build.env`

### Required env key

Each service env file must contain:

- `PORT=<number>`

The sync script validates this and fails if missing.

### Build env files (optional)

Used for additional build-time variables (example: Vite API URL for freight web), with `export` syntax:

```bash
export FREIGHT_VITE_API_URL=https://freight-api.example.com/api
```

These are injected into `GITHUB_ENV` during workflow execution.

> **Passenger web:** `NEXT_PUBLIC_API_URL` does **not** need a separate build env file. Place it directly in the service runtime env file (`passenger-portal.env` / `passenger-backoffice.env`) and the sync script will forward it to the build automatically.

## Docker Compose Port Mapping

`docker-compose.yaml` uses per-service env variables for host/container port mappings:

- `FREIGHT_API_PORT`
- `PASSENGER_API_PORT`
- `FREIGHT_PORTAL_PORT`
- `FREIGHT_BACKOFFICE_PORT`
- `PASSENGER_PORTAL_PORT`
- `PASSENGER_BACKOFFICE_PORT`

`scripts/deploy/sync-env-from-server.sh` extracts `PORT` from each synced `.env` and exports the corresponding `*_PORT` variable to `GITHUB_ENV`.

## Passenger Web Docker Configuration

The passenger web apps (portal and backoffice) are deployed as **Next.js applications** using a dedicated Dockerfile:

- Dockerfile: `infrastructure/docker/Dockerfile.passenger-web`
- Apps: `apps/edr-passenger-web/portal` and `apps/edr-passenger-web/backoffice`

### Key differences from freight-web

| Aspect | Freight Web | Passenger Web |
| --- | --- | --- |
| Framework | Vite (SPA) | Next.js (SSR/SSG) |
| Deployment | Static export + nginx | Node.js server |
| Dockerfile | `Dockerfile.web` | `Dockerfile.passenger-web` |
| Final port (container) | 80 (nginx) | driven by `PORT` in service `.env` |
| Build arg | `TURBO_FILTER` | `APP_PACKAGE` + `APP_PATH` + `NEXT_PUBLIC_API_URL` |

### Build arguments

The Dockerfile accepts the following build args:

- `APP_PACKAGE`: Turbo package filter (e.g., `@edr/passenger-portal`)
- `APP_PATH`: App directory path (e.g., `apps/edr-passenger-web/portal`)
- `NEXT_PUBLIC_API_URL`: API URL visible to browser — sourced from `NEXT_PUBLIC_API_URL` in the service `.env` file

### Port mapping

Both host and container ports are driven by `PORT` in the service env file. The sync script reads `PORT`, exports `PASSENGER_PORTAL_PORT` / `PASSENGER_BACKOFFICE_PORT` to `GITHUB_ENV`, and `docker-compose.yaml` uses those variables for both sides of the mapping:

```
${PASSENGER_PORTAL_PORT:-5174}:${PASSENGER_PORTAL_PORT:-5174}
```

This ensures `docker ps` shows `0.0.0.0:<port>-><port>/tcp` with matching ports.

### Runtime

The final image uses Next.js `output: 'standalone'` and runs:

```bash
node server.js
```

Next.js reads `PORT` from the runtime environment (supplied via `env_file` in docker-compose). The standalone output bundles only the required `node_modules`, producing a significantly smaller image than a full `pnpm deploy`.

## Database consolidation runbook (single schema-separated database)

The platform runs on **one** Postgres database, separated by schema — `iam`, `freight`,
`passenger`, `edr_payment`, `audit`. This is the production topology (the
`dump-smart_office_prod-*.sql` dump is the `freight` schema of a Smart Office database),
and dev/local configs now match it.

Environments that predate this ran freight on its own server (`5433`, database
`edr_freight`) with a **second copy of the `iam` schema**, while passenger and payment used
another database. Use this sequence to collapse them. It is a data move — the config
changes alone do not migrate a single row.

**Authoritative copy: `edr_freight`'s `iam`.** It is the copy freight-api's
`iam:migration:run|show|revert` scripts have been applying migrations to, so its schema is
the most current. The other copy's rows are backfilled into it, never the reverse.

### 1. Snapshot both sources

```bash
pg_dump -Fc -h <old-freight-host> -p 5433 -d edr_freight     -f pre-consolidation-freight.dump
pg_dump -Fc -h <old-shared-host>  -p 5432 -d edr_database    -f pre-consolidation-shared.dump
```

### 2. Create the target database and schemas

Under compose this is automatic (`infrastructure/docker/initdb/01-schemas.sql`). Against an
existing server:

```sql
CREATE DATABASE edr_database;
\connect edr_database
CREATE SCHEMA IF NOT EXISTS iam;
CREATE SCHEMA IF NOT EXISTS freight;
CREATE SCHEMA IF NOT EXISTS passenger;
CREATE SCHEMA IF NOT EXISTS edr_payment;
CREATE SCHEMA IF NOT EXISTS audit;
```

### 3. Restore the authoritative IAM first, then freight

```bash
# iam — including iam.typeorm_migrations, so freight-api does not re-run applied migrations
pg_restore -d edr_database -n iam     pre-consolidation-freight.dump
# freight — including freight.migrations
pg_restore -d edr_database -n freight pre-consolidation-freight.dump
```

### 4. Restore the other domains

```bash
pg_restore -d edr_database -n passenger   pre-consolidation-shared.dump
pg_restore -d edr_database -n edr_payment pre-consolidation-shared.dump
```

Do **not** restore the second `iam` schema over the first. Back its rows in instead:
`iam.users.username`, `.email` and `.phone_number` are all `UNIQUE`, so the two copies
reconcile on those columns. Insert only users present in the secondary copy and absent from
the authoritative one, and record the old→new id mapping — anything that stored the
secondary copy's user ids (audit rows, `*_user_id` columns in `passenger`) must be remapped
with it. Cross-schema references are soft UUIDs with no FK to catch a miss.

### 5. Repoint the applications

| App | Variables | Value |
| --- | --- | --- |
| `edr-freight-api` | `DB_HOST` `DB_PORT` `DB_USER` `DB_PASSWORD` `DB_NAME` | the consolidated database |
| `edr-gps-tracker` | same `DB_*` | same |
| `edr-payment-api` | same `DB_*` (+ `DB_SCHEMA=edr_payment`) | same |
| `edr-passenger-api` | `DATABASE_URL` (`?schema=passenger`) **and** `DATABASE_HOST/PORT/NAME/USER/PASSWORD` (+ `DATABASE_SCHEMA=iam`) | same |

Both of passenger-api's connections must resolve to this one database — its Prisma URL and
its read-only TypeORM IAM connection are configured separately and can silently diverge.

### 6. Verify before opening traffic

```sql
-- every expected schema present
SELECT nspname FROM pg_namespace WHERE nspname IN
  ('iam','freight','passenger','edr_payment','audit');
-- migration histories carried over: no re-runs, no gaps
SELECT count(*) FROM iam.typeorm_migrations;   -- matches the source count
SELECT count(*) FROM freight.migrations;       -- matches the source count
-- no duplicate humans after the IAM backfill
SELECT username, count(*) FROM iam.users GROUP BY username HAVING count(*) > 1;
```

Then run freight-api's migration step (`docker build --target migration`) and confirm it
applies **zero** new migrations — a non-zero count means step 3 dropped a history table.

### Out of scope

The e2e harnesses stay hermetic and are deliberately untouched: `edr_freight_e2e` on 5533
(`e2e/freight/`) and the passenger test database on 5544 (`e2e/docker-compose.yml`,
`apps/edr-passenger-api/.env.test`).

## Rollback Procedure

Each build is tagged with the short git SHA (`${COMPOSE_PROJECT_NAME}-<service>:<sha8>`).

### Rollback a single service

```bash
# 1. Find the last known-good image tag
docker images | grep passenger-api

# 2. Re-tag it as the current image
docker tag edr-passenger-main-passenger-api:<previous-sha> edr-passenger-main-passenger-api:latest

# 3. Restart the container from the previous image
docker compose --project-name edr-passenger-main up -d passenger-api --force-recreate
```

### Rollback via re-run

Alternatively, trigger a `workflow_dispatch` on the last known-good commit SHA from the GitHub Actions UI — this rebuilds and redeploys that exact commit.

## Production Security Checklist

Before deploying to production, verify:

- [ ] `JWT_SECRET`, `JWT_ACCESS_TOKEN_SECRET`, `JWT_REFRESH_TOKEN_SECRET` are set to random 32+ char strings (`openssl rand -hex 32`)
- [ ] `DATABASE_URL` includes `?sslmode=require&connection_limit=10`
- [ ] `WAAFI_INSECURE_TLS` is `false` (app will refuse to start if `true` in production)
- [ ] `NODE_ENV=production` is set
- [ ] `GITHUB_PACKAGE_TOKEN` is a scoped read-only token, not a personal admin token
- [ ] No `.env` files are committed to the repository (`git status` should show none)

## Data Retention Policy

The `TasksService` runs a daily purge cron at 02:00 EAT that automatically deletes:

| Table | Retention |
|---|---|
| `OtpCode` | 1 hour after expiry or verification |
| `FaydaVerificationSession` | 1 hour after expiry or completion |
| `AuditLog` | 365 days |
| `PaymentWebhookEvent` | 90 days |
| `GateValidationLog` | 180 days |

No manual intervention is required. Monitor the `TasksService` log output for purge counts.

## GitHub Actions Deployment Flow

Workflow file: `.github/workflows/deploy.yml`

### 1) `prepare` job

- Checks out repository once.
- Creates workspace artifact (`workspace.tgz`) and uploads it.

### 2) `deploy` matrix job (parallel)

For each service:

- Downloads and extracts workspace artifact.
- Syncs that service env file from server path.
- Computes branch slug and sets:
  - `COMPOSE_PROJECT_NAME=<project>-<branch-slug>`
- Creates `.npmrc`/`.npmrc_temp` from `NPM_TOKEN`.
- For `passenger-api` and `payment-api`: builds and runs the migration image as a gated step before the app image.
- Builds the service image and tags it with the short git SHA.
- Runs `docker compose up -d <service> --force-recreate`.
- For API services: polls `GET /health/ready` every 10s for up to 120s. Fails the job if the service does not become healthy.
- Cleans `.npmrc`/`.npmrc_temp`.

## Branch/Environment Isolation

Compose project name is generated as:

`<project>-<branch-slug>`

Examples:

- `edr-freight-main`
- `edr-freight-staging`
- `edr-passenger-dev`

This prevents container/network/volume name collisions between branches.

## Local Manual Deployment (Optional)

From repo root:

```bash
DOCKER_BUILDKIT=1 docker compose build <service>
docker compose up -d <service>
```

If private packages are required locally, create `.npmrc`:

```bash
cat <<EOF > .npmrc
@tria-plc:registry=https://npm.pkg.github.com
//npm.pkg.github.com/:_authToken=<YOUR_TOKEN>
always-auth=true
EOF
```

## Passenger API Startup Behavior

Passenger container entrypoint runs on startup:

1. `npm run prisma:generate`
2. `npm run prisma:migrate` (deploy mode)
3. `npm run prisma:seed`
4. starts API process

## Troubleshooting

### Missing env file

Error:

- `Missing env file: ...`

Fix:

- Create the required file in the server env directory for that project/branch slug.

### Missing PORT in env file

Error:

- `Missing required PORT in env file: ...`

Fix:

- Add `PORT=<number>` to that service env file.

### Private package install fails

Check:

- `NPM_TOKEN` exists in repo secrets.
- Workflow created `.npmrc` successfully.

### Prisma seed/migrate failures (passenger)

Check:

- `DATABASE_URL` in `passenger-api.env`
- DB reachability from runner host/container network
- migration history consistency