diff --git a/.claude/skills/edr-db/SKILL.md b/.claude/skills/edr-db/SKILL.md new file mode 100644 index 000000000..47333128d --- /dev/null +++ b/.claude/skills/edr-db/SKILL.md @@ -0,0 +1,42 @@ +--- +name: edr-db +description: Query, EXPLAIN-validate, and inspect the remote EDR freight dev database. Use whenever you need to check data, verify a raw SQL statement before shipping it, list a table's columns, check schema drift, or see which migrations are recorded. psql is NOT installed on this machine — this runner is the sanctioned path. Triggers - "check the db", "query edr_dev", "does column X exist", "validate this SQL", "is migration recorded", "seed check", diagnosing a 400/500 whose cause may be data or schema. +--- + +# EDR dev-DB runner + +One script, runs from anywhere in the repo (resolves `pg` from `apps/edr-freight-api`): + +```bash +node .claude/skills/edr-db/query.cjs "SELECT ... " # run SQL, console.table output +node .claude/skills/edr-db/query.cjs explain "SELECT ..." # EXPLAIN-validate only (no rows touched) +node .claude/skills/edr-db/query.cjs columns # freight.
column list +node .claude/skills/edr-db/query.cjs migrations [like] # public.migrations rows (newest first) +node .claude/skills/edr-db/query.cjs drift
# bare column names, for diffing vs the entity +``` + +Connection comes from `DB_HOST` / `DB_PORT` / `DB_USER` / `DB_PASSWORD` / `DB_NAME`, +defaulting to the shared dev database (`edr_dev`). + +## Rules that go with it + +- **HARD RULE: every raw SQL statement you write into a service must pass + `explain` here before you ship it.** A typo'd column is a runtime 500 the + type-checker cannot catch. +- Never assume a recorded migration applied — check `migrations ` **and** + `columns
` together. Recorded-but-absent = schema drift; fix with a + NEW repair migration (idempotent DDL, no-op `down()`), never by editing the + recorded one. +- Writes to dev data are fine for seeding/diagnosis but keep them idempotent + (`WHERE NOT EXISTS` guards) — watch-mode API instances race `migrationsRun`, + and non-idempotent statements have double-run here before. +- Timestamps for new migrations: must be unique across `src/migrations/` AND + higher than `SELECT max(timestamp) FROM public.migrations`. + +## Diagnosing a pasted 400/500 (the recurring loop) + +1. Find the route: grep the path segment in `apps/edr-freight-api/src/modules/*/**.controller.ts`. +2. Read the service method — list its guard `throw`s. Most "bugs" are a guard + working as designed (handover unsigned, fee unpaid, not PAID, wrong direction). +3. Check the actual DB state for that record with this runner. +4. Only then decide: guard doing its job (fix the UI affordance) vs real defect. diff --git a/.claude/skills/edr-db/query.cjs b/.claude/skills/edr-db/query.cjs new file mode 100644 index 000000000..b4797bd45 --- /dev/null +++ b/.claude/skills/edr-db/query.cjs @@ -0,0 +1,88 @@ +#!/usr/bin/env node +/** + * Dev-DB runner for the EDR freight database. psql is NOT installed on this + * machine; this is the sanctioned way to query, EXPLAIN-validate, and inspect + * the remote dev DB. Resolves `pg` from apps/edr-freight-api so it runs from + * anywhere in the repo. + * + * node .claude/skills/edr-db/query.cjs "SELECT ... " run SQL (console.table) + * node .claude/skills/edr-db/query.cjs explain "SELECT..." EXPLAIN-validate only + * node .claude/skills/edr-db/query.cjs columns
list freight.
columns + * node .claude/skills/edr-db/query.cjs migrations [like] public.migrations rows + * node .claude/skills/edr-db/query.cjs drift
columns vs entity check helper + * + * Connection: DB_HOST/DB_PORT/DB_USER/DB_PASSWORD/DB_NAME env vars, falling + * back to the shared dev database. + */ +const path = require('path'); +const { createRequire } = require('module'); + +const repoRoot = path.resolve(__dirname, '..', '..', '..'); +const apiRequire = createRequire( + path.join(repoRoot, 'apps', 'edr-freight-api', 'package.json'), +); +const { Client } = apiRequire('pg'); + +const cfg = { + host: process.env.DB_HOST ?? '10.18.7.207', + port: parseInt(process.env.DB_PORT ?? '5432', 10), + user: process.env.DB_USER ?? 'postgres', + password: process.env.DB_PASSWORD ?? 'dcba@1234', + database: process.env.DB_NAME ?? 'edr_dev', +}; + +const [, , first, ...rest] = process.argv; + +async function main() { + if (!first) { + console.error('usage: query.cjs "" | explain "" | columns
| migrations [like] | drift
'); + process.exit(2); + } + const c = new Client(cfg); + await c.connect(); + try { + if (first === 'columns') { + const r = await c.query( + `SELECT column_name, data_type, is_nullable, column_default + FROM information_schema.columns + WHERE table_schema='freight' AND table_name=$1 + ORDER BY ordinal_position`, + [rest[0]], + ); + console.table(r.rows); + } else if (first === 'migrations') { + const like = rest[0] ? `%${rest[0]}%` : '%'; + const r = await c.query( + `SELECT id, timestamp, name FROM public.migrations + WHERE name ILIKE $1 ORDER BY id DESC LIMIT 40`, + [like], + ); + console.table(r.rows); + } else if (first === 'drift') { + // Quick drift signal: DB columns for the table. Compare by eye against + // the entity's @Column names; a recorded-but-absent column = drift. + const r = await c.query( + `SELECT column_name FROM information_schema.columns + WHERE table_schema='freight' AND table_name=$1 ORDER BY column_name`, + [rest[0]], + ); + console.log(r.rows.map((x) => x.column_name).join('\n')); + } else if (first === 'explain') { + await c.query('EXPLAIN ' + rest.join(' ')); + console.log('OK — statement is valid against', cfg.database); + } else { + const sql = [first, ...rest].join(' '); + const started = Date.now(); + const r = await c.query(sql); + if (Array.isArray(r.rows) && r.rows.length) console.table(r.rows); + console.log(`${r.rowCount ?? 0} row(s), ${Date.now() - started}ms`); + } + } finally { + await c.end(); + } +} + +main().catch((e) => { + console.error('FAIL:', e.message); + process.exit(1); +}); diff --git a/.claude/skills/standup/SKILL.md b/.claude/skills/standup/SKILL.md new file mode 100644 index 000000000..272f280e7 --- /dev/null +++ b/.claude/skills/standup/SKILL.md @@ -0,0 +1,40 @@ +--- +name: standup +description: Produce the work report Hagernesh asks for - "what have I done today", "tasks of yesterday and today", daily/period summaries for tickets or timesheets. Builds the answer from git history plus uncommitted work, never from memory alone. +--- + +# Work report (standup / ticket summary) + +Ground every line in git. Do not reconstruct from conversation memory — commits +are the record. + +## Gather + +```bash +# Commits in the window (adjust dates; author matches "Hagernesh") +git log --since="YYYY-MM-DD 00:00" --until="YYYY-MM-DD 00:00" --author=Hagernesh \ + --pretty=format:"%h|%ad|%s" --date=short + +# What each commit actually contains (subjects lie sometimes) +git show --stat --pretty=format:"%s" | head -12 + +# In-flight work = part of "today" even if uncommitted +git status --short +git log origin/dev..HEAD --oneline # branch commits not yet in dev +``` + +## Known pitfalls in this repo + +- **Check subjects against contents.** Commit titles here sometimes mismatch the + diff (e.g. a commit titled "unload export" that actually contained ISO + container validation). Use `git show --stat` before reporting a title as fact. +- A day with no commits usually still has uncommitted/in-flight work — report it + as its own section with per-item status (done / uncommitted / blocked). +- Merge commits from other authors are noise; filter with `--author`. + +## Output format + +One table per day: `# | Task (plain language, not the commit subject verbatim) | +Commit / Status`. Follow with a short "carry-over / blocked" list naming what +blocks each item. Keep it ticket-ready: no jargon that needs the repo open to +decode. diff --git a/.claude/skills/verify/SKILL.md b/.claude/skills/verify/SKILL.md new file mode 100644 index 000000000..a2c715db4 --- /dev/null +++ b/.claude/skills/verify/SKILL.md @@ -0,0 +1,59 @@ +--- +name: verify +description: Project definition-of-done runner for the EDR platform. Use before calling any code change finished, before committing, and whenever asked "is it done / does it work". Runs the targeted checks that actually catch this repo's failure modes - type-check with turbo filters, @edr/types dist rebuild, raw-SQL EXPLAIN validation, migration safety, and honest test reporting. +--- + +# Verify a change (EDR definition of done) + +Run these in order. Report which you ran and what each said — never call +unverified work done. + +## 1. Type-check exactly what you touched + +```bash +pnpm turbo type-check --filter=@edr/freight-api --filter=@edr/freight-backoffice --filter=@edr/freight-portal +``` + +Drop filters you didn't touch; whole-repo runs waste minutes. **If you edited +`packages/types`, rebuild it FIRST** — consumers read its `dist/`, not `src/`: + +```bash +pnpm turbo build --filter=@edr/types +``` + +## 2. Validate every raw SQL statement + +Each new/edited `dataSource.query` / `manager.query` string must pass: + +```bash +node .claude/skills/edr-db/query.cjs explain "" +``` + +## 3. Migration checklist (if you added one) + +- Timestamp unique in `src/migrations/` **and** greater than + `node .claude/skills/edr-db/query.cjs "SELECT max(timestamp) FROM public.migrations"`. +- DDL idempotent (`IF NOT EXISTS`, guarded backfills). +- Watch-mode reload does NOT run migrations — apply the SQL to the dev DB + yourself or fully restart the API, then confirm with + `query.cjs columns
`. + +## 4. Tests — honest bar + +`pnpm test` for `@edr/freight-api` is currently red on `dev`, so a green suite +is not the bar. The bar: run the specs nearest what you touched and introduce +**no new failure**. If you touched a service constructor, update its `.spec.ts` +mocks (constructor-arity breaks are this repo's most common test regression). + +## 5. Observe the behaviour + +Compiling is not working. Hit the endpoint, drive the UI flow, or query the +resulting rows. If you genuinely could not observe it, say so explicitly in the +summary — do not imply it was seen working. + +## 6. Before commit + +- Conventional message (`fix(warehouses): …`). Git hooks do NOT run in this + repo (husky shims exist but no user hooks) — nothing will catch it for you. +- Lint the files you touched if in doubt: `pnpm turbo lint --filter=`. +- Do not commit or push unless the user asked. diff --git a/CLAUDE_NEW.md b/CLAUDE_NEW.md new file mode 100644 index 000000000..11075e9f0 --- /dev/null +++ b/CLAUDE_NEW.md @@ -0,0 +1,294 @@ +# EDR Platform — Developer Guide + +> This file is the contract. If something here contradicts the code, the code is the +> truth and this file is a bug — fix it in the same PR. + +## Overview + +Monorepo for the Ethio Djibouti Railway (EDR) digital platform. Contains the Freight +Management and Passenger Management applications, a payment microservice, plus shared +types, NestJS utilities, and React component libraries. + +The freight domain is the largest and most active area. Its core flow is: +**booking → receive to warehouse → store → load onto train → dispatch → arrive → unload +→ customer truck (self-haul) or EDR last mile → handover → exit paper → delivered.** +Fees (storage, demurrage, double handling, truck detention) and allocation rules +(warehouse/yard/zone) hang off the warehouse stage. + +## Apps + +| App | Package name | Purpose | Default port | +| ------------------------------ | --------------------------- | -------------------------------------------------- | ------------ | +| `edr-freight-api` | `@edr/freight-api` | NestJS API for freight management | 3001 | +| `edr-freight-web/portal` | `@edr/freight-portal` | React frontend for freight customer/portal users | 5173 | +| `edr-freight-web/backoffice` | `@edr/freight-backoffice` | React frontend for freight backoffice employees | 5183 | +| `edr-passenger-api` | `@edr/passenger-api` | NestJS API for passenger management | 3002 | +| `edr-payment-api` | `@edr/payment-api` | NestJS payment microservice (intents, webhooks) | 3003 | +| `edr-passenger-web/portal` | `@edr/passenger-portal` | React frontend for passenger customer/portal users | 5174 | +| `edr-passenger-web/backoffice` | `@edr/passenger-backoffice` | React frontend for passenger backoffice employees | 5184 | + +`edr-freight-web` and `edr-passenger-web` are grouping folders, not workspace packages. +Each holds a `portal/` and `backoffice/` sub-app, both independent pnpm workspace +packages (see `pnpm-workspace.yaml`). + +`apps/edr-landing/` exists on disk but has **no `package.json`** — it is not a workspace +package and is not built, linted, or type-checked. Leave it alone unless asked. + +## Packages + +| Package | Purpose | +| ---------------------- | ---------------------------------------------------------------------------------- | +| `@edr/types` | Shared TypeScript interfaces and enums | +| `@edr/api-common` | Shared NestJS decorators, filters, interceptors, pipes, BaseEntity, BaseRepository | +| `@edr/ui-common` | Shared React components and theme | +| `@edr/eslint-config` | Shared ESLint configurations (base/nestjs/react) | +| `@edr/tsconfig` | Shared TypeScript configurations | +| `@edr/prettier-config` | Shared Prettier configuration | + +**`@edr/types` is consumed as its built `dist/`** (`main: ./dist/index.js`). Editing a +type in `packages/types/src` changes nothing for consumers until you rebuild: + +```bash +pnpm turbo build --filter=@edr/types +``` + +If a type-check fails on a field you just added to `@edr/types`, this is why. + +## Commands + +| Command | Description | +| --------------------------- | ---------------------------------------- | +| `pnpm install` | Install all workspace dependencies | +| `pnpm dev` | Run every app in dev mode | +| `pnpm dev:freight` | Freight API + portal + backoffice | +| `pnpm dev:freight:api` | Freight API only | +| `pnpm dev:freight:portal` | Freight portal only | +| `pnpm dev:freight:backoffice` | Freight backoffice only | +| `pnpm dev:passenger` | Passenger API + web | +| `pnpm dev:payment` | Payment API | +| `pnpm build` | Build every package and app | +| `pnpm test` | Run all tests (turbo) | +| `pnpm lint` | Lint everything | +| `pnpm type-check` | Type-check every package | +| `pnpm format` | Format all files with Prettier | + +Prefer targeted turbo filters over whole-repo runs — they are minutes faster: + +```bash +pnpm turbo type-check --filter=@edr/freight-api --filter=@edr/freight-backoffice +``` + +`apps/edr-freight-api` also carries many `seed:*` scripts (demo bookings, wagons, trains, +gate-pass scenarios). Read the script before running one; several write real rows. + +## Environment & database + +- Postgres is **external**. There is no postgres service in `docker-compose.yaml`, and + no port `5433`/`5434` is published anywhere in the repo. +- Freight API connection comes from `DB_HOST`, `DB_PORT`, `DB_USER`, `DB_PASSWORD`, + `DB_NAME` (defaults: `localhost:5433`, `edr_freight`). Development points these at a + remote database. +- The connection sits behind a **connection pooler**. Do **not** pass + `extra.options: '-c search_path=…'` — the pooler rejects it with + `08P01 unsupported startup parameter in options: search_path`. `search_path` is applied + per-connection in a pool `connect` handler instead. See + `apps/edr-freight-api/src/config/database.config.ts` before touching connection options. +- Each app owns its own database. **No cross-database joins**; cross-domain data flows + through API calls or message queues. +- `psql` is not installed on the dev machine. To query the database, write a short Node + script using the `pg` client and run it from `apps/edr-freight-api` (where `pg` resolves). + +## Hard rules + +These are non-negotiable. Everything else is a strong default. + +- **pnpm only.** Never run `npm install` or `yarn`. +- **TypeScript strict mode** is on in every package and app. Do not weaken it, and do not + reach for `any` to make an error go away. +- **Never `synchronize: true`.** Not in production, not anywhere. It is currently `false` + in every config and it has already corrupted this database twice (see *Migrations*). + All schema changes go through TypeORM migrations. +- **All entities** use UUID primary keys (`@PrimaryGeneratedColumn('uuid')`). +- **All entities** extend `BaseEntity` from `@edr/api-common` — `createdAt`, `updatedAt`, + `deletedAt` (soft delete). +- **All columns** are `snake_case` in the database (`@Column({ name: 'snake_case' })`); + TypeScript properties are `camelCase`. +- **Controllers contain no business logic.** They validate, delegate, and shape the response. +- **Conventional commits.** `fix(warehouses): …`, `feat(bookings): …`. +- **Do not commit or push unless asked.** Propose the change; let the human decide when it lands. +- **Do not break working behaviour to add new behaviour.** When a fix is risky, say so and + offer the safe version. + +## Architecture + +### NestJS module shape + +`module → controller → service → repository`, with `entities/` and `dto/` alongside. + +### Data access — the real model + +There are two sanctioned ways to read and write, and you must pick the right one: + +1. **Entity CRUD → the custom repository class.** Extends `BaseRepository` from + `@edr/api-common`. Services inject the repository class, never `Repository` directly. +2. **Read projections, queue endpoints, cross-table reports → raw SQL** via + `this.dataSource.query(...)` or `manager.query(...)` inside a transaction. + +Raw SQL is normal here, not a smell — the warehouse and scheduling modules are built on it. +It carries one obligation: + +> **HARD RULE — validate every raw SQL statement against a real database before you ship it.** +> A typo'd column name is a runtime 500 that no type-checker will catch. Run it through +> `EXPLAIN` against the dev database. Column drift is real (see *Migrations*). + +Writes inside a transaction use `manager.getRepository(Entity)`, not the injected repository, +so they join the caller's transaction. + +**Never do slow I/O inside a database transaction.** Queue the work and fan it out after +commit. An SMS awaited inside a transaction once held capacity locks open for the whole +gateway timeout. Any outbound HTTP call must set an explicit `timeout` — axios defaults to +no timeout and will wait forever. + +### Migrations + +Migrations are the most dangerous surface in this repo. Two production-grade incidents have +already come from it. + +- `migrationsRun: true` — **migrations run automatically on API boot**, with + `migrationsTransactionMode: 'each'`. +- Consequences you must design for: + - Running several `nest start --watch` instances races `migrationsRun`. A non-idempotent + data migration can execute twice. Keep one instance. + - A watch-mode hot reload does **not** re-run migrations. If you add a column that new + code reads, apply it to the dev database yourself (idempotently) or fully restart. +- **Give every migration a unique timestamp.** 34 timestamps are currently shared by two or + more migrations. TypeORM orders by timestamp and breaks ties non-deterministically. Before + adding one, check the filename prefix is unused *and* higher than the newest recorded row. +- **Write idempotent DDL**: `ADD COLUMN IF NOT EXISTS`, `CREATE INDEX IF NOT EXISTS`, and + backfills guarded by `WHERE col IS NULL`. +- **Never assume a recorded migration actually applied.** `AddGrnNumberToWarehouseInventory` + was recorded in `migrations` while its column was absent — it had been dropped out of band. + TypeORM will never re-run a recorded migration, so the fix is a *new repair migration*. +- **A repair migration's `down()` should be a no-op.** Reverting a repair must not + re-introduce the outage it fixed. + +### Auth + +Auth **is implemented in this repo.** Do not add TODO stubs, and do not write your own. + +- `@CurrentUser()` (`@edr/api-common`) is a real `createParamDecorator`, not a metadata stub. +- Route protection uses `@UseGuards(JwtGuard)` and `@UseGuards(PermissionGuard([...]))`. +- Freight-domain checks use `hasFreightPermission(user, FREIGHT_PERMS..)`. +- Permissions are declared in `apps/edr-freight-api/src/seed/freight-permissions.registry.ts`. + Add a permission there before referencing it. +- IAM has its own migrations, run ahead of freight migrations from the same data source, and + its own CLI scripts (`iam:migration:run`, `iam:seed:run`). + +Ownership checks are separate from permission checks. A staff user passes +`hasFreightPermission`; a customer must additionally pass an ownership assertion such as +`assertCustomerCanAccessBooking`. Do not drop the ownership check because the permission check passed. + +## Frontend conventions + +- The web apps use **Mantine v9**. Its APIs differ from v6/v7 — check the installed version + before copying a snippet. +- `@edr/ui-common` holds shared components and theme; it is imported in ~94 files across the + freight web apps. Prefer it over re-implementing a component. +- **Blob downloads need the async error decoder.** A request with `responseType: 'blob'` + delivers the JSON error body as a `Blob`, so the synchronous `extractErrorMessage` finds no + `.message` and degrades to `"Request failed with status code 400"`. Use + `await extractDownloadErrorMessage(error)` in every PDF/blob catch block. Mutation catches + keep the synchronous version — their bodies are already parsed JSON. +- Server-side guards must be reflected in the UI. If the API will reject the action, the + button should be disabled, hidden, or explain the blocker — not fire and surface a 400. +- Prefer disabling a control with a visible reason over silently hiding it. + +## Notifications + +In-app notifications resolve recipients from the company's **linked portal users**. If a +company has none, `notify()` logs `0 recipients — skipped` and stores nothing, with no error. +SMS and email still send, because they address the company's phone and email directly. Check +this before debugging a "missing notification". + +## PDF generation + +Chromium is not installed in every environment. PDF paths must fall back to the hand-rolled +generators (`styled-pdf.util.ts`, `buildFallbackPdf`, `buildTabularFallbackPdf`) rather than +assume a headless browser exists. + +## Adding a new module to a NestJS app + +1. Create `modules//` with `entities/`, `dto/`, and the four + `.{module,controller,service,repository}.ts` files. +2. The entity extends `BaseEntity` from `@edr/api-common`. +3. The repository extends `BaseRepository` from `@edr/api-common`. +4. The service injects the repository class (not `Repository` directly). +5. The controller uses `@ApiTags()` + `@ApiOperation()` for Swagger, and guards the route. +6. Register the module in the app's `app.module.ts`. + +## Adding a new shared component to `@edr/ui-common` + +1. Create `src/components//.tsx` and `src/components//index.ts`. +2. Export from `src/index.ts`. +3. Component is a functional component with a `ComponentNameProps` interface + (named-exported alongside the default). + +## Definition of done + +A change is done when **all** of these hold. State explicitly which you ran. + +1. **It type-checks.** `pnpm turbo type-check --filter=` passes. + If you edited `packages/types`, you ran `pnpm turbo build --filter=@edr/types` first. +2. **Raw SQL is verified.** Every new or edited SQL statement ran under `EXPLAIN` against the + dev database without error. +3. **Migrations are safe.** Unique timestamp, idempotent DDL, and — if the migration adds + something the new code reads — applied to the dev database, since watch mode will not run it. +4. **No new test failures.** `pnpm test` for `@edr/freight-api` is **currently red on `dev`**, + so a fully green suite is not the bar. Run the specs covering what you touched and confirm + you introduced no new failure. +5. **Lint and format are clean** for the files you touched. Git hooks do **not** run these + automatically (see below), so run them yourself. +6. **The behaviour was actually observed**, not merely compiled — you drove the flow, hit the + endpoint, or ran the query. If you could not, say so plainly. +7. **Report honestly.** If a check was skipped, tests failed, or a fix is unverified, say it in + the summary. Never describe unverified work as done. + +### Hooks do not run + +`commitlint.config.js` and a `lint-staged` config both exist, and husky's shims are installed +at `.husky/_/`. But there are **no user hook scripts** (`.husky/pre-commit`, +`.husky/commit-msg`), so husky's shim exits 0 and **neither lint-staged nor commitlint ever +fire.** Nothing validates your commit message or formats your staged files. Run the checks by +hand; do not assume the hook caught it. + +## Known traps + +| Trap | What happens | What to do | +| --- | --- | --- | +| Schema drift | A recorded migration's column is missing; queries and inserts 500 | Write a new repair migration; never edit the recorded one | +| Duplicate migration timestamps | Non-deterministic ordering; a migration can be skipped | Pick a fresh, higher timestamp | +| `@edr/types` not rebuilt | Consumers can't see your new field | `pnpm turbo build --filter=@edr/types` | +| Slow I/O in a transaction | Locks held for the gateway timeout | Queue it; fan out after commit; always set an HTTP timeout | +| Blob error bodies | Real 400 message replaced by "Request failed with status code 400" | `await extractDownloadErrorMessage(error)` | +| Company with no portal user | In-app notification silently vanishes | Check portal users before debugging | +| Watch-mode reload | New code, old schema → 500 | Apply the migration to the dev DB or restart fully | + +## Project skills + +Reusable workflows live in `.claude/skills/`. Use them instead of re-deriving the steps: + +| Skill | Use for | +| --- | --- | +| `edr-db` | Query / `EXPLAIN`-validate / inspect the remote dev DB (`node .claude/skills/edr-db/query.cjs …`). psql is not installed — this is the sanctioned path. Also carries the 400/500 diagnosis loop. | +| `verify` | The definition-of-done runner: targeted type-check, `@edr/types` rebuild, SQL validation, migration checklist, honest test bar. Run before calling anything finished. | +| `standup` | "What did I do today / this week" reports for tickets, grounded in `git log` — including the check that commit subjects match their contents. | + +## Working style + +- **Verify before asserting.** Read the code or query the database. Do not infer behaviour + from a filename. +- **Investigate, then propose.** For anything risky or wide-reaching, present the plan and the + trade-off before changing files. +- **Small, reviewable commits**, one logical change each, conventional message. +- **Branch from `dev`; PRs target `dev`.** +- When a finding turns out to be wrong, say so and retract it. A rejected finding is a result. diff --git a/docs/qa/edr-freight-qa-test-plan.md b/docs/qa/edr-freight-qa-test-plan.md new file mode 100644 index 000000000..f9102bc18 --- /dev/null +++ b/docs/qa/edr-freight-qa-test-plan.md @@ -0,0 +1,292 @@ +# EDR Freight — Operations QA Test Plan + +End-to-end test flows from booking through warehouse, rail, and delivery — import and export, +with and without first/last mile, self-haul and EDR haulage. Every status, guard, and endpoint +below is taken from the code, not assumed. + +| | | +|---|---| +| **Branch** | `Truckdetantion` | +| **Scope** | Warehouse · Fees · Allocation · First mile · Last mile | +| **Depth** | Tester steps + technical refs | + +--- + +## 00 · Test data setup + +Nothing below passes without this. Set it up once per environment and confirm each line before +opening a single flow. + +- [ ] **Warehouse tree.** At least one `ACTIVE` warehouse with a yard and a zone. Capacities are in **tonnes**, not kg. +- [ ] **Company has a linked portal user.** Critical — in-app notifications resolve recipients from the company's portal users. With none linked, `notify()` logs `0 recipients — skipped` and stores nothing. SMS/email still fire. +- [ ] **Customer saved signature.** Required for Approve delivery; without it the API returns *"Please save your signature before approving delivery"*. +- [ ] **Allocation rules** covering the freight type and trade direction under test (see §08), or accept the capacity-balanced fallback. +- [ ] **Fee rules** — at least one each of `STORAGE_FEE`, `DEMURRAGE_FEE`, `DOUBLE_HANDLING_FEE`, `TRUCK_DETENTION_FEE` (see §07). +- [ ] **Drivers and vehicles** registered; a train schedule with wagons for the route under test. +- [ ] **Booking reaches `PAID`.** Receive-to-warehouse skips any booking that is not PAID. +- [ ] **Container numbers are ISO 6346** — 4 letters + 7 digits, uppercase (`ABCU1234567`). Enforced at booking input and at every reference point. + +> **Direction is derived, not declared.** Receive-to-warehouse computes trade direction from the +> *origin and destination yard countries*, not the booking's stored `trade_direction`. A booking +> whose route says IMPORT will be skipped from an EXPORT receive with *"Booking route is IMPORT, +> not EXPORT"*. Set up yards accordingly. + +--- + +## 01 · Lifecycle reference + +The three state machines a tester needs to read a failure. Anything not listed as an allowed +transition is rejected by `assertTransition`. + +### Warehouse inventory transitions + +| From | Allowed next | Notes | +|---|---|---| +| `UNLOADED` | `STORED`, `READY_FOR_PICKUP` | Import landing state after train unload | +| `RECEIVED` | `STORED`, `READY_FOR_PICKUP` | Export landing state after truck receive | +| `STORED` | `RESERVED`, `READY_FOR_LOADING` | Reserve is retired from the UI; `STORED → READY_FOR_LOADING` is the live path | +| `READY_FOR_LOADING` | `LOADED` | Onto a wagon | +| `LOADED` | `DISPATCHED` | | +| `DISPATCHED` | `UNLOADED_AT_DJIBOUTI_PORT` | Export only, at Djibouti | +| `READY_FOR_PICKUP` | `DELIVERED`, `STORED`, `DISPATCHED` | Import; may be put back into storage | + +### Container item stages + +``` +PENDING → RECEIVED → GRN → ASSIGNED → LOADED → LEFT → DELIVERED +``` + +**ASSIGNED** means the customer picked which containers ride which truck — planning only. +**LOADED** requires the operator to actually load them, and only after the truck has arrived +(`loaded_at` is stamped then). Assignment alone must never show LOADED. + +### First mile & last mile + +| Leg | Statuses, in order | Gate it controls | +|---|---|---| +| First mile | `PAYMENT_PENDING → READY_TO_TRANSIT → IN_TRANSIT → RECEIVED_TO_PORT` | Export receive is blocked until `RECEIVED_TO_PORT` | +| Last mile | `PAYMENT_PENDING → READY_TO_TRANSIT → IN_TRANSIT → DELIVERED` | Truck-detention window: `arrivedAt` (reached destination) → `deliveredAt` (vehicle returned) | + +--- + +## 02 · Export — without first mile + +Customer brings the cargo to the facility themselves. The happy path from a paid booking to cargo +unloaded at Djibouti port with an interchange document. + +| Step | Tester action | Expected result | Technical ref | +|---|---|---|---| +| E1.1 | Create an export booking (route origin ET → destination DJ), pay it. | Booking reaches `PAID`. | Container numbers must be ISO 6346 | +| E1.2 | Export Operations → **Receive for Loading**. Select the booking, capture the truck entrance (plate, driver, weights), pick warehouse/yard/zone. | Inventory created; **GRN issued** as `GRN-EXPORT-YYYYMMDD-XXXXXXXX`. Customer gets a receive SMS. | `POST /warehouse-inventory/receive-bulk` | +| E1.3 | Store the item — auto-allocate, or pick warehouse/yard/zone in the Store modal. | Status `STORED`; note records "allocation rule", "capacity-balanced", or "operator-selected". | Capacity decremented in tonnes | +| E1.4 | Inspect: mark selected items as inspected, outcome **PASSED**. | Export items advance straight to `READY_FOR_LOADING`. | Reserve step is retired | +| E1.5 | Ready To Load tab → load onto the allocated wagon. | Status `LOADED`; a warehouse loading record exists. | Requires an allocated wagon | +| E1.6 | Download the **export marshalling / load list** PDF. | PDF lists the train's wagons, bookings, containers. | train-scheduling controller | +| E1.7 | Dispatch Queue → dispatch. | Status `DISPATCHED`. Customer receives **"Shipment dispatched"** naming origin → destination. | Per booking on the schedule | +| E1.8 | Move the schedule to arrived at the Djibouti-side port. | Train appears in the Djibouti unloading queue. Customer receives **"Shipment arrived"**. | status `ARRIVED` / `ARRIVED_AT_DJIBOUTI` | +| E1.9 | Grant the gate pass for the train, then **Unload at Djibouti**. | Items become `UNLOADED_AT_DJIBOUTI_PORT` and an **interchange document** is generated. | Unload checks `gatepass_granted_at` | +| **E1.G** | Try to unload at Djibouti *before* granting the gate pass. | 🚫 **Blocked.** Items skipped with a gate-pass reason; no interchange document. | See §06 | + +--- + +## 03 · Export — with first mile + +EDR collects the cargo from the customer's premises. Identical to §02 from the store step onward; +the difference is entirely in the gate before receive. + +| Step | Tester action | Expected result | Technical ref | +|---|---|---|---| +| E2.1 | Create the export booking with a **first-mile pickup address** (or a service type that includes first mile). | Booking is flagged `hasFirstMile`. | Derived from address *or* `service_types.includes_first_mile` | +| E2.2 | Create the first-mile request; assign a vehicle and driver. | Driver receives an SMS naming the vehicle, booking, pickup and destination. | First Mile page | +| E2.3 | Walk the leg: `READY_TO_TRANSIT → IN_TRANSIT → RECEIVED_TO_PORT`. | Each transition persists. | | +| E2.4 | Now run **Receive for Loading**. | Booking is received; truck entrance pre-filled from the first-mile vehicle and driver. | Then continue at E1.3 | +| **E2.G1** | Attempt receive with **no first-mile request created**. | 🚫 Skipped: *"First-mile request not created"*. | | +| **E2.G2** | Attempt receive while first-mile status is `IN_TRANSIT`. | 🚫 Skipped: *"First-mile truck has not arrived"*. | Only `RECEIVED_TO_PORT` passes | + +--- + +## 04 · Import — self-haul (customer collects) + +The longest flow, and the one with the most guards. The customer assigns their own trucks, signs a +booking-level handover, and collects. Test this one first — it exercises truck arrival, loading, +weighing, handover, exit paper, and fees. + +``` +train arrives → unload → inspect → ready for pickup → assign truck → truck arrival + → sign handover → load → truck leaving → exit paper → deliver +``` + +| Step | Tester action | Expected result | Technical ref | +|---|---|---|---| +| I1.1 | Import booking (route DJ → ET), paid, **no last-mile address** and a service type that excludes last mile. | Booking is self-haul. | Drives `usesCustomerTruck` | +| I1.2 | Import Operations → **Arrival Queue**. Open the arrived train, assign warehouse/yard/zone per booking, **Auto Unload Arrived Bookings**. | Inventory created at `UNLOADED`. Counter shows `n/total unloaded`. | Train must be `ARRIVED` | +| I1.3 | Inspect the item, outcome **PASSED**. | Item advances to `READY_FOR_PICKUP`. Customer receives **"Assign a truck for pickup"** (in-app + SMS + email). | Fires only when self-haul *and* no truck assigned | +| I1.4 | **Portal:** customer assigns truck(s), entering ISO container numbers per truck. | Containers move to stage `ASSIGNED` and show their planned truck. Booking becomes `TRUCK_ASSIGNED`. | 20ft → max 2/truck; 40ft → 1/truck; trucks ≤ containers | +| I1.5 | Backoffice row menu → **Truck Arrival**. Select the assigned truck, record gate-in time and **tare** weight (tonnes). | Truck stamped arrived. A **SELF_HAUL handover** is generated (booking-level) and the customer is notified to sign, on all three channels. | Truck Arrival disabled until a truck is assigned | +| I1.6 | **Portal:** customer opens the booking → **Approve delivery**. | The handover PDF opens for review; approving applies their saved signature and returns the signed PDF. | Signs all unsigned handovers on the booking | +| I1.7 | Open the container list → select the assigned containers → **Load onto truck** (pick the arrived truck). | Containers move to `LOADED`; `loaded_at` stamped. | Only arrived, not-departed trucks are listed | +| I1.8 | Row menu → **Truck Leaving**. Select the containers on the truck, record gate-out time and **gross** weight. | Net is computed from the selected containers' cargo weight and must equal `gross − tare`. Release document issued. | Weight match enforced client- *and* server-side | +| I1.9 | Generate the **exit paper** for the truck. | PDF lists the truck, driver, and its containers. Containers move to `LEFT`. | Requires signed handover + cleared fees | +| I1.10 | **Deliver** the inventory, recording the receiver name. | Status `DELIVERED`; handovers stamped delivered. | Requires handover signed *and* truck departed | + +> 🚫 **The single most likely bug you will hit.** Exit paper returns `400` when the handover is not +> fully signed, or when a warehouse fee invoice is `ISSUED` / `PARTIALLY_PAID`. In the container +> list, the Exit Paper button turns grey and clicking it **sends the customer a signature request** +> instead of erroring. That is correct behaviour — verify the message, don't file it as a bug. + +--- + +## 05 · Import — EDR last mile + +EDR delivers to the customer's door. No customer truck, no portal Approve delivery, and the +handover is *per delivering truck* — not per booking. This is where truck detention accrues. + +| Step | Tester action | Expected result | Technical ref | +|---|---|---|---| +| I2.1 | Import booking with a **last-mile delivery address** (or service type including last mile). | `hasLastMile` is true; no "assign a truck" notification is sent. | EDR haulage — customer assigns nothing | +| I2.2 | Unload from the arrived train, inspect PASSED. | Item becomes `READY_FOR_PICKUP` and the last-mile leg is accepted automatically. | | +| I2.3 | Last Mile page: assign vehicle + driver. | Leg reaches `READY_TO_TRANSIT`. Row shows Assigned. | Driver notified by SMS | +| I2.4 | **Truck Arrival** from the Last Mile row: gate-in, tare weight. | Weighing saved; the assigned last-mile truck is pre-selected. | Re-opening must show the saved details | +| I2.5 | **Truck Leaving**: gate-out, gross weight. | Release document issued; leg moves to `IN_TRANSIT`. `arrivedAt` starts the detention clock. | Detention window opens | +| I2.6 | Deliver at the customer's door, recording the receiver name. | Item `DELIVERED`. An **EDR_LAST_MILE handover is generated per delivering truck**, resolved from the container's allocated vehicle. | Not booking-level | +| I2.7 | Return the vehicle → mark the leg `DELIVERED`. | `deliveredAt` stamped; detention clock stops. | | +| I2.8 | Preview, then generate the **truck detention invoice**. | Charged per truck per day beyond the grace hours, at the matching tier. See §07. | `POST /last-mile/:id/generate-truck-detention-invoice` | +| **I2.G** | Open the booking in the portal. | 🚫 **Approve delivery must NOT appear.** The portal flag counts only `SELF_HAUL` handovers; EDR handovers are signed by the receiver at the door. | Regression check | + +--- + +## 06 · Marshalling, gate pass, interchange + +Documents are generated, not uploaded. Each has a precondition; test the precondition, not just the PDF. + +| Document | When | Precondition | Verify | +|---|---|---|---| +| **GRN** | On receive to warehouse | Booking `PAID`; route direction matches; export needs a truck entrance | Number format `GRN---<8>`; PDF opens; customer SMS sent | +| **Import load list / marshalling** | Import train, before unload | Schedule has bookings assigned | Summary endpoint and printable PDF agree; portrait/landscape both render | +| **Export marshalling / load list** | Export train, after loading | Items `LOADED` onto wagons | Wagon, booking, container rows are complete | +| **Gate pass** | Djibouti-side operations | Granted per schedule | `gatepass_granted_at` is set; **export Djibouti unload reads this same field** | +| **Interchange document** | Automatically, after a successful Djibouti unload | At least one item unloaded | Document number returned in the unload response; visible in Interchange Documents | +| **Handover** | Self-haul: on truck arrival. EDR: at delivery. | See §04 / §05 | Self-haul is booking-level, one per booking; EDR is one per delivering truck | +| **Exit paper / release doc** | Truck leaving | Handover fully signed *and* no unpaid warehouse fee | Weights on the paper match the gate weighing | + +> ⚠️ **Cross-module quirk worth confirming with the team.** The *export* Djibouti unload checks the +> gate-pass flag stored on the *import* Djibouti operations record +> (`import_djibouti_operations.gatepass_granted_at`). It works, but it is surprising. If an export +> unload silently skips every item, check that field first. + +--- + +## 07 · Fee rules + +Four rule types. Two are day-based with free days and tiers; double handling is a flat rate +multiplied by a basis; truck detention is hour-graced and vehicle-scoped. + +| Rule type | Charged on | Key fields | Test cases | +|---|---|---|---| +| `STORAGE_FEE` | Days in storage | free days, `tiers` | Within free days → zero. One day past → tier 1. Cross a tier boundary → correct tier rate. | +| `DEMURRAGE_FEE` | Days beyond free time | free days, `tiers` | Same boundary tests. Confirm it blocks exit paper and delivery while `ISSUED`. | +| `DOUBLE_HANDLING_FEE` | Flat rate × quantity | `basis`: `PER_CONTAINER` \| `PER_TON` \| `PER_ITEM` | Container booking → PER_CONTAINER uses container count. Bulk → PER_TON uses tonnage. Break-bulk machinery → PER_ITEM uses item count. **Import only.** Free days and tiers must not apply. | +| `TRUCK_DETENTION_FEE` | Per truck, per day | `free_hours` grace, `tiers`, vehicle type scope | Return inside the grace window → zero. Just past grace → day 1 at tier 1. Multi-day → tier escalation. A vehicle type outside the rule's scope → no charge. **Import only.** | + +### Fee behaviour to verify on every rule + +- [ ] **Preview before invoice.** The preview amount must equal the issued invoice total. +- [ ] **Notification on issue.** Issuing a warehouse fee invoice sends the customer an in-app `INVOICE_ISSUED` notification *and* an SMS, deep-linked to pay. +- [ ] **Clearance gate.** While a warehouse-source invoice is `ISSUED` or `PARTIALLY_PAID`, exit paper, terminal release, and Approve delivery are all blocked. +- [ ] **Payable-but-uninvoiced.** If fees are payable and no invoice exists yet, release is still blocked with *"Generate and fully pay…"*. Confirm the operator can generate it from that state. +- [ ] **Fully paid** → release proceeds; a receipt PDF is available. +- [ ] **Edit a rule** (rate, free days, tiers, grace hours) and confirm the next preview reflects it. + +--- + +## 08 · Allocation rules + +Where an item is stored is decided by the first matching rule, in priority order. Test the +precedence, not just one rule. + +| Match criteria (any may be null = wildcard) | Targets | +|---|---| +| `freight_type`, `trade_direction`, `cargo_type_code`, `container_status`, `requires_inspection`, ordered by `priority` | `target_facility_code`, `target_warehouse_code`, `target_yard_code` (required), `target_zone_code`, `storage_type` | + +### Precedence, highest first + +| # | Source of the location | How to trigger | Note recorded | +|---|---|---|---| +| 1 | **Operator selection** | Store modal → pick warehouse + yard + zone | "Stored at operator-selected location" | +| 2 | **Allocation rule** | Leave the Store modal blank; a matching rule exists | "Stored by allocation rule ``" | +| 3 | **Capacity-balanced fallback** | Leave blank; no rule matches | "Stored by capacity-balanced allocation" | + +- [ ] Two matching rules → the **lower priority number wins**. +- [ ] Yard dropdowns are filtered by **freight type** — container bookings offer container yards only. +- [ ] Only `ACTIVE` warehouses, yards and zones are selectable. +- [ ] Storing beyond a zone's capacity is rejected; capacities are compared in **tonnes**. +- [ ] **Move** an item to another warehouse/yard/zone → capacity released at source, taken at destination. +- [ ] Edit a rule, an existing warehouse, a yard, and a zone — all four must be editable. + +--- + +## 09 · Negative & guard cases + +Every row here is intended behaviour. The test passes when the action is **refused** with the +stated message. Anything that succeeds is the bug. + +| Area | Attempt | Expected refusal | +|---|---|---| +| Booking | Enter a container number that is not 4 letters + 7 digits (e.g. `MSKU10105185`, `3456789`). | *"Enter a valid ISO container number"*. Lowercase is auto-uppercased; input capped at 11 characters. | +| Booking | Enter the **same container number twice** in one shipment. | *"Duplicate container number in this shipment."* | +| Receive | Receive a booking that is not `PAID`. | Skipped: *"Booking not PAID"*. | +| Receive | Receive the same booking twice. | Skipped: *"Already received"*. | +| Receive | Export receive with no truck entrance captured. | Rejected before any inventory is created. | +| Truck assign | Put **two 40ft containers** on one truck. | *"A 40ft container fills the truck — assign only 1 container to this truck"*. | +| Truck assign | Put **three containers** on one truck. | *"A truck carries at most 2 containers"*. | +| Truck assign | Assign **more trucks than the booking has containers**. | *"Cannot assign more trucks than containers…"*. | +| Truck assign | Assign a container from another booking, or one already on another truck. | *"…is not one of this booking's containers"* / *"…already loaded onto another truck"*. | +| Truck assign | Edit a truck **after it has arrived**. | Refused — edits are allowed only until arrival. | +| Loading | **Load containers onto a truck that has not arrived.** | *"Record the truck arrival before loading…"*. The truck picker lists only arrived, not-departed trucks. | +| Loading | Load onto a truck that has already departed. | *"This truck has already left — its load is locked"*. | +| Stages | Customer assigns containers to a truck, then check the container list. | Stage is `ASSIGNED`, **never** `LOADED`. Exit Paper is not offered. | +| Truck leaving | Enter a gross weight where `gross − tare` ≠ the selected containers' cargo weight. | *"Weight mismatch…"*. Exit paper and gate clearance blocked, client and server. | +| Truck leaving | Save leaving with **no containers selected**. | *"Select the containers loaded on this truck"*. | +| Exit paper | Generate before the handover is signed. | *"Handover must be signed…"*. In the container list the button is grey and instead **sends the customer a signature request**. | +| Exit paper | Generate with an `ISSUED` demurrage/storage invoice. | *"…must be fully paid before terminal release"*. | +| Approve delivery | Approve without a saved signature. | *"Please save your signature…"* and the portal routes to the signature page. | +| Approve delivery | Approve before warehouse inspection has passed. | *"Delivery can be approved after warehouse inspection has passed"*. | +| Approve delivery | Approve when a truck is assigned but has not arrived. | *"Customer truck arrival must be recorded before delivery approval"*. | +| Deliver | Deliver before a release order was issued, or before the self-haul truck has left. | *"A release order must be issued…"* / *"Deliver is available only after the customer truck has left"*. | +| Djibouti unload | Unload an export train with no gate pass granted. | Items skipped; no interchange document generated. | +| Warehouse | Store into an `INACTIVE` warehouse/yard/zone, or beyond capacity. | Not selectable / *"No active warehouse yard/zone is available"* / capacity error. | + +--- + +## 10 · Notifications + +All customer notifications land in the same portal inbox. Verify the message, the deep link, and — +where noted — the SMS and email. + +| Notification | Fires when | Channels | Deep link | +|---|---|---|---| +| Shipment dispatched | Train schedule dispatched, per booking | In-app, SMS, email | Booking | +| Shipment arrived | Train schedule arrived, per booking | In-app, SMS, email | Booking | +| Assign a truck for pickup | Export: on warehouse receive. Import: on inspection pass → ready for pickup. Only if self-haul *and* no truck assigned. | In-app, SMS, email | Booking → assign truck | +| Handover — signature needed | Self-haul truck arrives (handover generated), and re-sent when an operator requests a signature from the Exit Paper button | In-app, SMS, email | Booking → approve delivery | +| Warehouse fee due | Storage / demurrage invoice issued | In-app, SMS | Booking → pay | +| Wagon allocated / payment window | Scheduling | In-app | Booking | + +> 🚫 **Do not chase a missing in-app notification before checking this.** Recipients are resolved +> from the company's linked portal users. If a company has none, `notify()` logs *"0 recipients — +> skipped"* and stores nothing — the notification simply never appears, with no error. SMS and +> email still go out, because they address the company's phone and email directly. On a fresh +> environment this is the usual explanation. + +--- + +## 11 · Known open issues + +Do not raise duplicates for these. Each is already identified. + +| Status | Issue | Impact on testing | +|---|---|---| +| 🔴 **Open** | **Export receive returns 500** on the deployed environment (`POST /warehouse-inventory/receive-bulk`). | Blocks flows E1 and E2 at step .2. Awaiting the response body / server log to diagnose. Likely schema drift, not the SQL. | +| 🟠 **Fix pending deploy** | **Export receive was extremely slow.** The owner SMS was awaited inside the DB transaction, and the SMS client had no HTTP timeout. | Fixed on branch: SMS now has a timeout, and notifications are sent after commit. Re-test receive latency once deployed. | +| 🟠 **Data** | **Drivers table has no unique constraints** on licence number, email, or phone, despite the entity declaring them unique. | Duplicate drivers can be created. Do not rely on uniqueness in test assertions. | +| 🔵 **Behaviour** | **Handover generated before truck arrival.** If an operator triggers a signature request from the Exit Paper button while a truck is assigned but not arrived, Approve delivery refuses with *"truck arrival must be recorded"*. | Only reachable off the normal path. Follow flow I1 in order and it will not occur. |