edr-platform/integration/README.md

# Freight API integration suite

Headless, API-level tests for the freight API running against the **real**
`edr-payment-api`. Only the bank/wallet gateways are stubbed.

The freight API is **not containerized** here — it boots inside each vitest
worker from `apps/edr-freight-api/dist`, which `it:test` rebuilds every run. A
code change needs no image rebuild, and a breakpoint in the API is hit by the
test that provoked it.

```
pnpm it:up      # build + start the stack (first run ~5 min)
pnpm it:test    # vitest run (auto-ups the stack if needed)
pnpm it:test -- --reporter=basic src/payment-happy.it.ts
pnpm it:test --no-reset -- src/payment-happy.it.ts   # skip the DB re-clone
pnpm it:logs payment-api-it-0
pnpm it:down    # -v, wipes the throwaway DBs and the generated shard file

IT_SHARDS=1 pnpm it:up     # 1 shard when RAM is tight or output must be readable
```

## Stack

`docker-compose.it.yaml` is an **overlay** on `docker-compose.e2e.yaml`, plus a
third **generated** file (`integration/.it-shards.yaml`, written by
`scripts/gen-shards.mjs`, gitignored) holding the per-shard services. It is a
separate compose project (`edr-freight-it`) on offset ports, so the Cypress e2e
stack can run alongside.

Each vitest worker is a whole shard of the topology. Nothing mutable is shared
between shards, so spec files run in parallel:

```
shard i  (vitest worker i, VITEST_POOL_ID)

  freight app  in-process, host :3111+i
       │  ▲
       │  └──── HTTP ──── payment-api-it-{i} :3131+i    inbound CBE bill query,
       │                        │                       via host.docker.internal
       └── HTTP ──────────────► │ ──HTTP──> gateway-mock-it-{i} :4600+i
       ▲                        │
       └── RabbitMQ vhost payment_s{i} ──┘  (outbox → payment.events → consumer)

  database  edr_it_s{i}   ← CREATE DATABASE … TEMPLATE edr_freight_e2e
                            freight tables AND the payment API's edr_payment
                            schema live in it, so `db()` and `paymentDb()` are
                            one connection
```

Shared by every shard: Postgres (one container, one database each), RabbitMQ
(one container, one vhost each), MinIO, and the one-shot migration.

`IT_SHARDS` defaults to **4**. RAM is the ceiling, not cores — a shard is an
in-process Nest app plus two containers.

Never `docker compose -f docker-compose.it.yaml` on its own — it needs the base
file first, and the generated shard file last. Use `it.mjs`.

## Setup order

The sequence is load-bearing, which is why `scripts/prepare-shards.mjs` exists
rather than vitest's `globalSetup`:

1. `freight-migration-e2e` migrates the **template** database `edr_freight_e2e`.
2. `prepare-shards.mjs` boots the freight app once against the template and
   calls `app.init()` — that fires `onApplicationBootstrap`, the always-on
   seeders (org, units, positions, permissions).
3. …then applies the SQL fixtures, which declare exactly those as prerequisites
   (`seed-users.sql`: *"created by the API's always-on boot seeders"*).
4. `CREATE DATABASE edr_it_s{i} TEMPLATE edr_freight_e2e` per shard — a file
   copy on the tmpfs Postgres, not a re-run of migrations and five seeders.
5. The per-shard payment APIs start; each creates its own `edr_payment` schema
   inside its shard database.

`it:test` re-runs step 4 on **every run** (stop payment APIs → drop → clone →
start), so each run is hermetic. `--no-reset` skips it for a quick rerun.

## Gateway mock

`gateway-mock/server.js` — one zero-dep `node:http` process serving every
provider under a path prefix, plus a control plane the tests drive:

| call | effect |
| --- | --- |
| `POST /__control/provider/:name` `{mode, times}` | `ok` / `fail` / `timeout` / `pending` / `paid` |
| `POST /__control/webhook` `{merchantOrderId, status, eventId, signature}` | fires a **correctly signed** callback at the payment API |
| `POST /__control/settle` `{merchantOrderId}` | pays at the bank with no callback (reconciliation path) |
| `GET /__control/calls` | every inbound provider call |
| `POST /__control/reset` | clear modes, orders and calls |

Signatures are real: the mock shares `CBE_SECRET_KEY` with the API, so
`verifyWebhookSignature` runs for real and `signature: "bad"` is a genuine
negative test. The suite drives **CBE Birr** end to end (plain HMAC, no key
material); other providers answer a generic stub until a scenario needs them.

Editing `server.js` needs a container restart (`docker compose … restart
gateway-mock-it-0`) — the code is a read-only mount, not baked into an image.

**One mock per shard.** `modes`, `calls` and `orders` are process-global in
`server.js`, and 20 of 25 specs call `POST /__control/reset` in `beforeAll`. On a
shared mock every starting spec would wipe every running spec's live orders —
and that fails as a wrong *assertion*, not an error: an unknown order still gets
a correctly-signed webhook, just carrying the mock's default amount. Per-shard
mocks are why the suite can run `payment-failure` (which forces providers into
`fail`/`timeout`) beside a spec that settles normally.

## Files

| file | covers |
| --- | --- |
| `src/payment-happy.it.ts` | initiate → webhook → outbox → broker → invoice PAID → booking advances |
| `src/payment-failure.it.ts` | provider down, decline, forged signature, reconciliation sweep, `unverifiable`, late capture |
| `src/concurrency.it.ts` | duplicate callbacks, two intents on one invoice, wagon budget, pay-window gate |
| `src/authz.it.ts` | cross-tenant isolation, login audience, service-token gates |
| `src/cbe-bill.it.ts` | inbound CBE Unified Bill: token → query (hops into freight) → payment |
| `src/expired-invoice-late-settle.it.ts` | pay-window drain tail; a settlement landing after the hold expired still pays the invoice and revives the booking |
| `src/bulk-import-full-train.it.ts` | six wheat bookings fill 54 wagons, then gate pass → T1 → dispatch → corridor → arrival → customs tail |
| `src/bulk-import-waiting-expiry.it.ts` | exact-fill trio selected, waiting three expire with the day |
| `src/bulk-import-split-promote.it.ts` | partial offer, split on settlement, expiry promotion, exact-remainder rebooking |
| `src/bulk-import-window-reopen.it.ts` | nobody pays → cycle 2 opens on the same train |
| `src/bulk-import-matrix.it.ts` | no-window day, sub-corridor, ride-along, whole-train giant |
| `src/bulk-export-full-train.it.ts` | FCFS accept = reservation, deadlines clamped to window close, export tail |
| `src/bulk-export-fcfs-space.it.ts` | reservations hold capacity, whole-or-nothing giant |
| `src/bulk-export-pay-or-lose.it.ts` | expiry frees space; window close expires the unpaid, no reopen |
| `src/bulk-export-matrix.it.ts` | mid-route boarding, leg occupancy, ride-along, sibling train windows |
| `src/bulk-b1-priority-expiry-refill.it.ts` | rule-engine priority band, expiry refill |
| `src/bulk-b2-per-item-floor.it.ts` | PER_ITEM wagon floor vs tonnage math |
| `src/bulk-b3-per-item-giant.it.ts` | PER_ITEM giant + line quantities (pins two defects) |
| `src/g1-s1-expiry-promotes-waitlist.it.ts` | 53-wagon built train: expiry frees exactly the waiting list's space |
| `src/g1-s2-exact-fill.it.ts` | exact 53/53 fill, every one of 68 containers mapped to a slot |
| `src/g1-s3-underfill-day-open.it.ts` | 28/53 is NOT FULL — the day still offers 25 wagons |
| `src/g1-s4-split-closes-gap.it.ts` | container split closes the last 3-wagon gap, 14-box remainder |
| `src/g1-s5-cascading-expiry.it.ts` | one settle promotes twice; expiries terminal, invoices closed |
| `src/g1-s6-s8-offers-government-tiers.it.ts` | ignored offer, government preemption, USD/customs/plain tiers |
| `src/g2-weight.it.ts` | weight before slots: base pull, overage tolerance, split sized on base only, light cargo |
| `src/flows.ts` | freight business steps, ported from `e2e/freight/cypress/e2e/flows/import-utils.ts` |

## Findings pinned by these tests

Where the platform's live behaviour differs from the scenario, the test asserts
what it actually does and says so in a comment, so a fix fails loudly:

- **Refill never supersedes an open partial offer** (`bulk-b1`, `g1-s5`). After
  the giant expires and frees 28 wagons, the offered booking is re-selected but
  keeps its stale 16-wagon offer; `applySplit` applies it, so the customer ships
  16 of 20 with room to spare. `g1-s5` pins the container half: an expiry frees
  8 more wagons and the 2-wagon offer beside them is never resized.
- **PER_ITEM bookings never get a partial offer** (`bulk-b3`). `sizeOffer` sizes
  bulk offers by weight off `cargoTotalWeightVgm`, which for PER_ITEM holds the
  ITEM COUNT — a 240-auto booking needing 60 wagons looks like 4, gets no offer,
  allocates nothing, and expires with the day.
- **The contract booking path drops bulk `hazardousQuantity` / `reeferQuantity`**
  (`bulk-b3`). Only `POST /api/bookings` maps and clamps them.
- **Paid intercity ride-alongs are unpinned back to the pool** (both matrices) —
  staff must place them again. The Cypress twin never sees this because its
  staff mark-paid shortcut leaves the reservation pinned.
- **Bulk priority is recomputed at doc-review** (`bulk-b1`), so writing
  `priority_score` directly is a no-op; ranking has to come from a WAGON
  priority config.
- **Freight sends a dev-shortcut amount** (1 minor unit, 10 for CAC) for every
  non-`CBE_BILL` provider, with no short-payment guard.
- **A BUILT train's batch is blind to the pull limit** (`g2-weight`, last
  describe). `remainingBudget` replaces the locomotive limits with
  `{wagons: physicalWagons, weightTons: Infinity, lengthMeters: Infinity}` the
  moment a schedule has a built train, so the batch reserves — and invoices —
  a load the locomotives cannot pull. The only check left is at wagon
  allocation, which then fails every tick with "Train set locomotives cannot
  pull the gross weight … limit 3500T incl. tolerance". The customer is PAID
  with zero wagons. Group 2 therefore runs its real weight scenarios on
  locomotive PAIRS, where the limits survive.
- **The allocator weighs the whole consist, the batch weighs the booking.**
  Allocation charges the tare of every wagon in the train set (53 × 22.4 T on
  the G2 consist), while `needFor` charges only the tare of the wagons the
  booking occupies — so the same 45-wagon load reads 3 528 T at reservation and
  3 707.2 T at allocation. Two capacity models, one train.
- **Government preemption cannot reach a FULL train** (`g1-s6-s8`). `isFillable`
  rejects a schedule whose `booking_window_status` is FULL before any budget or
  victim is considered, and `refreshWindowStatus` re-derives that flag from live
  capacity — so a genuinely full train is skipped and no commercial booking is
  ever displaced. S7 therefore commits 52 of 53 slots.
- **The CUSTOMS priority band is dead in the shipped fixture.** It applies only
  when the booking's SERVICE TYPE has `includes_customs`, and the corridor seed
  ships one service type that does not — so the two CUSTOMS bands never score.
  `seed-customs-service-type.sql` adds `RAIL_CUSTOMS` so the tier can be tested.
- **A customs SERVICE TYPE without a customs CONTRACT cannot finalize
  clearance.** `finalizeClearance` looks up `clearance_output_<op>_<freight>`,
  which the seeder deliberately leaves commented out, so `getByCode` 404s. Only
  the phased path (`customs_clearing_enabled` on the contract) avoids it — which
  is why S8's customs tenant is Path B.

- **A live intent makes a hold unexpirable here** (`expired-invoice-late-settle`).
  Reconcile-before-expire live-queries every non-FAILED intent; the mock answers
  `PROCESSING` for an unpaid order, which the payment API reads as "money in
  flight" → `unverifiable` → never expire on unknown. So while a CBE Birr intent
  is open, NOTHING retires the reservation — not the settle tick, not the staff
  `bookings/:id/expire` override (it runs the same guard). Producing the
  expired-invoice-with-a-payment case therefore needs the intent retired first
  (`status = 'FAILED'`, which reconcile skips), after which a webhook still
  late-captures it (`applyProviderResult`).

## Gotchas

- **One unpaid hold per company.** `assertNoUnpaidHold` blocks a company with a
  `SELECTED_FOR_BATCH` booking from creating another. Every file starts with
  `releaseUnpaidHolds()`, which also hard-deletes retired
  `train_schedule_bookings` rows (`booking_id` is UNIQUE and the constraint
  ignores `deleted_at`, so a soft-unlinked booking can never be re-batched).
- **Arrange is slow.** Contract → booking → clearance → ops accept → batch is
  30–60s of real API work per booking, so files share one schedule day.
- **Files run in parallel across shards, sequentially within one.** A shard owns
  its whole topology, so two files never contend; two files on the *same* shard
  still run one after the other, which is what the day-offset partitioning and
  the per-file `releaseUnpaidHolds()` / `resetCorridorDay()` hygiene still
  assume. Concurrency inside a single test is exercised with `Promise.all`.
- **Day offsets are the de-facto partition key** (`departureAt(n)` per file).
  `payment-happy` and `expired-invoice-late-settle` both claim day 4 — harmless
  now that a shard has its own database, but do not read the offsets as unique.
- **A retired fixture keeps its shipment day at its peril.**
  `rescueStrandedPaidForDay` sweeps every unlinked booking whose
  `payment_status` is PAID and whose `scheduled_date` falls on the day being
  filled, and re-places it on the fresh train. Within a run this still bites
  across files on one shard; *between* runs it no longer can — `it:test`
  re-clones every shard database from the template first.
- **Only a FULL train rests at DONE.** An under-filled day CONCLUDES and
  REOPENS (`window_phase` back to OPEN, `booking_cycle_no` 2), so waiting for
  DONE there waits forever — use `pollCycleConcluded`.
- **Wagon stock is finite and shared** *within a shard*. Paid bookings keep
  their wagons, so `releaseUnpaidHolds()` also frees every earlier `CTR-IT-%`
  allocation — without it the fifth or sixth file on one shard silently gets a
  short consist.
- **`poll()` samples every 250ms, but keeps the caller's deadline.** Callers pass
  `attempts × intervalMs`; that product is the timeout, and only the sampling
  rate changed. Override with `IT_POLL_INTERVAL_MS`.
- **The tick cadence is env-driven.** `BOOKING_WINDOW_TICK_CRON` is `*/1` here
  and `*/10` in production. Every window phase, allocation and expiry the suite
  waits on lands on that tick. A spec that proves sensitive to it should pin its
  own value rather than the whole suite reverting.
- **One tenant per booking.** `seedTenantContracts` mints a company per booking
  because a company may hold only one unpaid reservation at a time; staff book
  and pay on their behalf, which is also the real Path B flow.
- **The weight axis is GROSS, but the column is not.**
  `wagon_booking_allocations.allocated_weight_tons` holds CARGO only;
  `allocatedGrossTons` adds each wagon type's tare, because the pull limit is
  spent on both. Two 20ft at 28 T ride one wagon at 78.4 T gross — 35 of those
  spend a 3 500 T locomotive pair, and reading the raw column would report
  1 960 T and hide it.
- **Group 1 rides a BUILT train, not a loco pair.** `maxWagonsPerTrain` is not a
  cap: `syncScheduleMaxWagons` recomputes it from locomotive length (54 here)
  every fill pass. A built train's coupled consist wins outright, so
  `seed-g1-train.sql`'s 53 wagons ARE the capacity — the number every G1
  scenario's arithmetic is written in. `createBuiltTrainSchedule` asserts it.
- **Customs bookings walk the phased chain** (`clearBookingPhasedCustoms`):
  transit assignee → declaration draft → accept → declaration → duty →
  transit permit → pre-clearance → delivery order. `clearance/finalize` refuses
  them outright.
- Freight sends a dev-shortcut amount (1 minor unit, 10 for CAC) for every
  non-`CBE_BILL` provider. The tests assert that as-is.