# Freight API integration suite Headless, API-level tests for the freight API running against the **real** `edr-payment-api`. Only the bank/wallet gateways are stubbed. The freight API is **not containerized** here — it boots inside each vitest worker from `apps/edr-freight-api/dist`, which `it:test` rebuilds every run. A code change needs no image rebuild, and a breakpoint in the API is hit by the test that provoked it. ``` pnpm it:up # build + start the stack (first run ~5 min) pnpm it:test # vitest run (auto-ups the stack if needed) pnpm it:test -- --reporter=basic src/payment-happy.it.ts pnpm it:test --no-reset -- src/payment-happy.it.ts # skip the DB re-clone pnpm it:logs payment-api-it-0 pnpm it:down # -v, wipes the throwaway DBs and the generated shard file IT_SHARDS=1 pnpm it:up # 1 shard when RAM is tight or output must be readable ``` ## Stack `docker-compose.it.yaml` is an **overlay** on `docker-compose.e2e.yaml`, plus a third **generated** file (`integration/.it-shards.yaml`, written by `scripts/gen-shards.mjs`, gitignored) holding the per-shard services. It is a separate compose project (`edr-freight-it`) on offset ports, so the Cypress e2e stack can run alongside. Each vitest worker is a whole shard of the topology. Nothing mutable is shared between shards, so spec files run in parallel: ``` shard i (vitest worker i, VITEST_POOL_ID) freight app in-process, host :3111+i │ ▲ │ └──── HTTP ──── payment-api-it-{i} :3131+i inbound CBE bill query, │ │ via host.docker.internal └── HTTP ──────────────► │ ──HTTP──> gateway-mock-it-{i} :4600+i ▲ │ └── RabbitMQ vhost payment_s{i} ──┘ (outbox → payment.events → consumer) database edr_it_s{i} ← CREATE DATABASE … TEMPLATE edr_freight_e2e freight tables AND the payment API's edr_payment schema live in it, so `db()` and `paymentDb()` are one connection ``` Shared by every shard: Postgres (one container, one database each), RabbitMQ (one container, one vhost each), MinIO, and the one-shot migration. `IT_SHARDS` defaults to **4**. RAM is the ceiling, not cores — a shard is an in-process Nest app plus two containers. Never `docker compose -f docker-compose.it.yaml` on its own — it needs the base file first, and the generated shard file last. Use `it.mjs`. ## Setup order The sequence is load-bearing, which is why `scripts/prepare-shards.mjs` exists rather than vitest's `globalSetup`: 1. `freight-migration-e2e` migrates the **template** database `edr_freight_e2e`. 2. `prepare-shards.mjs` boots the freight app once against the template and calls `app.init()` — that fires `onApplicationBootstrap`, the always-on seeders (org, units, positions, permissions). 3. …then applies the SQL fixtures, which declare exactly those as prerequisites (`seed-users.sql`: *"created by the API's always-on boot seeders"*). 4. `CREATE DATABASE edr_it_s{i} TEMPLATE edr_freight_e2e` per shard — a file copy on the tmpfs Postgres, not a re-run of migrations and five seeders. 5. The per-shard payment APIs start; each creates its own `edr_payment` schema inside its shard database. `it:test` re-runs step 4 on **every run** (stop payment APIs → drop → clone → start), so each run is hermetic. `--no-reset` skips it for a quick rerun. ## Gateway mock `gateway-mock/server.js` — one zero-dep `node:http` process serving every provider under a path prefix, plus a control plane the tests drive: | call | effect | | --- | --- | | `POST /__control/provider/:name` `{mode, times}` | `ok` / `fail` / `timeout` / `pending` / `paid` | | `POST /__control/webhook` `{merchantOrderId, status, eventId, signature}` | fires a **correctly signed** callback at the payment API | | `POST /__control/settle` `{merchantOrderId}` | pays at the bank with no callback (reconciliation path) | | `GET /__control/calls` | every inbound provider call | | `POST /__control/reset` | clear modes, orders and calls | Signatures are real: the mock shares `CBE_SECRET_KEY` with the API, so `verifyWebhookSignature` runs for real and `signature: "bad"` is a genuine negative test. The suite drives **CBE Birr** end to end (plain HMAC, no key material); other providers answer a generic stub until a scenario needs them. Editing `server.js` needs a container restart (`docker compose … restart gateway-mock-it-0`) — the code is a read-only mount, not baked into an image. **One mock per shard.** `modes`, `calls` and `orders` are process-global in `server.js`, and 20 of 25 specs call `POST /__control/reset` in `beforeAll`. On a shared mock every starting spec would wipe every running spec's live orders — and that fails as a wrong *assertion*, not an error: an unknown order still gets a correctly-signed webhook, just carrying the mock's default amount. Per-shard mocks are why the suite can run `payment-failure` (which forces providers into `fail`/`timeout`) beside a spec that settles normally. ## Files | file | covers | | --- | --- | | `src/payment-happy.it.ts` | initiate → webhook → outbox → broker → invoice PAID → booking advances | | `src/payment-failure.it.ts` | provider down, decline, forged signature, reconciliation sweep, `unverifiable`, late capture | | `src/concurrency.it.ts` | duplicate callbacks, two intents on one invoice, wagon budget, pay-window gate | | `src/authz.it.ts` | cross-tenant isolation, login audience, service-token gates | | `src/cbe-bill.it.ts` | inbound CBE Unified Bill: token → query (hops into freight) → payment | | `src/expired-invoice-late-settle.it.ts` | pay-window drain tail; a settlement landing after the hold expired still pays the invoice and revives the booking | | `src/bulk-import-full-train.it.ts` | six wheat bookings fill 54 wagons, then gate pass → T1 → dispatch → corridor → arrival → customs tail | | `src/bulk-import-waiting-expiry.it.ts` | exact-fill trio selected, waiting three expire with the day | | `src/bulk-import-split-promote.it.ts` | partial offer, split on settlement, expiry promotion, exact-remainder rebooking | | `src/bulk-import-window-reopen.it.ts` | nobody pays → cycle 2 opens on the same train | | `src/bulk-import-matrix.it.ts` | no-window day, sub-corridor, ride-along, whole-train giant | | `src/bulk-export-full-train.it.ts` | FCFS accept = reservation, deadlines clamped to window close, export tail | | `src/bulk-export-fcfs-space.it.ts` | reservations hold capacity, whole-or-nothing giant | | `src/bulk-export-pay-or-lose.it.ts` | expiry frees space; window close expires the unpaid, no reopen | | `src/bulk-export-matrix.it.ts` | mid-route boarding, leg occupancy, ride-along, sibling train windows | | `src/bulk-b1-priority-expiry-refill.it.ts` | rule-engine priority band, expiry refill | | `src/bulk-b2-per-item-floor.it.ts` | PER_ITEM wagon floor vs tonnage math | | `src/bulk-b3-per-item-giant.it.ts` | PER_ITEM giant + line quantities (pins two defects) | | `src/g1-s1-expiry-promotes-waitlist.it.ts` | 53-wagon built train: expiry frees exactly the waiting list's space | | `src/g1-s2-exact-fill.it.ts` | exact 53/53 fill, every one of 68 containers mapped to a slot | | `src/g1-s3-underfill-day-open.it.ts` | 28/53 is NOT FULL — the day still offers 25 wagons | | `src/g1-s4-split-closes-gap.it.ts` | container split closes the last 3-wagon gap, 14-box remainder | | `src/g1-s5-cascading-expiry.it.ts` | one settle promotes twice; expiries terminal, invoices closed | | `src/g1-s6-s8-offers-government-tiers.it.ts` | ignored offer, government preemption, USD/customs/plain tiers | | `src/g2-weight.it.ts` | weight before slots: base pull, overage tolerance, split sized on base only, light cargo | | `src/flows.ts` | freight business steps, ported from `e2e/freight/cypress/e2e/flows/import-utils.ts` | ## Findings pinned by these tests Where the platform's live behaviour differs from the scenario, the test asserts what it actually does and says so in a comment, so a fix fails loudly: - **Refill never supersedes an open partial offer** (`bulk-b1`, `g1-s5`). After the giant expires and frees 28 wagons, the offered booking is re-selected but keeps its stale 16-wagon offer; `applySplit` applies it, so the customer ships 16 of 20 with room to spare. `g1-s5` pins the container half: an expiry frees 8 more wagons and the 2-wagon offer beside them is never resized. - **PER_ITEM bookings never get a partial offer** (`bulk-b3`). `sizeOffer` sizes bulk offers by weight off `cargoTotalWeightVgm`, which for PER_ITEM holds the ITEM COUNT — a 240-auto booking needing 60 wagons looks like 4, gets no offer, allocates nothing, and expires with the day. - **The contract booking path drops bulk `hazardousQuantity` / `reeferQuantity`** (`bulk-b3`). Only `POST /api/bookings` maps and clamps them. - **Paid intercity ride-alongs are unpinned back to the pool** (both matrices) — staff must place them again. The Cypress twin never sees this because its staff mark-paid shortcut leaves the reservation pinned. - **Bulk priority is recomputed at doc-review** (`bulk-b1`), so writing `priority_score` directly is a no-op; ranking has to come from a WAGON priority config. - **Freight sends a dev-shortcut amount** (1 minor unit, 10 for CAC) for every non-`CBE_BILL` provider, with no short-payment guard. - **A BUILT train's batch is blind to the pull limit** (`g2-weight`, last describe). `remainingBudget` replaces the locomotive limits with `{wagons: physicalWagons, weightTons: Infinity, lengthMeters: Infinity}` the moment a schedule has a built train, so the batch reserves — and invoices — a load the locomotives cannot pull. The only check left is at wagon allocation, which then fails every tick with "Train set locomotives cannot pull the gross weight … limit 3500T incl. tolerance". The customer is PAID with zero wagons. Group 2 therefore runs its real weight scenarios on locomotive PAIRS, where the limits survive. - **The allocator weighs the whole consist, the batch weighs the booking.** Allocation charges the tare of every wagon in the train set (53 × 22.4 T on the G2 consist), while `needFor` charges only the tare of the wagons the booking occupies — so the same 45-wagon load reads 3 528 T at reservation and 3 707.2 T at allocation. Two capacity models, one train. - **Government preemption cannot reach a FULL train** (`g1-s6-s8`). `isFillable` rejects a schedule whose `booking_window_status` is FULL before any budget or victim is considered, and `refreshWindowStatus` re-derives that flag from live capacity — so a genuinely full train is skipped and no commercial booking is ever displaced. S7 therefore commits 52 of 53 slots. - **The CUSTOMS priority band is dead in the shipped fixture.** It applies only when the booking's SERVICE TYPE has `includes_customs`, and the corridor seed ships one service type that does not — so the two CUSTOMS bands never score. `seed-customs-service-type.sql` adds `RAIL_CUSTOMS` so the tier can be tested. - **A customs SERVICE TYPE without a customs CONTRACT cannot finalize clearance.** `finalizeClearance` looks up `clearance_output__`, which the seeder deliberately leaves commented out, so `getByCode` 404s. Only the phased path (`customs_clearing_enabled` on the contract) avoids it — which is why S8's customs tenant is Path B. - **A live intent makes a hold unexpirable here** (`expired-invoice-late-settle`). Reconcile-before-expire live-queries every non-FAILED intent; the mock answers `PROCESSING` for an unpaid order, which the payment API reads as "money in flight" → `unverifiable` → never expire on unknown. So while a CBE Birr intent is open, NOTHING retires the reservation — not the settle tick, not the staff `bookings/:id/expire` override (it runs the same guard). Producing the expired-invoice-with-a-payment case therefore needs the intent retired first (`status = 'FAILED'`, which reconcile skips), after which a webhook still late-captures it (`applyProviderResult`). ## Gotchas - **One unpaid hold per company.** `assertNoUnpaidHold` blocks a company with a `SELECTED_FOR_BATCH` booking from creating another. Every file starts with `releaseUnpaidHolds()`, which also hard-deletes retired `train_schedule_bookings` rows (`booking_id` is UNIQUE and the constraint ignores `deleted_at`, so a soft-unlinked booking can never be re-batched). - **Arrange is slow.** Contract → booking → clearance → ops accept → batch is 30–60s of real API work per booking, so files share one schedule day. - **Files run in parallel across shards, sequentially within one.** A shard owns its whole topology, so two files never contend; two files on the *same* shard still run one after the other, which is what the day-offset partitioning and the per-file `releaseUnpaidHolds()` / `resetCorridorDay()` hygiene still assume. Concurrency inside a single test is exercised with `Promise.all`. - **Day offsets are the de-facto partition key** (`departureAt(n)` per file). `payment-happy` and `expired-invoice-late-settle` both claim day 4 — harmless now that a shard has its own database, but do not read the offsets as unique. - **A retired fixture keeps its shipment day at its peril.** `rescueStrandedPaidForDay` sweeps every unlinked booking whose `payment_status` is PAID and whose `scheduled_date` falls on the day being filled, and re-places it on the fresh train. Within a run this still bites across files on one shard; *between* runs it no longer can — `it:test` re-clones every shard database from the template first. - **Only a FULL train rests at DONE.** An under-filled day CONCLUDES and REOPENS (`window_phase` back to OPEN, `booking_cycle_no` 2), so waiting for DONE there waits forever — use `pollCycleConcluded`. - **Wagon stock is finite and shared** *within a shard*. Paid bookings keep their wagons, so `releaseUnpaidHolds()` also frees every earlier `CTR-IT-%` allocation — without it the fifth or sixth file on one shard silently gets a short consist. - **`poll()` samples every 250ms, but keeps the caller's deadline.** Callers pass `attempts × intervalMs`; that product is the timeout, and only the sampling rate changed. Override with `IT_POLL_INTERVAL_MS`. - **The tick cadence is env-driven.** `BOOKING_WINDOW_TICK_CRON` is `*/1` here and `*/10` in production. Every window phase, allocation and expiry the suite waits on lands on that tick. A spec that proves sensitive to it should pin its own value rather than the whole suite reverting. - **One tenant per booking.** `seedTenantContracts` mints a company per booking because a company may hold only one unpaid reservation at a time; staff book and pay on their behalf, which is also the real Path B flow. - **The weight axis is GROSS, but the column is not.** `wagon_booking_allocations.allocated_weight_tons` holds CARGO only; `allocatedGrossTons` adds each wagon type's tare, because the pull limit is spent on both. Two 20ft at 28 T ride one wagon at 78.4 T gross — 35 of those spend a 3 500 T locomotive pair, and reading the raw column would report 1 960 T and hide it. - **Group 1 rides a BUILT train, not a loco pair.** `maxWagonsPerTrain` is not a cap: `syncScheduleMaxWagons` recomputes it from locomotive length (54 here) every fill pass. A built train's coupled consist wins outright, so `seed-g1-train.sql`'s 53 wagons ARE the capacity — the number every G1 scenario's arithmetic is written in. `createBuiltTrainSchedule` asserts it. - **Customs bookings walk the phased chain** (`clearBookingPhasedCustoms`): transit assignee → declaration draft → accept → declaration → duty → transit permit → pre-clearance → delivery order. `clearance/finalize` refuses them outright. - Freight sends a dev-shortcut amount (1 minor unit, 10 for CAC) for every non-`CBE_BILL` provider. The tests assert that as-is.