mirror of
https://github.com/Tria-plc/edr-platform.git
synced 2026-08-27 00:52:50 +00:00
398 lines
20 KiB
Markdown
398 lines
20 KiB
Markdown
# Integration map — HR, Finance and the operational systems
|
||
|
||
> Section 5 of the ERP expansion. Written 2026-08-22, after HR (3.1–3.7) and
|
||
> Finance (4.1–4.6) were built and verified.
|
||
>
|
||
> This describes how six services share one database without owning each other's
|
||
> data. If something here contradicts the code, the code is the truth and this
|
||
> file is a bug — fix it in the same PR.
|
||
|
||
---
|
||
|
||
## 1. The services
|
||
|
||
| Service | Owns schema | Stack | Port |
|
||
|---|---|---|---|
|
||
| `edr-freight-api` | `freight` **and `iam`** | NestJS + TypeORM | 3001 |
|
||
| `edr-passenger-api` | `passenger` | NestJS + Prisma | 4000 |
|
||
| `edr-payment-api` | `edr_payment` | NestJS + TypeORM | 3003 |
|
||
| `edr-hr-api` | `hr` | NestJS + TypeORM | 3005 |
|
||
| `finance-api` | `finance` | NestJS + TypeORM | 3004 |
|
||
| `edr-gps-tracker` | — | separate service | — |
|
||
|
||
**One Postgres database, separated by schema.** Not one database per service.
|
||
Everything below follows from that: the isolation is a discipline, not an
|
||
infrastructure boundary, so it has to be enforced by rules people can read.
|
||
|
||
`edr-freight-api` is the authoritative owner of `iam` — it ships the
|
||
`iam:migration:*` scripts. No other service migrates that schema.
|
||
|
||
---
|
||
|
||
## 2. Who may write what
|
||
|
||
This is the load-bearing table. Everything else is detail.
|
||
|
||
| Schema | Written by | Read by |
|
||
|---|---|---|
|
||
| `iam` | freight-api (migrations); **HR and Finance only through IAM's own services** | everyone |
|
||
| `freight` | freight-api | freight-api, finance-api (read-only) |
|
||
| `passenger` | passenger-api | passenger-api, finance-api (read-only) |
|
||
| `edr_payment` | payment-api | payment-api, finance-api (read-only) |
|
||
| `hr` | hr-api | hr-api, finance-api (read-only) |
|
||
| `finance` | **finance-api only** | finance-api |
|
||
|
||
**Finance writes nothing but `finance`.** It is a reader of the whole platform
|
||
and an owner of one schema. No source system writes `finance.*` either — there
|
||
is no path by which freight could insert a journal entry, and that is deliberate:
|
||
the ledger's invariants live in one service, and anything that bypassed it would
|
||
bypass them too.
|
||
|
||
**HR writes `iam`, but never directly.** `IamOperationsService` resolves IAM's
|
||
own services out of the container (`ModuleRef.get(..., {strict: false})`) and
|
||
calls them, so IAM's validation, transactions and audit trail all still apply.
|
||
`IamDirectoryService` is read projections only.
|
||
|
||
---
|
||
|
||
## 3. The three integration mechanisms
|
||
|
||
Only three. Anything that looks like a fourth is a mistake.
|
||
|
||
### 3.1 Embedded module (in-process, transactional)
|
||
|
||
`IamModule.forRoot({...})` is imported by freight-api, hr-api and finance-api.
|
||
IAM is a library every service takes as a dependency, not a service they call.
|
||
|
||
This is why: **the hire flow creates an IAM user, an employee, an
|
||
employee-position and an HR profile in ONE transaction.** Over HTTP it could
|
||
not, and a partial failure would strand an IAM user with no HR profile.
|
||
|
||
Consequence, and it is intended: hr-api and finance-api both serve IAM's own
|
||
routes, including `POST /api/v1/auth/login`. They are second front ends to IAM's
|
||
capabilities, not forks of them.
|
||
|
||
Registration must use **globs over both package dists** with
|
||
`autoLoadEntities: false` — see `apps/finance-api/src/config/database.config.ts`.
|
||
A hand-maintained entity list goes stale on every package bump; the partial
|
||
`forFeature` set plus `autoLoadEntities` is what throws
|
||
`Entity metadata for User#sessions was not found` at boot.
|
||
|
||
### 3.2 Events (asynchronous, at-least-once)
|
||
|
||
One exchange: **`payment.events`**, a durable topic exchange, with
|
||
`payment.events.dlx` behind it.
|
||
|
||
```
|
||
┌──────────────────┐
|
||
│ edr-payment-api │
|
||
│ transactional │
|
||
│ outbox │
|
||
└────────┬─────────┘
|
||
│ publish payment.<service>.<outcome>
|
||
┌─────────▼──────────┐
|
||
│ payment.events │ topic, durable
|
||
└──┬───────┬──────┬──┘
|
||
payment.freight.* payment.passenger.* payment.#
|
||
┌────────▼──┐ ┌──▼──────────┐ ┌──▼──────────┐
|
||
│ freight │ │ passenger │ │ finance │
|
||
│ .payment- │ │ .payment- │ │ .payment- │
|
||
│ events │ │ events │ │ events │
|
||
└───────────┘ └─────────────┘ └─────────────┘
|
||
```
|
||
|
||
**Routing keys are three words**: `payment.passenger.succeeded`,
|
||
`payment.freight.failed` — built as `payment.${service}.${outcome}`. AMQP's `*`
|
||
matches exactly one word, so a `payment.*` binding matches **nothing**. Finance
|
||
binds `payment.#` because a ledger should see every payment.
|
||
|
||
Finance declares its **own** queue (`finance.payment-events`). It must never
|
||
share freight's or passenger's: a shared queue delivers each message to whichever
|
||
consumer takes it first, and freight would start losing payments to the ledger.
|
||
|
||
**Delivery is at-least-once.** The publisher is a transactional outbox with
|
||
retry and backoff; `eventId` is the outbox row id and is stable across every
|
||
redelivery. Every consumer must be idempotent — see §5.
|
||
|
||
The envelope (`PaymentEvent` in `@edr/types`):
|
||
|
||
| Field | Note |
|
||
|---|---|
|
||
| `version`, `eventId`, `eventType`, `occurredAt` | `eventId` is the dedupe key |
|
||
| `service`, `referenceType`, `referenceId` | which domain order |
|
||
| `merchantOrderId`, `intentId`, `provider` | provider-side identity |
|
||
| `amountMinor`, `currency` | **carries MAJOR units despite the name** — see §4 |
|
||
| `paidAt` / `failureCode` | per event type |
|
||
|
||
### 3.3 Read-only cross-schema projection
|
||
|
||
Finance reads `freight`, `passenger`, `hr` and `edr_payment` with
|
||
schema-qualified raw SQL. This is sanctioned — it is the same approach HR's 3.7
|
||
reports use, and it is justified by the single-database topology.
|
||
|
||
It carries one obligation, from the platform's hard rules:
|
||
|
||
> **Validate every raw SQL statement against a real database before shipping it.**
|
||
> A typo'd column name is a runtime 500 no type-checker will catch.
|
||
|
||
That rule earned its place twice during this build (§6).
|
||
|
||
Because upstream schemas deploy independently, Finance **probes for source
|
||
availability** (`to_regclass`) and reports it at `GET /api/v1/revenue/sources`.
|
||
The UI then says "this source is not present" instead of showing a zero. On the
|
||
current dev replica, freight's billing tables and the whole `edr_payment` schema
|
||
are absent — "no freight revenue" and "freight billing is not deployed" are
|
||
completely different facts and must not look alike.
|
||
|
||
### 3.4 What is NOT an integration mechanism
|
||
|
||
`${source}.invoice.paid`, `booking.cancelled` and similar are **in-process
|
||
`EventEmitter2`** events inside a single app. They do not cross a service
|
||
boundary and cannot be subscribed to from another service, despite reading like
|
||
broker events. Note also that freight emits `lastmile.*`/`firstmile.*` while
|
||
several listeners subscribe to `last_mile.*`/`first_mile.*` — a pre-existing
|
||
mismatch, not something Finance relies on.
|
||
|
||
---
|
||
|
||
## 4. The money contract
|
||
|
||
`apps/finance-api/src/common/money.ts` is the authority. Summary:
|
||
|
||
| Store | Representation | Unit |
|
||
|---|---|---|
|
||
| `freight.*` | `numeric(14,2)` | **major** |
|
||
| `passenger.*` Prisma `Int` columns | integer | **minor (cents)** |
|
||
| `passenger."PaymentIntent".amountMinor` | double | **major**, despite the name |
|
||
| `edr_payment.payment_intent.amount_minor` | double | **major**, despite the name |
|
||
| payment events `amountMinor` | JSON number | **major**, despite the name |
|
||
| `hr.*` payroll, `finance.*` | `numeric(14,2)` | **major** |
|
||
|
||
**Never infer the unit from the name.** Every `*Minor` field on the payment path
|
||
carries major units; the names are a wire contract across service APIs and are
|
||
not worth a cross-service rename.
|
||
|
||
**Currencies are never summed together.** Confirmed passenger bookings are
|
||
charged in ETB **and DJF and USD** — 151.4M / 6.18M / 133k on the dev replica.
|
||
Adding them reports 157.7M and overstates ETB revenue by **6.32M**. Finance
|
||
groups every revenue projection by currency, posts only ledger-currency amounts,
|
||
and reports the rest as "needs a rate" rather than converting at a guess.
|
||
|
||
---
|
||
|
||
## 5. Idempotency — how each path avoids double-counting
|
||
|
||
Every automated write into the ledger is keyed, because at-least-once delivery
|
||
and human re-clicks are both certain.
|
||
|
||
| Path | Key | Where enforced |
|
||
|---|---|---|
|
||
| Payment event → cash receipt | `eventId` | unique index on `finance.inbound_events.event_id`; claimed with `INSERT … ON CONFLICT DO NOTHING` |
|
||
| Any automated journal | `(organization, source_module, source_id)` | partial unique index on `finance.journal_entries` |
|
||
| Revenue recognition | `('<source>-revenue', 'YYYY-MM')` | same index |
|
||
| Payroll → GL | `('hr-payroll', <run id>)` | same index |
|
||
| Supplier bill approval | `('supplier-bill', <bill id>)` | same index |
|
||
| Depreciation | `('depreciation', <period id>)` | same index + one run per period |
|
||
| Statutory remittance | `(type, period)` | unique index on `statutory_remittances` |
|
||
|
||
The inbound-event claim is a **single statement**, not a read-then-write: two
|
||
concurrent deliveries of the same event would both pass a prior `SELECT`.
|
||
|
||
An event that cannot be posted is recorded **FAILED with its full payload** and
|
||
acknowledged, not dead-lettered — it is already durably stored and replayable.
|
||
Only a failure to *record at all* nacks, because then the broker holds the only
|
||
copy.
|
||
|
||
---
|
||
|
||
## 6. Traps this integration has already paid for
|
||
|
||
Each of these was found by an assertion, not by review. They are in the
|
||
platform's `CLAUDE.md` "Known traps" table.
|
||
|
||
| Trap | Consequence |
|
||
|---|---|
|
||
| `payment.*` binding | Receives **nothing** — keys are three words |
|
||
| Summing across currencies | 6.32M overstatement on real data |
|
||
| `DATE` via a JS `Date` | `toISOString()` returns the previous day east of UTC; a month-end lands in the wrong period |
|
||
| `invoice_lines.line_total` | The column is `amount`; a plausible name that does not exist |
|
||
| `iam.employees.first_name` | There is one JSONB `name` column, no first/middle/last |
|
||
| `CHECK` wider than the column | `varchar(16)` accepted every status until the 17-character one was first used |
|
||
| Nested transactions | An inner `dataSource.transaction` commits independently and orphans rows |
|
||
| `is_contra` as a sign flip | Double-flips accumulated depreciation; balance sheet out by exactly 2× |
|
||
|
||
---
|
||
|
||
## 7. End-to-end flows
|
||
|
||
### 7.1 Passenger ticket sale
|
||
|
||
```
|
||
passenger-api payment-api finance-api
|
||
│ │ │
|
||
booking ──initiate──────► intent │
|
||
│ │ │
|
||
│ provider settles │
|
||
│ │ │
|
||
│◄──payment.passenger.succeeded──────────────►│
|
||
│ │ │
|
||
confirm booking │ Dr 1114 Gateway clearing
|
||
issue ticket │ Cr 1122 Trade receivable
|
||
│
|
||
monthly ──────────► recognize revenue
|
||
Dr 1122 / Cr 4210 Ticket revenue
|
||
(ONE summarised entry per period)
|
||
```
|
||
|
||
Revenue is recognised from `Booking.totalMinor` (genuine cents), **not** from
|
||
`PaymentIntent.amountMinor` — 18 rows there are 100× overstated by a
|
||
force-confirm path that writes cents into a major-unit column, and that path is
|
||
still live upstream.
|
||
|
||
The two halves meet at **1122 Trade Receivables**: recognition creates the
|
||
receivable, the payment clears it. A payment is not revenue, and posting both
|
||
would count the sale twice.
|
||
|
||
### 7.2 Freight shipment
|
||
|
||
Same shape, with `payment.freight.*` clearing **1121**. Revenue projects from
|
||
`freight.invoice_lines.charge_type` through `finance.revenue_mappings` (42 seeded
|
||
from freight's own 36-value canonical list). An unmapped charge type posts to
|
||
**4900 Unclassified Revenue** — a real, visible account, so an unexpected balance
|
||
there is the prompt to add a mapping rather than a silent misclassification.
|
||
|
||
Cash receipts read the invoice `payments` **jsonb**, not the `freight.payments`
|
||
table: the table is a one-row-per-booking gateway-intent projection updated in
|
||
place, while the jsonb is the only per-settlement ledger that exists.
|
||
|
||
### 7.3 Payroll cycle
|
||
|
||
```
|
||
hr-api finance-api
|
||
│ │
|
||
calculate run ──► APPROVED │
|
||
│ │
|
||
│◄────── read-only projection ───────────────┤
|
||
│ hr.payroll_runs / hr.payslips │
|
||
│ │
|
||
│ Dr 5110 Basic salary
|
||
│ Dr 5120 Allowances
|
||
│ Dr 5140 Employer pension
|
||
│ Cr 2121 PAYE payable
|
||
│ Cr 2122 Pension payable (ee + er)
|
||
│ Cr 2130 Salaries payable
|
||
│ Cr 2160 Other deductions
|
||
│ │
|
||
│ disbursement register (the list HR lacks)
|
||
│ │
|
||
│ remittance: Dr 212x / Cr cash
|
||
```
|
||
|
||
Only **APPROVED** runs are posted — a run that can still be recalculated would
|
||
leave the ledger describing a payroll that no longer exists. The entry is dated
|
||
the period **end**, because the cost belongs to the month worked even when the
|
||
money leaves later. `gross − deductions = net` is checked explicitly before
|
||
posting, aggregated from the payslips rather than the run header: the employer's
|
||
pension is both a debit and part of the credit, and getting that wrong still
|
||
*balances*.
|
||
|
||
Finance stores no copy of the payroll. Only the journal link.
|
||
|
||
---
|
||
|
||
## 8. Known gaps — things this map does NOT claim work
|
||
|
||
Stated plainly, because a map that hides its blank areas is worse than no map.
|
||
|
||
1. **Refunds are never executed anywhere.** `freight.payment_refunds` has no
|
||
writer at all; passenger's `PaymentRefund` is never created; and
|
||
`BookingCancellation` rows are terminal at creation — `refundStatus` is never
|
||
updated and `processedAt` never set. The 80% cancellation refund is
|
||
*computed and recorded* but never paid. Finance therefore posts it as a
|
||
**liability provision (2150 Refunds Payable)**, never as cash. Do **not** add
|
||
a `payment.refunded` event until payment-api actually processes refunds.
|
||
2. **Cash outside the payment rails.** Agent counter cash (confirmed with no
|
||
PaymentIntent), excess-baggage `CASH_COLLECTED`, freight offline settlements,
|
||
and wallet top-ups. The wallet one is a control gap: `topUp` credits
|
||
unconditionally with no payment record and no transaction wrapper.
|
||
3. **The upstream money bugs were not fixed** — that was a deliberate decision.
|
||
Finance defends at its own projection boundary instead. The cost is that the
|
||
defence is permanent: the force-confirm path still writes bad rows.
|
||
4. ~~**Not verified against a live broker.**~~ **Closed 2026-08-22.** The
|
||
credentials had never existed: the local broker had only `guest` and the `/`
|
||
vhost, so both configured URLs failed auth. With an `edr` user and a
|
||
`payment` vhost created, the whole path was observed end to end — the
|
||
handler registers as `payment.events::payment.#::finance.payment-events`,
|
||
both exchanges and both queues are declared durable, and the DLQ really is
|
||
bound on the DLX. Six events were published through the shared
|
||
`paymentRoutingKey()` helper the real publisher uses: a passenger receipt
|
||
posted to 1122, a freight receipt to 1121 (so `payment.#` does catch a
|
||
second service), a REDELIVERY of the first eventId was deduplicated and did
|
||
not double-post, `payment.failed` was SKIPPED, a USD payment was recorded
|
||
FAILED rather than converted at a guessed rate, and an event with no
|
||
`eventId` was dead-lettered and actually arrived in the DLQ.
|
||
|
||
**Mind the vhost.** In an AMQP URI the path IS the vhost and the leading
|
||
slash is only a separator: `.../payment` means vhost `payment`, not
|
||
`/payment`. payment-api defaults to `amqp://localhost:5672/payment`, so
|
||
Finance must be on vhost `payment` too. Neither side errors when they
|
||
disagree — both connect with `wait: false`.
|
||
|
||
**What is still off:** `PAYMENT_RABBITMQ_URL` is absent from finance-api's
|
||
`.env` and was absent from `.env.example`, so `RevenueModule` skips the
|
||
RabbitMQ import entirely and live ingest is silently disabled. Both files now
|
||
document it. Setting it (plus `FINANCE_DEFAULT_ORG_ID`, without which the
|
||
consumer posts against `""` and every account lookup fails) is what turns
|
||
ingest on — that is go-live checklist step 3.
|
||
5. **Freight billing and `edr_payment` are absent from the dev replica.** Their
|
||
projections are shape-validated against entity-derived tables, not real data.
|
||
6. ~~**No browser render check** on any Finance screen.~~ **Closed 2026-08-22.**
|
||
All 13 screens were driven in Chromium: 20 tabs, 11 modals, and a full
|
||
draft → post → reverse journal cycle checked against the trial balance,
|
||
P&L, balance sheet, cash and general-ledger reports. No page errors and no
|
||
failed requests. Four defects were found and fixed, all invisible to
|
||
type-check and build — the largest being that the theme toggle drove only
|
||
Tailwind's `dark` class and never Mantine's colour scheme, so every page
|
||
rendered black-on-black in dark mode. Still NOT checked: the write paths for
|
||
suppliers, bills, assets, depreciation and budgets were opened but not
|
||
submitted, and no screen has been viewed below 1440px.
|
||
7. **P7 cutover: the tooling is BUILT (2026-08-22), the migration is not RUN.**
|
||
`finance.org_settings` holds a per-organization cutover date;
|
||
`GET /api/v1/cutover/readiness` answers five questions with live numbers
|
||
(suspense zero, no operational entry before the boundary, register agrees
|
||
with the ledger, migrated assets carry a period count, date set);
|
||
`POST /api/v1/cutover/opening-balances` turns pasted rows into a DRAFT
|
||
OPENING entry with the 3900 plug **calculated, never supplied**; and the
|
||
payment consumer now SKIPs any event settled before the cutover, recording
|
||
the reason — without that, a replayed backlog would post money the opening
|
||
balances already carry. Screen at `/cutover`.
|
||
|
||
What remains is the business act, not a build: choose the date, enter the
|
||
real balances, and post them. Two traps were found and fixed while building
|
||
this — a migrated fixed asset would silently never depreciate again, and the
|
||
suspense check was counting DRAFT lines because a status filter sat in a
|
||
LEFT JOIN's ON clause.
|
||
|
||
---
|
||
|
||
## 9. Before going live
|
||
|
||
1. Run freight's migrations against the target database so billing tables exist.
|
||
2. Boot payment-api so `edr_payment` is created, and confirm the broker vhost.
|
||
3. Set `PAYMENT_RABBITMQ_URL` and `FINANCE_DEFAULT_ORG_ID` on finance-api; verify
|
||
a real event reaches `finance.payment-events`.
|
||
4. Seed the chart of accounts and revenue mappings per organization.
|
||
5. Create the fiscal year (8 Jul – 7 Jul) and its periods.
|
||
6. Set the cutover date on `/cutover` **before** entering anything — the
|
||
consumer needs it to know which events are already history.
|
||
7. Enter opening balances on `/cutover`, in as many batches as suits (cash,
|
||
receivables, payables, equity). Each becomes a DRAFT you post yourself; the
|
||
difference goes to **3900 Opening Balance Suspense**, which must end at
|
||
**zero**. That is the check that the migration was entered correctly.
|
||
Migrate fixed assets through the register with their accumulated
|
||
depreciation **and** their opening period count, and with no funding account
|
||
— the ledger side comes from the opening entry.
|
||
8. Work the readiness panel until every check passes.
|
||
9. Reconcile the first period end: trial balance balances, balance sheet
|
||
balances, and the asset register agrees with the ledger.
|