Files
edr-platform/docs/integration-map.md
2026-08-25 00:11:39 +03:00

398 lines
20 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Integration map — HR, Finance and the operational systems
> Section 5 of the ERP expansion. Written 2026-08-22, after HR (3.13.7) and
> Finance (4.14.6) were built and verified.
>
> This describes how six services share one database without owning each other's
> data. If something here contradicts the code, the code is the truth and this
> file is a bug — fix it in the same PR.
---
## 1. The services
| Service | Owns schema | Stack | Port |
|---|---|---|---|
| `edr-freight-api` | `freight` **and `iam`** | NestJS + TypeORM | 3001 |
| `edr-passenger-api` | `passenger` | NestJS + Prisma | 4000 |
| `edr-payment-api` | `edr_payment` | NestJS + TypeORM | 3003 |
| `edr-hr-api` | `hr` | NestJS + TypeORM | 3005 |
| `finance-api` | `finance` | NestJS + TypeORM | 3004 |
| `edr-gps-tracker` | — | separate service | — |
**One Postgres database, separated by schema.** Not one database per service.
Everything below follows from that: the isolation is a discipline, not an
infrastructure boundary, so it has to be enforced by rules people can read.
`edr-freight-api` is the authoritative owner of `iam` — it ships the
`iam:migration:*` scripts. No other service migrates that schema.
---
## 2. Who may write what
This is the load-bearing table. Everything else is detail.
| Schema | Written by | Read by |
|---|---|---|
| `iam` | freight-api (migrations); **HR and Finance only through IAM's own services** | everyone |
| `freight` | freight-api | freight-api, finance-api (read-only) |
| `passenger` | passenger-api | passenger-api, finance-api (read-only) |
| `edr_payment` | payment-api | payment-api, finance-api (read-only) |
| `hr` | hr-api | hr-api, finance-api (read-only) |
| `finance` | **finance-api only** | finance-api |
**Finance writes nothing but `finance`.** It is a reader of the whole platform
and an owner of one schema. No source system writes `finance.*` either — there
is no path by which freight could insert a journal entry, and that is deliberate:
the ledger's invariants live in one service, and anything that bypassed it would
bypass them too.
**HR writes `iam`, but never directly.** `IamOperationsService` resolves IAM's
own services out of the container (`ModuleRef.get(..., {strict: false})`) and
calls them, so IAM's validation, transactions and audit trail all still apply.
`IamDirectoryService` is read projections only.
---
## 3. The three integration mechanisms
Only three. Anything that looks like a fourth is a mistake.
### 3.1 Embedded module (in-process, transactional)
`IamModule.forRoot({...})` is imported by freight-api, hr-api and finance-api.
IAM is a library every service takes as a dependency, not a service they call.
This is why: **the hire flow creates an IAM user, an employee, an
employee-position and an HR profile in ONE transaction.** Over HTTP it could
not, and a partial failure would strand an IAM user with no HR profile.
Consequence, and it is intended: hr-api and finance-api both serve IAM's own
routes, including `POST /api/v1/auth/login`. They are second front ends to IAM's
capabilities, not forks of them.
Registration must use **globs over both package dists** with
`autoLoadEntities: false` — see `apps/finance-api/src/config/database.config.ts`.
A hand-maintained entity list goes stale on every package bump; the partial
`forFeature` set plus `autoLoadEntities` is what throws
`Entity metadata for User#sessions was not found` at boot.
### 3.2 Events (asynchronous, at-least-once)
One exchange: **`payment.events`**, a durable topic exchange, with
`payment.events.dlx` behind it.
```
┌──────────────────┐
│ edr-payment-api │
│ transactional │
│ outbox │
└────────┬─────────┘
│ publish payment.<service>.<outcome>
┌─────────▼──────────┐
│ payment.events │ topic, durable
└──┬───────┬──────┬──┘
payment.freight.* payment.passenger.* payment.#
┌────────▼──┐ ┌──▼──────────┐ ┌──▼──────────┐
│ freight │ │ passenger │ │ finance │
│ .payment- │ │ .payment- │ │ .payment- │
│ events │ │ events │ │ events │
└───────────┘ └─────────────┘ └─────────────┘
```
**Routing keys are three words**: `payment.passenger.succeeded`,
`payment.freight.failed` — built as `payment.${service}.${outcome}`. AMQP's `*`
matches exactly one word, so a `payment.*` binding matches **nothing**. Finance
binds `payment.#` because a ledger should see every payment.
Finance declares its **own** queue (`finance.payment-events`). It must never
share freight's or passenger's: a shared queue delivers each message to whichever
consumer takes it first, and freight would start losing payments to the ledger.
**Delivery is at-least-once.** The publisher is a transactional outbox with
retry and backoff; `eventId` is the outbox row id and is stable across every
redelivery. Every consumer must be idempotent — see §5.
The envelope (`PaymentEvent` in `@edr/types`):
| Field | Note |
|---|---|
| `version`, `eventId`, `eventType`, `occurredAt` | `eventId` is the dedupe key |
| `service`, `referenceType`, `referenceId` | which domain order |
| `merchantOrderId`, `intentId`, `provider` | provider-side identity |
| `amountMinor`, `currency` | **carries MAJOR units despite the name** — see §4 |
| `paidAt` / `failureCode` | per event type |
### 3.3 Read-only cross-schema projection
Finance reads `freight`, `passenger`, `hr` and `edr_payment` with
schema-qualified raw SQL. This is sanctioned — it is the same approach HR's 3.7
reports use, and it is justified by the single-database topology.
It carries one obligation, from the platform's hard rules:
> **Validate every raw SQL statement against a real database before shipping it.**
> A typo'd column name is a runtime 500 no type-checker will catch.
That rule earned its place twice during this build (§6).
Because upstream schemas deploy independently, Finance **probes for source
availability** (`to_regclass`) and reports it at `GET /api/v1/revenue/sources`.
The UI then says "this source is not present" instead of showing a zero. On the
current dev replica, freight's billing tables and the whole `edr_payment` schema
are absent — "no freight revenue" and "freight billing is not deployed" are
completely different facts and must not look alike.
### 3.4 What is NOT an integration mechanism
`${source}.invoice.paid`, `booking.cancelled` and similar are **in-process
`EventEmitter2`** events inside a single app. They do not cross a service
boundary and cannot be subscribed to from another service, despite reading like
broker events. Note also that freight emits `lastmile.*`/`firstmile.*` while
several listeners subscribe to `last_mile.*`/`first_mile.*` — a pre-existing
mismatch, not something Finance relies on.
---
## 4. The money contract
`apps/finance-api/src/common/money.ts` is the authority. Summary:
| Store | Representation | Unit |
|---|---|---|
| `freight.*` | `numeric(14,2)` | **major** |
| `passenger.*` Prisma `Int` columns | integer | **minor (cents)** |
| `passenger."PaymentIntent".amountMinor` | double | **major**, despite the name |
| `edr_payment.payment_intent.amount_minor` | double | **major**, despite the name |
| payment events `amountMinor` | JSON number | **major**, despite the name |
| `hr.*` payroll, `finance.*` | `numeric(14,2)` | **major** |
**Never infer the unit from the name.** Every `*Minor` field on the payment path
carries major units; the names are a wire contract across service APIs and are
not worth a cross-service rename.
**Currencies are never summed together.** Confirmed passenger bookings are
charged in ETB **and DJF and USD** — 151.4M / 6.18M / 133k on the dev replica.
Adding them reports 157.7M and overstates ETB revenue by **6.32M**. Finance
groups every revenue projection by currency, posts only ledger-currency amounts,
and reports the rest as "needs a rate" rather than converting at a guess.
---
## 5. Idempotency — how each path avoids double-counting
Every automated write into the ledger is keyed, because at-least-once delivery
and human re-clicks are both certain.
| Path | Key | Where enforced |
|---|---|---|
| Payment event → cash receipt | `eventId` | unique index on `finance.inbound_events.event_id`; claimed with `INSERT … ON CONFLICT DO NOTHING` |
| Any automated journal | `(organization, source_module, source_id)` | partial unique index on `finance.journal_entries` |
| Revenue recognition | `('<source>-revenue', 'YYYY-MM')` | same index |
| Payroll → GL | `('hr-payroll', <run id>)` | same index |
| Supplier bill approval | `('supplier-bill', <bill id>)` | same index |
| Depreciation | `('depreciation', <period id>)` | same index + one run per period |
| Statutory remittance | `(type, period)` | unique index on `statutory_remittances` |
The inbound-event claim is a **single statement**, not a read-then-write: two
concurrent deliveries of the same event would both pass a prior `SELECT`.
An event that cannot be posted is recorded **FAILED with its full payload** and
acknowledged, not dead-lettered — it is already durably stored and replayable.
Only a failure to *record at all* nacks, because then the broker holds the only
copy.
---
## 6. Traps this integration has already paid for
Each of these was found by an assertion, not by review. They are in the
platform's `CLAUDE.md` "Known traps" table.
| Trap | Consequence |
|---|---|
| `payment.*` binding | Receives **nothing** — keys are three words |
| Summing across currencies | 6.32M overstatement on real data |
| `DATE` via a JS `Date` | `toISOString()` returns the previous day east of UTC; a month-end lands in the wrong period |
| `invoice_lines.line_total` | The column is `amount`; a plausible name that does not exist |
| `iam.employees.first_name` | There is one JSONB `name` column, no first/middle/last |
| `CHECK` wider than the column | `varchar(16)` accepted every status until the 17-character one was first used |
| Nested transactions | An inner `dataSource.transaction` commits independently and orphans rows |
| `is_contra` as a sign flip | Double-flips accumulated depreciation; balance sheet out by exactly 2× |
---
## 7. End-to-end flows
### 7.1 Passenger ticket sale
```
passenger-api payment-api finance-api
│ │ │
booking ──initiate──────► intent │
│ │ │
│ provider settles │
│ │ │
│◄──payment.passenger.succeeded──────────────►│
│ │ │
confirm booking │ Dr 1114 Gateway clearing
issue ticket │ Cr 1122 Trade receivable
monthly ──────────► recognize revenue
Dr 1122 / Cr 4210 Ticket revenue
(ONE summarised entry per period)
```
Revenue is recognised from `Booking.totalMinor` (genuine cents), **not** from
`PaymentIntent.amountMinor` — 18 rows there are 100× overstated by a
force-confirm path that writes cents into a major-unit column, and that path is
still live upstream.
The two halves meet at **1122 Trade Receivables**: recognition creates the
receivable, the payment clears it. A payment is not revenue, and posting both
would count the sale twice.
### 7.2 Freight shipment
Same shape, with `payment.freight.*` clearing **1121**. Revenue projects from
`freight.invoice_lines.charge_type` through `finance.revenue_mappings` (42 seeded
from freight's own 36-value canonical list). An unmapped charge type posts to
**4900 Unclassified Revenue** — a real, visible account, so an unexpected balance
there is the prompt to add a mapping rather than a silent misclassification.
Cash receipts read the invoice `payments` **jsonb**, not the `freight.payments`
table: the table is a one-row-per-booking gateway-intent projection updated in
place, while the jsonb is the only per-settlement ledger that exists.
### 7.3 Payroll cycle
```
hr-api finance-api
│ │
calculate run ──► APPROVED │
│ │
│◄────── read-only projection ───────────────┤
│ hr.payroll_runs / hr.payslips │
│ │
│ Dr 5110 Basic salary
│ Dr 5120 Allowances
│ Dr 5140 Employer pension
│ Cr 2121 PAYE payable
│ Cr 2122 Pension payable (ee + er)
│ Cr 2130 Salaries payable
│ Cr 2160 Other deductions
│ │
│ disbursement register (the list HR lacks)
│ │
│ remittance: Dr 212x / Cr cash
```
Only **APPROVED** runs are posted — a run that can still be recalculated would
leave the ledger describing a payroll that no longer exists. The entry is dated
the period **end**, because the cost belongs to the month worked even when the
money leaves later. `gross deductions = net` is checked explicitly before
posting, aggregated from the payslips rather than the run header: the employer's
pension is both a debit and part of the credit, and getting that wrong still
*balances*.
Finance stores no copy of the payroll. Only the journal link.
---
## 8. Known gaps — things this map does NOT claim work
Stated plainly, because a map that hides its blank areas is worse than no map.
1. **Refunds are never executed anywhere.** `freight.payment_refunds` has no
writer at all; passenger's `PaymentRefund` is never created; and
`BookingCancellation` rows are terminal at creation — `refundStatus` is never
updated and `processedAt` never set. The 80% cancellation refund is
*computed and recorded* but never paid. Finance therefore posts it as a
**liability provision (2150 Refunds Payable)**, never as cash. Do **not** add
a `payment.refunded` event until payment-api actually processes refunds.
2. **Cash outside the payment rails.** Agent counter cash (confirmed with no
PaymentIntent), excess-baggage `CASH_COLLECTED`, freight offline settlements,
and wallet top-ups. The wallet one is a control gap: `topUp` credits
unconditionally with no payment record and no transaction wrapper.
3. **The upstream money bugs were not fixed** — that was a deliberate decision.
Finance defends at its own projection boundary instead. The cost is that the
defence is permanent: the force-confirm path still writes bad rows.
4. ~~**Not verified against a live broker.**~~ **Closed 2026-08-22.** The
credentials had never existed: the local broker had only `guest` and the `/`
vhost, so both configured URLs failed auth. With an `edr` user and a
`payment` vhost created, the whole path was observed end to end — the
handler registers as `payment.events::payment.#::finance.payment-events`,
both exchanges and both queues are declared durable, and the DLQ really is
bound on the DLX. Six events were published through the shared
`paymentRoutingKey()` helper the real publisher uses: a passenger receipt
posted to 1122, a freight receipt to 1121 (so `payment.#` does catch a
second service), a REDELIVERY of the first eventId was deduplicated and did
not double-post, `payment.failed` was SKIPPED, a USD payment was recorded
FAILED rather than converted at a guessed rate, and an event with no
`eventId` was dead-lettered and actually arrived in the DLQ.
**Mind the vhost.** In an AMQP URI the path IS the vhost and the leading
slash is only a separator: `.../payment` means vhost `payment`, not
`/payment`. payment-api defaults to `amqp://localhost:5672/payment`, so
Finance must be on vhost `payment` too. Neither side errors when they
disagree — both connect with `wait: false`.
**What is still off:** `PAYMENT_RABBITMQ_URL` is absent from finance-api's
`.env` and was absent from `.env.example`, so `RevenueModule` skips the
RabbitMQ import entirely and live ingest is silently disabled. Both files now
document it. Setting it (plus `FINANCE_DEFAULT_ORG_ID`, without which the
consumer posts against `""` and every account lookup fails) is what turns
ingest on — that is go-live checklist step 3.
5. **Freight billing and `edr_payment` are absent from the dev replica.** Their
projections are shape-validated against entity-derived tables, not real data.
6. ~~**No browser render check** on any Finance screen.~~ **Closed 2026-08-22.**
All 13 screens were driven in Chromium: 20 tabs, 11 modals, and a full
draft → post → reverse journal cycle checked against the trial balance,
P&L, balance sheet, cash and general-ledger reports. No page errors and no
failed requests. Four defects were found and fixed, all invisible to
type-check and build — the largest being that the theme toggle drove only
Tailwind's `dark` class and never Mantine's colour scheme, so every page
rendered black-on-black in dark mode. Still NOT checked: the write paths for
suppliers, bills, assets, depreciation and budgets were opened but not
submitted, and no screen has been viewed below 1440px.
7. **P7 cutover: the tooling is BUILT (2026-08-22), the migration is not RUN.**
`finance.org_settings` holds a per-organization cutover date;
`GET /api/v1/cutover/readiness` answers five questions with live numbers
(suspense zero, no operational entry before the boundary, register agrees
with the ledger, migrated assets carry a period count, date set);
`POST /api/v1/cutover/opening-balances` turns pasted rows into a DRAFT
OPENING entry with the 3900 plug **calculated, never supplied**; and the
payment consumer now SKIPs any event settled before the cutover, recording
the reason — without that, a replayed backlog would post money the opening
balances already carry. Screen at `/cutover`.
What remains is the business act, not a build: choose the date, enter the
real balances, and post them. Two traps were found and fixed while building
this — a migrated fixed asset would silently never depreciate again, and the
suspense check was counting DRAFT lines because a status filter sat in a
LEFT JOIN's ON clause.
---
## 9. Before going live
1. Run freight's migrations against the target database so billing tables exist.
2. Boot payment-api so `edr_payment` is created, and confirm the broker vhost.
3. Set `PAYMENT_RABBITMQ_URL` and `FINANCE_DEFAULT_ORG_ID` on finance-api; verify
a real event reaches `finance.payment-events`.
4. Seed the chart of accounts and revenue mappings per organization.
5. Create the fiscal year (8 Jul 7 Jul) and its periods.
6. Set the cutover date on `/cutover` **before** entering anything — the
consumer needs it to know which events are already history.
7. Enter opening balances on `/cutover`, in as many batches as suits (cash,
receivables, payables, equity). Each becomes a DRAFT you post yourself; the
difference goes to **3900 Opening Balance Suspense**, which must end at
**zero**. That is the check that the migration was entered correctly.
Migrate fixed assets through the register with their accumulated
depreciation **and** their opening period count, and with no funding account
— the ledger side comes from the opening entry.
8. Work the readiness panel until every check passes.
9. Reconcile the first period end: trial balance balances, balance sheet
balances, and the asset register agrees with the ledger.