Muluhabt ERP modules

docs/integration-map.md

@@ -0,0 +1,397 @@
+# Integration map — HR, Finance and the operational systems
+
+> Section 5 of the ERP expansion. Written 2026-08-22, after HR (3.1–3.7) and
+> Finance (4.1–4.6) were built and verified.
+>
+> This describes how six services share one database without owning each other's
+> data. If something here contradicts the code, the code is the truth and this
+> file is a bug — fix it in the same PR.
+
+---
+
+## 1. The services
+
+| Service | Owns schema | Stack | Port |
+|---|---|---|---|
+| `edr-freight-api` | `freight` **and `iam`** | NestJS + TypeORM | 3001 |
+| `edr-passenger-api` | `passenger` | NestJS + Prisma | 4000 |
+| `edr-payment-api` | `edr_payment` | NestJS + TypeORM | 3003 |
+| `edr-hr-api` | `hr` | NestJS + TypeORM | 3005 |
+| `finance-api` | `finance` | NestJS + TypeORM | 3004 |
+| `edr-gps-tracker` | — | separate service | — |
+
+**One Postgres database, separated by schema.** Not one database per service.
+Everything below follows from that: the isolation is a discipline, not an
+infrastructure boundary, so it has to be enforced by rules people can read.
+
+`edr-freight-api` is the authoritative owner of `iam` — it ships the
+`iam:migration:*` scripts. No other service migrates that schema.
+
+---
+
+## 2. Who may write what
+
+This is the load-bearing table. Everything else is detail.
+
+| Schema | Written by | Read by |
+|---|---|---|
+| `iam` | freight-api (migrations); **HR and Finance only through IAM's own services** | everyone |
+| `freight` | freight-api | freight-api, finance-api (read-only) |
+| `passenger` | passenger-api | passenger-api, finance-api (read-only) |
+| `edr_payment` | payment-api | payment-api, finance-api (read-only) |
+| `hr` | hr-api | hr-api, finance-api (read-only) |
+| `finance` | **finance-api only** | finance-api |
+
+**Finance writes nothing but `finance`.** It is a reader of the whole platform
+and an owner of one schema. No source system writes `finance.*` either — there
+is no path by which freight could insert a journal entry, and that is deliberate:
+the ledger's invariants live in one service, and anything that bypassed it would
+bypass them too.
+
+**HR writes `iam`, but never directly.** `IamOperationsService` resolves IAM's
+own services out of the container (`ModuleRef.get(..., {strict: false})`) and
+calls them, so IAM's validation, transactions and audit trail all still apply.
+`IamDirectoryService` is read projections only.
+
+---
+
+## 3. The three integration mechanisms
+
+Only three. Anything that looks like a fourth is a mistake.
+
+### 3.1 Embedded module (in-process, transactional)
+
+`IamModule.forRoot({...})` is imported by freight-api, hr-api and finance-api.
+IAM is a library every service takes as a dependency, not a service they call.
+
+This is why: **the hire flow creates an IAM user, an employee, an
+employee-position and an HR profile in ONE transaction.** Over HTTP it could
+not, and a partial failure would strand an IAM user with no HR profile.
+
+Consequence, and it is intended: hr-api and finance-api both serve IAM's own
+routes, including `POST /api/v1/auth/login`. They are second front ends to IAM's
+capabilities, not forks of them.
+
+Registration must use **globs over both package dists** with
+`autoLoadEntities: false` — see `apps/finance-api/src/config/database.config.ts`.
+A hand-maintained entity list goes stale on every package bump; the partial
+`forFeature` set plus `autoLoadEntities` is what throws
+`Entity metadata for User#sessions was not found` at boot.
+
+### 3.2 Events (asynchronous, at-least-once)
+
+One exchange: **`payment.events`**, a durable topic exchange, with
+`payment.events.dlx` behind it.
+
+```
+                    ┌──────────────────┐
+                    │  edr-payment-api │
+                    │  transactional   │
+                    │     outbox       │
+                    └────────┬─────────┘
+                             │ publish  payment.<service>.<outcome>
+                   ┌─────────▼──────────┐
+                   │  payment.events    │  topic, durable
+                   └──┬───────┬──────┬──┘
+       payment.freight.*  payment.passenger.*  payment.#
+          ┌────────▼──┐  ┌──▼──────────┐  ┌──▼──────────┐
+          │ freight   │  │ passenger   │  │ finance     │
+          │ .payment- │  │ .payment-   │  │ .payment-   │
+          │  events   │  │  events     │  │  events     │
+          └───────────┘  └─────────────┘  └─────────────┘
+```
+
+**Routing keys are three words**: `payment.passenger.succeeded`,
+`payment.freight.failed` — built as `payment.${service}.${outcome}`. AMQP's `*`
+matches exactly one word, so a `payment.*` binding matches **nothing**. Finance
+binds `payment.#` because a ledger should see every payment.
+
+Finance declares its **own** queue (`finance.payment-events`). It must never
+share freight's or passenger's: a shared queue delivers each message to whichever
+consumer takes it first, and freight would start losing payments to the ledger.
+
+**Delivery is at-least-once.** The publisher is a transactional outbox with
+retry and backoff; `eventId` is the outbox row id and is stable across every
+redelivery. Every consumer must be idempotent — see §5.
+
+The envelope (`PaymentEvent` in `@edr/types`):
+
+| Field | Note |
+|---|---|
+| `version`, `eventId`, `eventType`, `occurredAt` | `eventId` is the dedupe key |
+| `service`, `referenceType`, `referenceId` | which domain order |
+| `merchantOrderId`, `intentId`, `provider` | provider-side identity |
+| `amountMinor`, `currency` | **carries MAJOR units despite the name** — see §4 |
+| `paidAt` / `failureCode` | per event type |
+
+### 3.3 Read-only cross-schema projection
+
+Finance reads `freight`, `passenger`, `hr` and `edr_payment` with
+schema-qualified raw SQL. This is sanctioned — it is the same approach HR's 3.7
+reports use, and it is justified by the single-database topology.
+
+It carries one obligation, from the platform's hard rules:
+
+> **Validate every raw SQL statement against a real database before shipping it.**
+> A typo'd column name is a runtime 500 no type-checker will catch.
+
+That rule earned its place twice during this build (§6).
+
+Because upstream schemas deploy independently, Finance **probes for source
+availability** (`to_regclass`) and reports it at `GET /api/v1/revenue/sources`.
+The UI then says "this source is not present" instead of showing a zero. On the
+current dev replica, freight's billing tables and the whole `edr_payment` schema
+are absent — "no freight revenue" and "freight billing is not deployed" are
+completely different facts and must not look alike.
+
+### 3.4 What is NOT an integration mechanism
+
+`${source}.invoice.paid`, `booking.cancelled` and similar are **in-process
+`EventEmitter2`** events inside a single app. They do not cross a service
+boundary and cannot be subscribed to from another service, despite reading like
+broker events. Note also that freight emits `lastmile.*`/`firstmile.*` while
+several listeners subscribe to `last_mile.*`/`first_mile.*` — a pre-existing
+mismatch, not something Finance relies on.
+
+---
+
+## 4. The money contract
+
+`apps/finance-api/src/common/money.ts` is the authority. Summary:
+
+| Store | Representation | Unit |
+|---|---|---|
+| `freight.*` | `numeric(14,2)` | **major** |
+| `passenger.*` Prisma `Int` columns | integer | **minor (cents)** |
+| `passenger."PaymentIntent".amountMinor` | double | **major**, despite the name |
+| `edr_payment.payment_intent.amount_minor` | double | **major**, despite the name |
+| payment events `amountMinor` | JSON number | **major**, despite the name |
+| `hr.*` payroll, `finance.*` | `numeric(14,2)` | **major** |
+
+**Never infer the unit from the name.** Every `*Minor` field on the payment path
+carries major units; the names are a wire contract across service APIs and are
+not worth a cross-service rename.
+
+**Currencies are never summed together.** Confirmed passenger bookings are
+charged in ETB **and DJF and USD** — 151.4M / 6.18M / 133k on the dev replica.
+Adding them reports 157.7M and overstates ETB revenue by **6.32M**. Finance
+groups every revenue projection by currency, posts only ledger-currency amounts,
+and reports the rest as "needs a rate" rather than converting at a guess.
+
+---
+
+## 5. Idempotency — how each path avoids double-counting
+
+Every automated write into the ledger is keyed, because at-least-once delivery
+and human re-clicks are both certain.
+
+| Path | Key | Where enforced |
+|---|---|---|
+| Payment event → cash receipt | `eventId` | unique index on `finance.inbound_events.event_id`; claimed with `INSERT … ON CONFLICT DO NOTHING` |
+| Any automated journal | `(organization, source_module, source_id)` | partial unique index on `finance.journal_entries` |
+| Revenue recognition | `('<source>-revenue', 'YYYY-MM')` | same index |
+| Payroll → GL | `('hr-payroll', <run id>)` | same index |
+| Supplier bill approval | `('supplier-bill', <bill id>)` | same index |
+| Depreciation | `('depreciation', <period id>)` | same index + one run per period |
+| Statutory remittance | `(type, period)` | unique index on `statutory_remittances` |
+
+The inbound-event claim is a **single statement**, not a read-then-write: two
+concurrent deliveries of the same event would both pass a prior `SELECT`.
+
+An event that cannot be posted is recorded **FAILED with its full payload** and
+acknowledged, not dead-lettered — it is already durably stored and replayable.
+Only a failure to *record at all* nacks, because then the broker holds the only
+copy.
+
+---
+
+## 6. Traps this integration has already paid for
+
+Each of these was found by an assertion, not by review. They are in the
+platform's `CLAUDE.md` "Known traps" table.
+
+| Trap | Consequence |
+|---|---|
+| `payment.*` binding | Receives **nothing** — keys are three words |
+| Summing across currencies | 6.32M overstatement on real data |
+| `DATE` via a JS `Date` | `toISOString()` returns the previous day east of UTC; a month-end lands in the wrong period |
+| `invoice_lines.line_total` | The column is `amount`; a plausible name that does not exist |
+| `iam.employees.first_name` | There is one JSONB `name` column, no first/middle/last |
+| `CHECK` wider than the column | `varchar(16)` accepted every status until the 17-character one was first used |
+| Nested transactions | An inner `dataSource.transaction` commits independently and orphans rows |
+| `is_contra` as a sign flip | Double-flips accumulated depreciation; balance sheet out by exactly 2× |
+
+---
+
+## 7. End-to-end flows
+
+### 7.1 Passenger ticket sale
+
+```
+passenger-api          payment-api            finance-api
+     │                      │                      │
+  booking ──initiate──────► intent                 │
+     │                      │                      │
+     │              provider settles               │
+     │                      │                      │
+     │◄──payment.passenger.succeeded──────────────►│
+     │                      │                      │
+ confirm booking            │             Dr 1114 Gateway clearing
+ issue ticket               │             Cr 1122 Trade receivable
+                                                   │
+                            monthly ──────────► recognize revenue
+                                          Dr 1122 / Cr 4210 Ticket revenue
+                                          (ONE summarised entry per period)
+```
+
+Revenue is recognised from `Booking.totalMinor` (genuine cents), **not** from
+`PaymentIntent.amountMinor` — 18 rows there are 100× overstated by a
+force-confirm path that writes cents into a major-unit column, and that path is
+still live upstream.
+
+The two halves meet at **1122 Trade Receivables**: recognition creates the
+receivable, the payment clears it. A payment is not revenue, and posting both
+would count the sale twice.
+
+### 7.2 Freight shipment
+
+Same shape, with `payment.freight.*` clearing **1121**. Revenue projects from
+`freight.invoice_lines.charge_type` through `finance.revenue_mappings` (42 seeded
+from freight's own 36-value canonical list). An unmapped charge type posts to
+**4900 Unclassified Revenue** — a real, visible account, so an unexpected balance
+there is the prompt to add a mapping rather than a silent misclassification.
+
+Cash receipts read the invoice `payments` **jsonb**, not the `freight.payments`
+table: the table is a one-row-per-booking gateway-intent projection updated in
+place, while the jsonb is the only per-settlement ledger that exists.
+
+### 7.3 Payroll cycle
+
+```
+hr-api                                    finance-api
+  │                                            │
+ calculate run ──► APPROVED                    │
+  │                                            │
+  │◄────── read-only projection ───────────────┤
+  │        hr.payroll_runs / hr.payslips       │
+  │                                            │
+  │                              Dr 5110 Basic salary
+  │                              Dr 5120 Allowances
+  │                              Dr 5140 Employer pension
+  │                                Cr 2121 PAYE payable
+  │                                Cr 2122 Pension payable (ee + er)
+  │                                Cr 2130 Salaries payable
+  │                                Cr 2160 Other deductions
+  │                                            │
+  │                              disbursement register (the list HR lacks)
+  │                                            │
+  │                              remittance: Dr 212x / Cr cash
+```
+
+Only **APPROVED** runs are posted — a run that can still be recalculated would
+leave the ledger describing a payroll that no longer exists. The entry is dated
+the period **end**, because the cost belongs to the month worked even when the
+money leaves later. `gross − deductions = net` is checked explicitly before
+posting, aggregated from the payslips rather than the run header: the employer's
+pension is both a debit and part of the credit, and getting that wrong still
+*balances*.
+
+Finance stores no copy of the payroll. Only the journal link.
+
+---
+
+## 8. Known gaps — things this map does NOT claim work
+
+Stated plainly, because a map that hides its blank areas is worse than no map.
+
+1. **Refunds are never executed anywhere.** `freight.payment_refunds` has no
+   writer at all; passenger's `PaymentRefund` is never created; and
+   `BookingCancellation` rows are terminal at creation — `refundStatus` is never
+   updated and `processedAt` never set. The 80% cancellation refund is
+   *computed and recorded* but never paid. Finance therefore posts it as a
+   **liability provision (2150 Refunds Payable)**, never as cash. Do **not** add
+   a `payment.refunded` event until payment-api actually processes refunds.
+2. **Cash outside the payment rails.** Agent counter cash (confirmed with no
+   PaymentIntent), excess-baggage `CASH_COLLECTED`, freight offline settlements,
+   and wallet top-ups. The wallet one is a control gap: `topUp` credits
+   unconditionally with no payment record and no transaction wrapper.
+3. **The upstream money bugs were not fixed** — that was a deliberate decision.
+   Finance defends at its own projection boundary instead. The cost is that the
+   defence is permanent: the force-confirm path still writes bad rows.
+4. ~~**Not verified against a live broker.**~~ **Closed 2026-08-22.** The
+   credentials had never existed: the local broker had only `guest` and the `/`
+   vhost, so both configured URLs failed auth. With an `edr` user and a
+   `payment` vhost created, the whole path was observed end to end — the
+   handler registers as `payment.events::payment.#::finance.payment-events`,
+   both exchanges and both queues are declared durable, and the DLQ really is
+   bound on the DLX. Six events were published through the shared
+   `paymentRoutingKey()` helper the real publisher uses: a passenger receipt
+   posted to 1122, a freight receipt to 1121 (so `payment.#` does catch a
+   second service), a REDELIVERY of the first eventId was deduplicated and did
+   not double-post, `payment.failed` was SKIPPED, a USD payment was recorded
+   FAILED rather than converted at a guessed rate, and an event with no
+   `eventId` was dead-lettered and actually arrived in the DLQ.
+
+   **Mind the vhost.** In an AMQP URI the path IS the vhost and the leading
+   slash is only a separator: `.../payment` means vhost `payment`, not
+   `/payment`. payment-api defaults to `amqp://localhost:5672/payment`, so
+   Finance must be on vhost `payment` too. Neither side errors when they
+   disagree — both connect with `wait: false`.
+
+   **What is still off:** `PAYMENT_RABBITMQ_URL` is absent from finance-api's
+   `.env` and was absent from `.env.example`, so `RevenueModule` skips the
+   RabbitMQ import entirely and live ingest is silently disabled. Both files now
+   document it. Setting it (plus `FINANCE_DEFAULT_ORG_ID`, without which the
+   consumer posts against `""` and every account lookup fails) is what turns
+   ingest on — that is go-live checklist step 3.
+5. **Freight billing and `edr_payment` are absent from the dev replica.** Their
+   projections are shape-validated against entity-derived tables, not real data.
+6. ~~**No browser render check** on any Finance screen.~~ **Closed 2026-08-22.**
+   All 13 screens were driven in Chromium: 20 tabs, 11 modals, and a full
+   draft → post → reverse journal cycle checked against the trial balance,
+   P&L, balance sheet, cash and general-ledger reports. No page errors and no
+   failed requests. Four defects were found and fixed, all invisible to
+   type-check and build — the largest being that the theme toggle drove only
+   Tailwind's `dark` class and never Mantine's colour scheme, so every page
+   rendered black-on-black in dark mode. Still NOT checked: the write paths for
+   suppliers, bills, assets, depreciation and budgets were opened but not
+   submitted, and no screen has been viewed below 1440px.
+7. **P7 cutover: the tooling is BUILT (2026-08-22), the migration is not RUN.**
+   `finance.org_settings` holds a per-organization cutover date;
+   `GET /api/v1/cutover/readiness` answers five questions with live numbers
+   (suspense zero, no operational entry before the boundary, register agrees
+   with the ledger, migrated assets carry a period count, date set);
+   `POST /api/v1/cutover/opening-balances` turns pasted rows into a DRAFT
+   OPENING entry with the 3900 plug **calculated, never supplied**; and the
+   payment consumer now SKIPs any event settled before the cutover, recording
+   the reason — without that, a replayed backlog would post money the opening
+   balances already carry. Screen at `/cutover`.
+
+   What remains is the business act, not a build: choose the date, enter the
+   real balances, and post them. Two traps were found and fixed while building
+   this — a migrated fixed asset would silently never depreciate again, and the
+   suspense check was counting DRAFT lines because a status filter sat in a
+   LEFT JOIN's ON clause.
+
+---
+
+## 9. Before going live
+
+1. Run freight's migrations against the target database so billing tables exist.
+2. Boot payment-api so `edr_payment` is created, and confirm the broker vhost.
+3. Set `PAYMENT_RABBITMQ_URL` and `FINANCE_DEFAULT_ORG_ID` on finance-api; verify
+   a real event reaches `finance.payment-events`.
+4. Seed the chart of accounts and revenue mappings per organization.
+5. Create the fiscal year (8 Jul – 7 Jul) and its periods.
+6. Set the cutover date on `/cutover` **before** entering anything — the
+   consumer needs it to know which events are already history.
+7. Enter opening balances on `/cutover`, in as many batches as suits (cash,
+   receivables, payables, equity). Each becomes a DRAFT you post yourself; the
+   difference goes to **3900 Opening Balance Suspense**, which must end at
+   **zero**. That is the check that the migration was entered correctly.
+   Migrate fixed assets through the register with their accumulated
+   depreciation **and** their opening period count, and with no funding account
+   — the ledger side comes from the opening entry.
+8. Work the readiness panel until every check passes.
+9. Reconcile the first period end: trial balance balances, balance sheet
+   balances, and the asset register agrees with the ledger.