customers, contracts, invoices, payments, train schedules, and locomotives /
trains / wagons via the fleet page's config.
FilterBar pages pass controls.params into the children slot. The four pages
still on ad-hoc filtering pass their own hand-built filter object instead,
which is why ExportButton takes plain params rather than a UseFilters — it
would otherwise have been blocked behind migrating those pages. FleetResource
serves seven slugs from config, so it gets an optional exportKey there and
renders nothing for the four slugs with no dataset yet.
Auditing each page's real filter keys against the dataset declarations turned
up three gaps where an on-screen filter would have silently not applied to the
export: invoices sends a singular "status" (the dataset only had the
multiselect "statuses"), contracts sends paymentCurrency, serviceTypeId and
route origin/destination, and train schedules sends freightType. Added all of
them — contract routes filter through EXISTS on contract_routes since they are
one-to-many, and train-schedule freightType through EXISTS on the bookings
aboard, matching the list service.
Verified in the browser: the button renders on each page, and the invoices
dialog follows that page's own filter object — selecting Paid moves the count
from 126 to 100, which matches the database. Filter pass-through checked
against the database for invoices, payments, wagons, contracts and train
schedules.
customers, contracts, invoices, payments, train-schedules, locomotives,
trains and wagons. 319 fields across the nine datasets, all reusing the
existing engine — no change to export.types.ts was needed, which is the
result the bookings-first phase was meant to test.
Per-dataset notes worth keeping:
- trains resolves route, stations and current yard, which the list endpoint
never loads — the UI shows raw FK uuids there today.
- wagons reads tare/payload/length off wagon_types (they are not on the
wagon), and reproduces the service's attachStatusDates() as correlated
subqueries. wagon_status_logs stores from_status/to_status, not status.
- payments applies no soft-delete guard: freight.payments has neither
deleted_at nor updated_at, so the usual predicate is a 42703. Failure
columns are failer_code/failer_message. payment_refunds stores MINOR
units, so refundedTotal divides by 100.
- train-schedules derives freightType from the bookings aboard rather than
a column, matching the list service.
- customers stays one row per company; profiles, bookings and invoice
totals aggregate in subqueries. Verified no row multiplication: trains,
customers and contracts each return exactly their counted row count while
selecting one-to-many aggregate fields.
EXPLAIN-validated against the database: every dataset's widest query, its
count query, and all 319 fields individually. That run caught five columns
typed varchar rather than timestamp (companies.date_registered,
renewal_date, renewed_from, renewed_to and invoices.eims_ack_date), which
were being pushed through to_char and would have 500'd the moment anyone
ticked them; they now export verbatim.
All nine count endpoints verified equal to SELECT count(*) on their table.
Adds a parallel export system the reports module can also draw on. A dataset
describes a table's exportable fields — including related-entity detail the
list page never shows — and the engine assembles a query from whichever fields
the caller picked.
GET /exports catalog (metadata only; select/requires never ship)
GET /exports/:key/count exact row count + per-format caps
GET /exports/:key/download csv | xlsx | pdf
Two invariants carry the design:
- Every lazy join is a LEFT join, and ExportJoin has no 'kind' field to make
anything else expressible. An inner join added because a checkbox was ticked
would change the rowset, so two exports of the same filters would disagree on
their row count.
- Because of that, the count cannot depend on field selection, so /count runs
base + alwaysJoin only and is exact rather than an estimate. Verified: count
and the delivered file both report 223 rows.
One-to-many relations (a booking's containers) aggregate in a correlated
subquery rather than joining, so a row can never multiply.
Export rides each dataset's existing view permission — no new permission keys
and no seeder change. Sensitive columns are simply never declared as fields:
raw gateway payloads, signature blobs, error dumps, raw jsonb snapshots,
internal user UUIDs and review notes are all absent by construction.
bookings ships 77 fields across 10 groups. scripts/validate-export-datasets.ts
EXPLAINs every dataset's widest query, its count query, and each field on its
own against the real database — the per-field pass is what catches a field
referencing a join it forgot to declare, which otherwise only fails when that
one field is picked alone.
The export path used one number for two different things: the format's hard
row cap, and the caller's explicit 'give me the first N rows'. Because
resolveExportCap() returned min(requested, formatCap) and runAll() then threw
when the result reached it, picking 'Records: First 100' in the export dialog
400'd on any report with more than 100 rows — the user asked to be truncated
and got an error instead.
Splits them: formatRowCap() is the hard, non-caller-controllable ceiling that
still throws when exceeded (a silently short file hides missing rows), while
resolveRowLimit() is the deliberate truncation and is honoured by slicing.
Verified against a 223-row dataset: limit=5 now returns 5 rows, and no limit
returns all 223.
Completes the writer extraction whose other half landed in fb21ad154.
reports.controller now builds a TabularDoc and calls TabularExportService,
so report-export.service.ts and report-export-request.util.ts are dead and
removed — HEAD was carrying both copies with the controller still on the old
one.
Reports gain CSV for free, and the PDF path now passes buildTabularFallbackPdf
as its fallback: previously it passed none, so a box without Chromium silently
returned PdfRenderService's ~900-character generic text dump instead of a
table. Adds a spec covering the CSV writer's quoting of embedded commas and
double quotes — the reason this uses ExcelJS's csv writer rather than a
hand-rolled join.
New endpoints:
POST invoices/eims/bulk-register { invoiceIds: [...] } — trigger
POST eims/webhook/bulk-register — MoR's callback
Fundamentally different shape from single register: bulkRegister
answers only {conversationId, status:202} immediately: MoR processes
the array asynchronously and pushes the real per-invoice results
(a mix of accepted/rejected in one array, per the collection's own
examples) to a webhook configured out of band. So this ships as two
halves that don't share a call stack — EimsBulkRegistrationService.
registerBulk() reserves a contiguous block of counters (durable
reservation, same doctrine as single register, extended to N items)
and submits; handleBulkCallback(), invoked by the new
EimsWebhookController whenever MoR gets around to it, settles.
New EimsSystemState.inFlightConversationId is the bulk equivalent of
inFlightInvoiceId — a whole batch outstanding, not one invoice — and
the two markers block each other since they share the same counter
sequence. The conversation id isn't known until MoR's 202 arrives, so
reservation stamps a locally-generated placeholder first (same
commit-before-the-network-call reasoning as single register), then
swaps it for MoR's real id right after — the only value the callback
can actually use to find the batch again.
Only the first invoice in a bulk batch chains via PreviousIrn — every
other item gets an empty string, matching the collection's own
two-invoice example exactly (MoR doesn't expect a batch to chain to
IRNs that don't exist yet at submission time).
Webhook has no auth (MoR has no JWT to send) — the conversation id
embedded in the payload is what stands between this and a forged
callback: an item only ever touches an invoice actually holding that
exact id, and an unknown id is logged and ignored, never applied.
Migration 3580000000000: eims_system_state.in_flight_conversation_id,
invoices.eims_bulk_conversation_id (tags which batch an invoice was
submitted in, so a stuck batch — webhook never arrived — can be found
and reconciled by conversation id). Applied to dev DB and recorded in
freight.migrations directly (idempotent IF NOT EXISTS DDL).
Not live-testable from this sandbox (no route to MoR's real gateway).
Signing the whole array as one envelope, the way single /v1/register
was confirmed live to need despite the collection's raw example
showing no envelope, is the reasonable extension of that confirmed
behavior, not a blind guess — but it has not itself been exercised
against the real gateway. Left for the first live bulk attempt to
confirm, same as every other MoR-facing assumption this integration
has made.
- Add migration for consolidation approvals table and status enum
- Create ConsolidationApprovalService to handle approval logic
- Implement repository for managing consolidation approvals
- Add entity for consolidation approval with necessary fields
- Develop frontend components for displaying and managing consolidation approvals
- Create tests for consolidation approval service to ensure correct behavior
- Add migration for consolidation approvals table and status enum
- Create ConsolidationApprovalService to handle approval logic
- Implement repository for managing consolidation approvals
- Add entity for consolidation approval with necessary fields
- Develop frontend components for displaying and managing consolidation approvals
- Create tests for consolidation approval service to ensure correct behavior
- Added support for viewing and managing consolidated bookings in BookingRequestDetailPage.
- Enhanced BookingRequestsPage to display paired bookings in a single row.
- Introduced pairedDecision method in bookings service to handle decisions for both halves of a consolidated pair.
- Updated contracts service to include methods for manual consolidation of odd-20ft bookings.
- Created new components for selecting and editing consolidation partners.
- Added tests for paired decision logic and manual consolidation scenarios.
- Updated UI to reflect changes in booking handling and provide user feedback for odd container counts.
Confirmed live 2026-08-17: buyer profiles store just the sub-city name
("Bole", "Arada") as their woreda, never the source CSV's specific
example-woreda name ("Bole Woreda 01") — hit for three different
buyers in a row (Bole, Arada, Lemi Kura) before any got past this
check. Since the CSV lists exactly one representative woreda per
Addis sub-city, alias the bare name to that same code instead of
waiting on a fuller table — use what's already been supplied first.
Lemi Kura itself isn't in the source CSV at all (one of Addis
Ababa's newer sub-cities) — still needs its real code from MoR.
Both manual-registration routes exist for one reason — eTrade holds no record
to fetch — so leaving one is the same act whichever it is. A co-operative that
has since taken out a trade licence had no way out at all: the flag is chosen
in the wizard, and an onboarded company can no longer reach it.
revert-to-etrade now accepts either flag and strips both, and a co-operative's
ACTIVE roles go back to Pending with it: those approvals were granted without a
per-role business licence, because a co-op owes none, and the moment it stops
being one that licence is due. Rejected, suspended and blacklisted roles are
left alone — promoting a blocked role to "awaiting approval" would launder a
staff decision away. An investor's roles are untouched; their licences were
always due and already reviewed.
Switching INTO a route stays with the wizard's nationality/role step, which
owns the mutually-exclusive rules; a second entry point would restate all three.
Going back in the wizard and un-ticking co-operative or investment licence used
to write the flag and nothing else. The registration the customer had typed
stayed on the company row, so `hasRegistrationDetails` still read as a passed
eTrade lookup, resume dropped them at their furthest step rather than the
company one, and the application could be finished on unverified data with no
flag left on it for the backoffice to show.
That transition now costs what the settings switch costs: the eTrade-sourced
columns and the manager captured beside them are cleared, and onboarding drops
back to the company step so the TIN actually goes through eTrade. Both the
reset payload and the attribute strip are now shared with
`revertToRegularCompany`, which did this correctly already.
A foreign company licensed by the Ethiopian Investment Commission is not on
the trade registry, so eTrade holds no record for its TIN and the lookup the
company step is built around returns nothing. Those customers could not get
past onboarding at all.
They now take the same route a co-operative does: an `investorLicence` flag in
`attributes`, and `applyEtradeSourcedFields` skips the eTrade re-check for any
company `usesManualRegistration` covers, so the registration they type is
persisted as sent instead of 400'ing "no registration found for this TIN".
Unlike a co-operative they still hold a business licence per operational role,
so that requirement is untouched, and the foreign document set already asks for
the investment licence itself — no new set.
Only a foreign company may carry the flag, and never alongside the
co-operative one: the two resolve to different document sets.
`POST /companies/onboarding/revert-to-etrade` gives it back. It clears the
typed registration rather than keeping it — the wizard treats a populated
registration block as a passed lookup, so leaving it would walk the customer
straight past the eTrade step the switch exists to reach — and returns the
company to pending, since an approval granted against typed data must not
carry over to a record that now claims to be eTrade's.
Confirmed live 2026-08-17 on INV-20260817-00008: MoR rejected the
document with a SCHEMA ERROR on ItemList[0].Unit — 'PER_CONTAINER'
(from the line's own metadata.unit) fails MoR's enum
(LTR/MTR/101/PCS/ROL/MTS/PKG/SET/KLG), its 8-char max, and its
^[A-Za-z]{3,8}$ regex all at once.
line.metadata.unit is our own fee-basis tag (PER_CONTAINER/PER_TON/
PER_ITEM — how a charge is computed) and was never a MoR unit of
measure; the mapper was reusing the same field name for two unrelated
concepts. Every line now sends the single configured
EIMS_UNIT_DEFAULT instead of guessing a per-line value that doesn't
exist in MoR's vocabulary.
New endpoint: POST invoices/eims/bulk-cancel, body { items: [{invoiceId,
reasonCode, remark?}] }. Same eimsCancel permission as single cancel — a
batch-scale version of the same irreversible-at-MoR action, not a new
capability.
Same local-eligibility doctrine as single cancel: an already-cancelled or
never-registered invoice is refused right here, no HTTP call, before it
gets a seat in the batch. Only genuinely eligible invoices go into the one
/v1/bulkCancel request; every outcome (local refusal or MoR's own
per-IRN result) is reported back independently — one invoice failing
never blocks the rest.
MoR's bulk response mixes success and error shapes in the same array,
disambiguated by Status (capital, error) vs status (lowercase, success)
— matched back to our invoices by IRN. Notably the bulk success shape
carries no cancellationDate at all, unlike single cancel.
Left out of this pass: bulkRegister. It's async (returns only a
conversationId immediately, results arrive via a webhook callback we
don't have yet) and needs manual counter/previousIrn management per
the collection's own docs — a materially different reservation model
than today's single-invoice TX1/TX2 pattern. Scoping that is a
separate, bigger piece of work.
EIMS_BUYER_REGION_CODES/WEREDA_CODES/CITY_CODES were hand-maintained,
empty-by-default env vars — every buyer from a not-yet-seen area
crashed filing until someone found the MoR code and redeployed.
Happened three times in one afternoon (2026-08-17): Somali region,
Fafan zone, Jigjiga woreda, even the Ethiopia country code were all
unset on the triaplc.com deployment.
Ethiopia's administrative divisions are fixed reference data, not
buyer-specific config. Added ethiopia-geo-codes.ts, a static table
(source: ethiopia_administrative_hierarchy_master.csv, supplied
2026-08-17 — a representative sample, not all ~1000 real woredas)
merged in as the fallback under the existing env vars, which still
win on a name collision — for a quick correction without a redeploy,
or a buyer profile spelled differently than the table (already hit:
DB has zone "Fafen", official spelling "Fafan").
Known limitation, documented in the file: zone/woreda names aren't
always unique nationwide (e.g. "North Shewa" is both an Amhara and
an Oromia zone) and Company stores region/zone/woreda as three
independent strings with no parent linkage, so a flat name map can't
always disambiguate. Only affects the optional City field — never
blocks filing, unlike Region/Wereda.
Conflict in ClearanceDocumentsPage: this branch migrated the page to the
pill FilterBar, dev added filters to the Select stack it replaced. Kept
the FilterBar and carried dev's additions across as a "Booked by"
(customerKind) FilterDef plus the shipping-line search placeholder; dev's
startOfDayIso/endOfDayIso went away because dateRangeParams already does
that. The Ship icon import is needed by dev's shipping-line customer cell,
which merged cleanly on its own.
Three gaps on the backoffice customer detail page:
- Rejecting a change request or sending it back for correction notified
nobody. Adds CompanyNotifierService.changeRequestReturned, which pings
the customer desk with the reviewer, the outcome and the note. Marketing
joins that desk via customers:view + customers:get_notification in the
role preset — grants still come from the IAM UI, the preset only sets
the default for new environments.
- submitted_by / reviewed_by / actor_id were stored but never resolved, so
the History tab could say what changed but never who asked or who sent
it back. Resolves them through a shared iam-user-name util (deduped from
the private copy in contract-document-history.service) and renders
"Requested by" / "Sent back to marketing by" lines. The
changes_requested badge is relabelled to match the workflow.
- "View" opened an in-page modal one document at a time. Adds
openFileInNewTab, which opens the tab inside the click gesture and fills
it once the authenticated fetch resolves, and an "Open all" button that
loops over the documents table so every file lands in its own tab.
toEimsInvoice/buildEimsContext sat outside the try/catch that calls
settleFailure — reservation happens (TX1), then request-building ran
unguarded, then submit() was the only thing actually wrapped. Any
exception during mapping (a validation error like an unmapped buyer
country, or a bug) skipped settleFailure entirely and left the
reservation permanently held: exactly the live incident just seen —
register 500'd, and every subsequent attempt on any invoice 409'd
'already in flight' until manually resolved.
Fix: the try block now starts right after reserve(), covering
request-building and submit() both. settleFailure's determinism
check is generalized to match — any error that is not an
EimsApiException is pre-wire and safe to release, not just
EimsConfigException (still labeled CONFIG; everything else pre-wire
is now labeled the new LOCAL kind). This is exhaustive by
construction: every error that actually touches the wire is already
normalized to EimsApiException inside EimsClientService.send()'s own
catch, so nothing outside that can be ambiguous.
settleFailure() treated any non-EimsApiException error as ambiguous
("might have reached MoR") and permanently blocked all further
filing until manually resolved. EimsConfigException (bad/missing
key, unparseable cert) is thrown by EimsSignerService before
EimsClientService.send()'s try/catch is even entered — by
construction it never reached the wire, so there is nothing
ambiguous about it.
This is exactly what happened live: a private-key parse failure
during the key/cert migration work reserved a counter, failed before
any HTTP call, and got treated as an unresolved in-flight submission
— blocking every other invoice from filing until someone manually
POSTs /eims/resolve.
Fix: EimsConfigException is now deterministic in settleFailure, same
treatment as a clean MoR rejection — both counters roll back, no
system-wide block, invoice marked FAILED (not UNKNOWN). Added a
CONFIG failure kind so the invoice's eimsLastError and the staff
alert both say plainly that the request never reached MoR, instead
of implying a MoR rejection.
