customers, contracts, invoices, payments, train schedules, and locomotives /
trains / wagons via the fleet page's config.
FilterBar pages pass controls.params into the children slot. The four pages
still on ad-hoc filtering pass their own hand-built filter object instead,
which is why ExportButton takes plain params rather than a UseFilters — it
would otherwise have been blocked behind migrating those pages. FleetResource
serves seven slugs from config, so it gets an optional exportKey there and
renders nothing for the four slugs with no dataset yet.
Auditing each page's real filter keys against the dataset declarations turned
up three gaps where an on-screen filter would have silently not applied to the
export: invoices sends a singular "status" (the dataset only had the
multiselect "statuses"), contracts sends paymentCurrency, serviceTypeId and
route origin/destination, and train schedules sends freightType. Added all of
them — contract routes filter through EXISTS on contract_routes since they are
one-to-many, and train-schedule freightType through EXISTS on the bookings
aboard, matching the list service.
Verified in the browser: the button renders on each page, and the invoices
dialog follows that page's own filter object — selecting Paid moves the count
from 126 to 100, which matches the database. Filter pass-through checked
against the database for invoices, payments, wagons, contracts and train
schedules.
customers, contracts, invoices, payments, train-schedules, locomotives,
trains and wagons. 319 fields across the nine datasets, all reusing the
existing engine — no change to export.types.ts was needed, which is the
result the bookings-first phase was meant to test.
Per-dataset notes worth keeping:
- trains resolves route, stations and current yard, which the list endpoint
never loads — the UI shows raw FK uuids there today.
- wagons reads tare/payload/length off wagon_types (they are not on the
wagon), and reproduces the service's attachStatusDates() as correlated
subqueries. wagon_status_logs stores from_status/to_status, not status.
- payments applies no soft-delete guard: freight.payments has neither
deleted_at nor updated_at, so the usual predicate is a 42703. Failure
columns are failer_code/failer_message. payment_refunds stores MINOR
units, so refundedTotal divides by 100.
- train-schedules derives freightType from the bookings aboard rather than
a column, matching the list service.
- customers stays one row per company; profiles, bookings and invoice
totals aggregate in subqueries. Verified no row multiplication: trains,
customers and contracts each return exactly their counted row count while
selecting one-to-many aggregate fields.
EXPLAIN-validated against the database: every dataset's widest query, its
count query, and all 319 fields individually. That run caught five columns
typed varchar rather than timestamp (companies.date_registered,
renewal_date, renewed_from, renewed_to and invoices.eims_ack_date), which
were being pushed through to_char and would have 500'd the moment anyone
ticked them; they now export verbatim.
All nine count endpoints verified equal to SELECT count(*) on their table.
Adds a parallel export system the reports module can also draw on. A dataset
describes a table's exportable fields — including related-entity detail the
list page never shows — and the engine assembles a query from whichever fields
the caller picked.
GET /exports catalog (metadata only; select/requires never ship)
GET /exports/:key/count exact row count + per-format caps
GET /exports/:key/download csv | xlsx | pdf
Two invariants carry the design:
- Every lazy join is a LEFT join, and ExportJoin has no 'kind' field to make
anything else expressible. An inner join added because a checkbox was ticked
would change the rowset, so two exports of the same filters would disagree on
their row count.
- Because of that, the count cannot depend on field selection, so /count runs
base + alwaysJoin only and is exact rather than an estimate. Verified: count
and the delivered file both report 223 rows.
One-to-many relations (a booking's containers) aggregate in a correlated
subquery rather than joining, so a row can never multiply.
Export rides each dataset's existing view permission — no new permission keys
and no seeder change. Sensitive columns are simply never declared as fields:
raw gateway payloads, signature blobs, error dumps, raw jsonb snapshots,
internal user UUIDs and review notes are all absent by construction.
bookings ships 77 fields across 10 groups. scripts/validate-export-datasets.ts
EXPLAINs every dataset's widest query, its count query, and each field on its
own against the real database — the per-field pass is what catches a field
referencing a join it forgot to declare, which otherwise only fails when that
one field is picked alone.
The export path used one number for two different things: the format's hard
row cap, and the caller's explicit 'give me the first N rows'. Because
resolveExportCap() returned min(requested, formatCap) and runAll() then threw
when the result reached it, picking 'Records: First 100' in the export dialog
400'd on any report with more than 100 rows — the user asked to be truncated
and got an error instead.
Splits them: formatRowCap() is the hard, non-caller-controllable ceiling that
still throws when exceeded (a silently short file hides missing rows), while
resolveRowLimit() is the deliberate truncation and is honoured by slicing.
Verified against a 223-row dataset: limit=5 now returns 5 rows, and no limit
returns all 223.
Completes the writer extraction whose other half landed in fb21ad154.
reports.controller now builds a TabularDoc and calls TabularExportService,
so report-export.service.ts and report-export-request.util.ts are dead and
removed — HEAD was carrying both copies with the controller still on the old
one.
Reports gain CSV for free, and the PDF path now passes buildTabularFallbackPdf
as its fallback: previously it passed none, so a box without Chromium silently
returned PdfRenderService's ~900-character generic text dump instead of a
table. Adds a spec covering the CSV writer's quoting of embedded commas and
double quotes — the reason this uses ExcelJS's csv writer rather than a
hand-rolled join.
Conflict in ClearanceDocumentsPage: this branch migrated the page to the
pill FilterBar, dev added filters to the Select stack it replaced. Kept
the FilterBar and carried dev's additions across as a "Booked by"
(customerKind) FilterDef plus the shipping-line search placeholder; dev's
startOfDayIso/endOfDayIso went away because dateRangeParams already does
that. The Ship icon import is needed by dev's shipping-line customer cell,
which merged cleanly on its own.
Three gaps on the backoffice customer detail page:
- Rejecting a change request or sending it back for correction notified
nobody. Adds CompanyNotifierService.changeRequestReturned, which pings
the customer desk with the reviewer, the outcome and the note. Marketing
joins that desk via customers:view + customers:get_notification in the
role preset — grants still come from the IAM UI, the preset only sets
the default for new environments.
- submitted_by / reviewed_by / actor_id were stored but never resolved, so
the History tab could say what changed but never who asked or who sent
it back. Resolves them through a shared iam-user-name util (deduped from
the private copy in contract-document-history.service) and renders
"Requested by" / "Sent back to marketing by" lines. The
changes_requested badge is relabelled to match the workflow.
- "View" opened an in-page modal one document at a time. Adds
openFileInNewTab, which opens the tab inside the click gesture and fills
it once the authenticated fetch resolves, and an "Open all" button that
loops over the documents table so every file lands in its own tab.
toEimsInvoice/buildEimsContext sat outside the try/catch that calls
settleFailure — reservation happens (TX1), then request-building ran
unguarded, then submit() was the only thing actually wrapped. Any
exception during mapping (a validation error like an unmapped buyer
country, or a bug) skipped settleFailure entirely and left the
reservation permanently held: exactly the live incident just seen —
register 500'd, and every subsequent attempt on any invoice 409'd
'already in flight' until manually resolved.
Fix: the try block now starts right after reserve(), covering
request-building and submit() both. settleFailure's determinism
check is generalized to match — any error that is not an
EimsApiException is pre-wire and safe to release, not just
EimsConfigException (still labeled CONFIG; everything else pre-wire
is now labeled the new LOCAL kind). This is exhaustive by
construction: every error that actually touches the wire is already
normalized to EimsApiException inside EimsClientService.send()'s own
catch, so nothing outside that can be ambiguous.
settleFailure() treated any non-EimsApiException error as ambiguous
("might have reached MoR") and permanently blocked all further
filing until manually resolved. EimsConfigException (bad/missing
key, unparseable cert) is thrown by EimsSignerService before
EimsClientService.send()'s try/catch is even entered — by
construction it never reached the wire, so there is nothing
ambiguous about it.
This is exactly what happened live: a private-key parse failure
during the key/cert migration work reserved a counter, failed before
any HTTP call, and got treated as an unresolved in-flight submission
— blocking every other invoice from filing until someone manually
POSTs /eims/resolve.
Fix: EimsConfigException is now deterministic in settleFailure, same
treatment as a clean MoR rejection — both counters roll back, no
system-wide block, invoice marked FAILED (not UNKNOWN). Added a
CONFIG failure kind so the invoice's eimsLastError and the staff
alert both say plainly that the request never reached MoR, instead
of implying a MoR rejection.
EIMS_PRIVATE_KEY / EIMS_CERTIFICATE — the PEM text pasted directly,
no encode/decode step at all. Precedence: raw PEM > base64 > path.
Motivated by the base64 path hitting a DECODER::unsupported error in
a live deployment with no way to tell whether the cause was transport
truncation, double-encoding, or an actually-bad file. Two fixes for
that class of problem together:
- the raw-PEM var removes the encode/decode step entirely, so
there's nothing left to corrupt in transit
- a literal \\n (two chars) is unescaped to a real newline, for
env stores that can't hold a literal line break
- getPrivateKey() now checks the decoded bytes look like a PEM
header before handing them to OpenSSL, so a still-bad value fails
with byte count + safe preview instead of an opaque decoder error
EIMS_PRIVATE_KEY_BASE64 / EIMS_CERTIFICATE_BASE64, alternative to the
existing _PATH vars. Wins over the path when set; falls back to the
file otherwise. Neither var required at boot on its own — the
either/or check moved out of the flat REQUIRED_VARS list.
Lets a dockerized deployment receive the key/cert the same way it
already receives every other EIMS_* secret (plain env var into the
container) instead of needing a host bind mount into the container
filesystem.
- Updated API documentation and summaries to reflect support for both USD and ETB invoices.
- Modified data structures to include trade direction for invoices.
- Adjusted UI components to accommodate manual payment confirmations and display relevant information.
- Implemented filtering options for currency in the manual payments worklist.
Ten EIMS_SELLER_* env vars were the only source of EDR's own seller identity,
duplicating data the platform already has via the same e-Trade lookup used
for every customer company at onboarding. EimsSellerCacheService now enriches
it — but static config remains the source of truth: MoR validates
SellerDetails against its own taxpayer registry (rule 7017, already cleared
against the current static values), so e-Trade fills a field only when the
static value is blank, never overrides one already confirmed. The static
config is therefore the durable fallback, not the cache; an in-memory
snapshot lost on restart is harmless.
ETradeService has no request timeout of its own and no AbortController, so
the cache enforces one locally (stops waiting, doesn't cancel the request)
and de-duplicates concurrent refresh() calls into the same in-flight promise.
getSellerDetails() is fully synchronous — zero I/O — so live registration
never depends on e-Trade being reachable, at boot or per invoice.
VatNumber and Email stay on static config permanently — confirmed by reading
e-Trade's actual response shapes, neither field exists anywhere in what it
returns. Region/Wereda/City reuse the existing EIMS_BUYER_*_CODES maps rather
than adding seller-specific ones — the geography is objective, not
buyer-specific.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
City: EimsMapperContext.buyerCity was declared but never wired anywhere —
always null, silently, for every buyer. No dedicated city column on Company;
derives from Zone via a new EIMS_BUYER_CITY_CODES map, same lookup mechanism
as Region/Wereda but optional (an unmapped zone resolves to null rather than
throwing) — MoR has already accepted a live filing with City null.
Country: previously a single flat EIMS_BUYER_COUNTRY_CODE applied to every
buyer regardless of Company.country. Now reads company.country, resolved via
a new EIMS_BUYER_COUNTRY_CODES name-to-code map; the flat env var becomes a
domestic-only fallback (applies only when country is empty/Ethiopia), so an
unmapped foreign buyer fails locally instead of silently filing as Ethiopia.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
scrollHeight is defined as the larger of an element's content height and its
own (viewport) height — for a receipt shorter than the placeholder 1123px
viewport, it silently returned the viewport height back, producing a
correctly-formatted but page-length-tall PDF with a huge trailing blank
strip below the real content. Found by actually rendering one and looking
at it, not caught by unit tests (buildThermalHtml is pure string output,
never exercises page.pdf() sizing).
Fix: use a deliberately tiny (100px) viewport height for the thermal
measurement pass, forcing content to overflow it so scrollHeight always
reflects the receipt's real height. Also round the computed mm value before
templating it into the CSS length string.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
GET billing/invoices/:id/document?format=thermal renders a dedicated 80mm
receipt template (72mm printable, 4mm margins each side), not a CSS variant
of the A4 layout — the A4 CSS is absolutely-positioned/fixed-px, tuned for a
210mm page, and doesn't reflow at thermal width. No seal (not a thermal
convention, renders badly on 1-bit thermal heads); line items stack
(description, then qty x rate = amount) instead of a table, since a real
table leaves ~10-14 chars for description at this width.
PdfRenderService gains a thermal render path: full 80mm-width viewport,
content height measured via page.evaluate after settle (continuous-roll
receipts have no fixed page length), and a noFallback option — a Chromium
failure throws a clear error instead of silently degrading to the generic
A4/no-QR fallback, which would hand back a different document than what was
asked for. The frontend surfaces that as a toast pointing at the existing A4
download.
format is strictly validated (a4|thermal only, BadRequestException
otherwise), not silently coerced.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
eims-receipt-document.mapper.ts maps an EimsReceipt onto the shared
InvoiceDocumentModel layout, reading amounts back out of the stored request
body. Refuses to render anything not REGISTERED. GET
invoices/:id/eims/receipts/:receiptId/document, scoped to the invoice.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
POST billing/invoices/:id/memo files a MoR DEB/CRE memo by reusing
createInvoice unchanged. sourceId is the original invoice's own id, not its
source's — this structurally keeps memos out of findPayable/expirePayable/
billQuery's sourceId-keyed lookups regardless of status. Credit notes are
created settled; debit notes are created open/unpaid as a genuine new
receivable, not force-settled. memoIssue is granted to the chief position,
not the general finance role.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
A BadRequestException thrown before reserve() (config assertion, DEB/CRE
validation) left the invoice NOT_SUBMITTED with nothing persisted, so the
same row was retried every tick forever — a permanent head-of-line block on
every invoice behind it. Now marked FAILED, guarded by a fresh status
re-read so a reservation's own SUBMITTING/UNKNOWN/blocked state is never
clobbered.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- New "route" FilterType: RouteBody popover (searchable origin +
destination selects, apply once both are picked), wired into
ContractRequestsPage and BookingRequestsPage. Bookings already had
server-side originYardId/destinationYardId; contracts gets both new
(contract_routes is one-to-many, so origin/destination are separate
EXISTS subqueries, not a join).
- Inactive FilterPill trigger restyled to read like a closed Mantine
Select (opaque solid border, trailing chevron) instead of a dashed
"+" pill — trigger only, popover body/position unchanged.
- Search box text set to regular weight.
- A couple more filters (Direction, Freight) pinned by default per
page on top of the existing always-pinned ones; the rest stay behind
More filters.
Replace the ad-hoc filter controls with a URL-linkable pill filter bar:
each filter is a pill that opens a type-aware popover (text/enum/date/
number/boolean, each with the right operator set), overflow filters live
behind a searchable "More filters" menu, sorting is a separate control,
and filter state round-trips through the URL query string (shareable,
back/forward-safe, backward compatible with existing ?statuses=A,B links).
Frontend (apps/edr-freight-web/backoffice/src/components/filters/):
- FilterDef schema + a pure url.ts codec (parse/serialize/toApiParams),
with a 24-case round-trip + malformed-input test suite
- useFilters hook driving react-query params straight from useSearchParams,
debounced search, saved views in localStorage (@mantine/hooks
useLocalStorage), page-reset-on-filter-change baked into one
setSearchParams call instead of a separate effect
- FilterBar/FilterPill/OperatorSelect/MoreFiltersMenu/SortControl +
per-type popover bodies (Mantine)
- ContractRequestsPage migrated end to end as the pilot
Backend (apps/edr-freight-api):
- pagination.util: applySort() — whitelisted sortBy resolved against a
per-module column map (never interpolated), with a mandatory `id ASC`
tiebreaker so paginating a non-unique sort can't drop/duplicate rows
- facets.util: computeFacets() — one GROUP BY per enum column, each
omitting its own predicate, so picking a value doesn't hide its siblings
- contracts/bookings: list-summary now returns real filter-scoped facet
counts (contracts' getStatusCounts was unfiltered/global; superseded)
- deleted drivers/vehicles findAllWithFilters — dead code that
interpolated an unwhitelisted sortBy straight into orderBy()
- migration: missing bookings(status)/wagons(status) indexes +
(created_at DESC, id ASC) partials on the hot list tables
UI polish pass: inactive pill uses the opaque "default" variant instead
of a faint tinted outline, active pill uses "light" not "filled", larger
X hit target, applied filters sort first, sort control separated behind
a divider on the right and wraps independently from the filter row,
popover option rows are fully clickable (count moved inside the native
label) with bigger hit area and font, fixed a real date-filter bug where
the calendar's own portal falsely registered as an "outside click" and
closed the popover, and fixed a timezone bug where bare YYYY-MM-DD
strings were parsed as UTC instead of local time (shifts a day for EAT).
Not in this commit: rollout to the other ~59 list pages, the Ethiopian-
calendar DateBody branch, and the Family-B (client-side) bridge mode —
tracked in the filter-bar plan.
Empty containers loaded onto an export departure carry no booking and no
wagon allocation, so their wagons printed as EMPTY — no cargo allocated —
staff checking the paper against the consist found boxes the list denied.
Those wagons now print the container numbers with cargo type EMPTY
CONTAINER, count toward the 40ft/20ft tallies, and get their own summary
tile; only genuinely bare wagons keep the empty wording.
Empties had no way onto a departure: the return record could name a train
but nothing seated it on a wagon. Export schedules now expose a loading
action that packs selected returns onto free wagons at one 40ft or two 20ft
each, enforced both in the picker and in the API (existing empties on the
schedule count against their wagon).
Adds container_size, train_schedule_id and wagon_sequence_no to
freight.empty_container_returns.
PNR isn't a column on Invoice/Payment — it's written onto Booking.pnrCode
when a CBE_BILL payment initiates. toDocumentModel() now looks it up by
invoice.sourceId and adds it to the existing summary grid, shown only
when present.
- Added ShippingLineBookingCompletionController and associated service to handle the completion of shipping line bookings.
- Introduced a new module for booking completion to maintain module separation and avoid cyclic dependencies.
- Updated the train scheduling global rules to set default desk hours to 24 hours.
- Modified existing services and entities to accommodate the new booking completion logic.
- Enhanced the front-end components to support the new booking completion flow, including updates to the booking detail and bookings pages.
- Implemented validation and error handling for booking completion, ensuring that only approved bookings can be completed.
- Added migration to set default desk hours in the database.
- Add ShippingLineBookingsPage for listing and managing shipping line bookings.
- Create ShippingLineDocumentsModal for document uploads related to bookings.
- Introduce ShippingLineInitiateModal for initiating new shipping line bookings.
- Implement booking document state management with booking-doc-state utility.
- Add shipping line bookings service for API interactions.
- Update index to export new components and services.
- Enhance types for freight to include shipping line credits.
