A GENERAL + customs contract does not let the customer book directly: they
submit a shipment request, and initiateForShipmentRequest opens a BARE booking
from it — "the request itself carries the quantities; the instance carries
none". Between initiation and completeUnderContract the booking legitimately
holds no cargo, so the export reported 0 containers for a customer who had
declared, say, 2 x 20FT. 23 bookings on dev data are in that state.
Adds two columns and one filter reading booking_requests.requested_lines:
- "Requested cargo" — the declared lines as text ("2 x 20FT"), handling the
bulk shape too (tons / item count), not only containers.
- "Requested containers" — the declared box count, with a matching min/max
filter on the list and the export.
Deliberately a separate column rather than a fallback inside the real container
count: a declared 2 x 20FT is a request, not two boxes on a booking, and
merging them would overstate operational totals. The two compose instead —
Containers = 0 AND Requested containers >= 1 is exactly the set awaiting
completion after clearance.
requested_lines is free-form jsonb, so the container array is guarded by
jsonb_typeof before jsonb_array_elements; one malformed row would otherwise
500 the whole list.
Container filters on the booking-requests list:
- "Container type" — bookings carrying that type.
- "Containers" — a count of BOXES (booking_container is one row per line with
a quantity, so this sums quantity rather than counting rows), as an exact
value or a range. It reads the container-type filter when one is set, so the
one control answers both "10 containers in total" and "10 forty-footers".
Export gains a column per container type ("20FT containers", "40FT
containers"), plus the total "Containers" column and the two filters. Container
types are reference rows, not a constant, so `ExportDataset` gains an optional
`dynamicFields` resolver — DB-driven columns appended to the static list and
cached for the process, mirroring the existing `ExportFilterDef.optionsQuery`.
Adding a 45ft container type adds its column with no code change. The type id
is interpolated into raw SQL (ExportField.select has no parameter bag), so the
resolver drops any id that is not a uuid.
Also repoints the export's "Container VGM" column at the per-line sum. It was
projecting bookings.cargo_total_weight_vgm, which the portal wizard leaves at 0
for container freight — the same trap the tonnage fix addressed — so the column
read 0 for every portal-created container booking. Non-zero on dev data goes
from 54 to 170 of 208 container bookings.
The booking-requests list could filter by freight type but not by what is
actually in the booking, and the export's only cargo column showed the
commodity name — blank for every container booking, which stores no
commodity at all.
Adds one resolver, `bookingContentSql`, that answers "what did the customer
say is in this booking" per freight type: the container lines they entered
("2 × 40FT, 1 × 20FT") for container freight, since the wizard asks them for
no description; the commodity they picked from the cargo tree for bulk,
falling back to their free-text description.
List filters:
- "Content" — a single select flattening the cargo tree the same way the
booking wizard presents it (group, then each commodity as "Bulk → Wheat").
Picking a GROUP matches its whole subtree via a recursive walk, so "Bulk"
returns all 44 bulk bookings rather than the 0 that carry the group id
itself. This makes the existing, previously unexposed `cargoTypeId` param
group-aware.
- "Content contains" — a contains-search over the description, the commodity
name and the container types, so container bookings are reachable by "40FT"
even though they carry no words of the customer's own.
Both apply through `applyListFilters`, so the list, its summary tiles and its
facets agree, and both are declared on the bookings export dataset — the
export button already forwards the page's filters verbatim.
Export fields: "Content" (default), plus "Cargo description" as its own
column. The old `cargo` column is unchanged and still selectable, relabelled
"Cargo (commodity)"; it loses only its default tick, so saved presets that
name it keep working.
Every SQL tonnage in the export datasets and report definitions used
`COALESCE(b.bulk_total_weight_tons, b.cargo_total_weight_vgm)`. COALESCE
falls through on NULL, never on 0 — and the portal booking wizard stores
`cargo_total_weight_vgm = 0` for container freight on purpose, because VGM
is captured per container line, not as a booking-level figure. So every
portal-created container booking reported as weighing nothing. The
backoffice wizard does store a booking-level total, so the same table holds
both shapes and the numbers looked erratic rather than uniformly zero.
Extract the resolver the TypeScript side already has three copies of
(bookingCargoTons, cargoTonsAndItems, totalVgmTons) into one SQL helper:
NULLIF both booking-level columns, then fall back to
SUM(booking_container.total_vgm_tons). Applied to the bookings and
train-schedules export datasets, the cargo-summary, contract-utilization and
booking-status-breakdown reports, and the intercity booking list.
On dev data this recovers 116 of 154 zero-weight container bookings and
raises live booking tonnage from 42,973 t to 61,424 t.
Adds a free-form `types` CSV filter to the invoice list DTO and query
(same treatment as `paymentMethods` — each billing source mints its own
type string, so an IsIn would drop real values), carries it into the
invoices export dataset, and surfaces a Type column plus filter pill on
both the Invoices and Manual Payments tables.
IAM lets an employee hold several positions, but the vendored JwtGuard
collapses employee.positions[] down to a single employee.position and
drops the rest. Non-delegate secondary positions vanished entirely, so
staff on two posts resolved to one post's permissions and every check
on the other rejected them.
FreightJwtGuard re-attaches the full list from the same session
snapshot the parent guard already read, so nothing extra is fetched
per request beyond a cached session lookup. employee.position is left
untouched, keeping audit logging and delegation unaffected.
collectPermissionKeys and collectPositionTypeKeys now union across
every position, and /me returns them all.
Verified against a real two-position user (djibouti-gl-director +
djibouti-gl-chief) on the local dev database:
/me positions 1 -> 2
/me permissionKeys 17 -> 28
GET /api/interchange-documents 403 -> 200
GET /api/trains 403 -> 200
11 permissions recovered, none lost. Six single-position users return
byte-identical payloads before and after.
Both are numbers a customer or a provider support desk quotes back, so they
belong in the free-text box rather than behind a filter pill.
The transaction id and merchant order id are plain ORs — the payment alias is
already joined by every caller of applyInvoiceFilters. The PNR folds into the
existing booking EXISTS block instead of adding a second subquery, so it
inherits that block's correlation and also matches warehouse-, first-mile-
and last-mile-sourced invoices, not just booking-sourced ones.
bk is promoted to alwaysJoin now that the export's scope() references it.
The PNR is the CBE_BILL reference the customer actually pays against, but it
is stamped onto the booking at payment-initiation time — it is a column on
neither the invoice nor the payment. Both surfaces read it back by source id,
the same lookup the sealed invoice PDF already did, so screen, export and
document now agree.
The export's join casts bk.id::text rather than i.source_id::uuid: source_id
is a bare varchar pointer that is not always a UUID (EIMS self-test rows
carry a slug), and casting that direction throws on those rows.
The settled method is split across two stores: a gateway settlement records
the real provider on the linked freight.payments row (cbe-bill, telebirr)
while the invoice's own payments ledger only writes a flat "GATEWAY"; a
manual settlement has no payments row at all and the ledger is the only
source (BANK_TRANSFER, OFFLINE, or whatever PayInvoiceDto.method carried).
invoicePaymentMethodExpr folds both into one UPPER_SNAKE vocabulary —
provider first, newest ledger entry as the fallback — and the list filter,
the export field and the export filter all use that same expression, so the
screen and the file can never disagree.
The paymentMethods param is deliberately not validated against a fixed list:
the manual pay endpoint takes a free-form method, so an IsIn would silently
drop real values.
customers, contracts, invoices, payments, train schedules, and locomotives /
trains / wagons via the fleet page's config.
FilterBar pages pass controls.params into the children slot. The four pages
still on ad-hoc filtering pass their own hand-built filter object instead,
which is why ExportButton takes plain params rather than a UseFilters — it
would otherwise have been blocked behind migrating those pages. FleetResource
serves seven slugs from config, so it gets an optional exportKey there and
renders nothing for the four slugs with no dataset yet.
Auditing each page's real filter keys against the dataset declarations turned
up three gaps where an on-screen filter would have silently not applied to the
export: invoices sends a singular "status" (the dataset only had the
multiselect "statuses"), contracts sends paymentCurrency, serviceTypeId and
route origin/destination, and train schedules sends freightType. Added all of
them — contract routes filter through EXISTS on contract_routes since they are
one-to-many, and train-schedule freightType through EXISTS on the bookings
aboard, matching the list service.
Verified in the browser: the button renders on each page, and the invoices
dialog follows that page's own filter object — selecting Paid moves the count
from 126 to 100, which matches the database. Filter pass-through checked
against the database for invoices, payments, wagons, contracts and train
schedules.
customers, contracts, invoices, payments, train-schedules, locomotives,
trains and wagons. 319 fields across the nine datasets, all reusing the
existing engine — no change to export.types.ts was needed, which is the
result the bookings-first phase was meant to test.
Per-dataset notes worth keeping:
- trains resolves route, stations and current yard, which the list endpoint
never loads — the UI shows raw FK uuids there today.
- wagons reads tare/payload/length off wagon_types (they are not on the
wagon), and reproduces the service's attachStatusDates() as correlated
subqueries. wagon_status_logs stores from_status/to_status, not status.
- payments applies no soft-delete guard: freight.payments has neither
deleted_at nor updated_at, so the usual predicate is a 42703. Failure
columns are failer_code/failer_message. payment_refunds stores MINOR
units, so refundedTotal divides by 100.
- train-schedules derives freightType from the bookings aboard rather than
a column, matching the list service.
- customers stays one row per company; profiles, bookings and invoice
totals aggregate in subqueries. Verified no row multiplication: trains,
customers and contracts each return exactly their counted row count while
selecting one-to-many aggregate fields.
EXPLAIN-validated against the database: every dataset's widest query, its
count query, and all 319 fields individually. That run caught five columns
typed varchar rather than timestamp (companies.date_registered,
renewal_date, renewed_from, renewed_to and invoices.eims_ack_date), which
were being pushed through to_char and would have 500'd the moment anyone
ticked them; they now export verbatim.
All nine count endpoints verified equal to SELECT count(*) on their table.
Adds a parallel export system the reports module can also draw on. A dataset
describes a table's exportable fields — including related-entity detail the
list page never shows — and the engine assembles a query from whichever fields
the caller picked.
GET /exports catalog (metadata only; select/requires never ship)
GET /exports/:key/count exact row count + per-format caps
GET /exports/:key/download csv | xlsx | pdf
Two invariants carry the design:
- Every lazy join is a LEFT join, and ExportJoin has no 'kind' field to make
anything else expressible. An inner join added because a checkbox was ticked
would change the rowset, so two exports of the same filters would disagree on
their row count.
- Because of that, the count cannot depend on field selection, so /count runs
base + alwaysJoin only and is exact rather than an estimate. Verified: count
and the delivered file both report 223 rows.
One-to-many relations (a booking's containers) aggregate in a correlated
subquery rather than joining, so a row can never multiply.
Export rides each dataset's existing view permission — no new permission keys
and no seeder change. Sensitive columns are simply never declared as fields:
raw gateway payloads, signature blobs, error dumps, raw jsonb snapshots,
internal user UUIDs and review notes are all absent by construction.
bookings ships 77 fields across 10 groups. scripts/validate-export-datasets.ts
EXPLAINs every dataset's widest query, its count query, and each field on its
own against the real database — the per-field pass is what catches a field
referencing a join it forgot to declare, which otherwise only fails when that
one field is picked alone.
The export path used one number for two different things: the format's hard
row cap, and the caller's explicit 'give me the first N rows'. Because
resolveExportCap() returned min(requested, formatCap) and runAll() then threw
when the result reached it, picking 'Records: First 100' in the export dialog
400'd on any report with more than 100 rows — the user asked to be truncated
and got an error instead.
Splits them: formatRowCap() is the hard, non-caller-controllable ceiling that
still throws when exceeded (a silently short file hides missing rows), while
resolveRowLimit() is the deliberate truncation and is honoured by slicing.
Verified against a 223-row dataset: limit=5 now returns 5 rows, and no limit
returns all 223.
Completes the writer extraction whose other half landed in fb21ad154.
reports.controller now builds a TabularDoc and calls TabularExportService,
so report-export.service.ts and report-export-request.util.ts are dead and
removed — HEAD was carrying both copies with the controller still on the old
one.
Reports gain CSV for free, and the PDF path now passes buildTabularFallbackPdf
as its fallback: previously it passed none, so a box without Chromium silently
returned PdfRenderService's ~900-character generic text dump instead of a
table. Adds a spec covering the CSV writer's quoting of embedded commas and
double quotes — the reason this uses ExcelJS's csv writer rather than a
hand-rolled join.
