customers, contracts, invoices, payments, train schedules, and locomotives /
trains / wagons via the fleet page's config.
FilterBar pages pass controls.params into the children slot. The four pages
still on ad-hoc filtering pass their own hand-built filter object instead,
which is why ExportButton takes plain params rather than a UseFilters — it
would otherwise have been blocked behind migrating those pages. FleetResource
serves seven slugs from config, so it gets an optional exportKey there and
renders nothing for the four slugs with no dataset yet.
Auditing each page's real filter keys against the dataset declarations turned
up three gaps where an on-screen filter would have silently not applied to the
export: invoices sends a singular "status" (the dataset only had the
multiselect "statuses"), contracts sends paymentCurrency, serviceTypeId and
route origin/destination, and train schedules sends freightType. Added all of
them — contract routes filter through EXISTS on contract_routes since they are
one-to-many, and train-schedule freightType through EXISTS on the bookings
aboard, matching the list service.
Verified in the browser: the button renders on each page, and the invoices
dialog follows that page's own filter object — selecting Paid moves the count
from 126 to 100, which matches the database. Filter pass-through checked
against the database for invoices, payments, wagons, contracts and train
schedules.
customers, contracts, invoices, payments, train-schedules, locomotives,
trains and wagons. 319 fields across the nine datasets, all reusing the
existing engine — no change to export.types.ts was needed, which is the
result the bookings-first phase was meant to test.
Per-dataset notes worth keeping:
- trains resolves route, stations and current yard, which the list endpoint
never loads — the UI shows raw FK uuids there today.
- wagons reads tare/payload/length off wagon_types (they are not on the
wagon), and reproduces the service's attachStatusDates() as correlated
subqueries. wagon_status_logs stores from_status/to_status, not status.
- payments applies no soft-delete guard: freight.payments has neither
deleted_at nor updated_at, so the usual predicate is a 42703. Failure
columns are failer_code/failer_message. payment_refunds stores MINOR
units, so refundedTotal divides by 100.
- train-schedules derives freightType from the bookings aboard rather than
a column, matching the list service.
- customers stays one row per company; profiles, bookings and invoice
totals aggregate in subqueries. Verified no row multiplication: trains,
customers and contracts each return exactly their counted row count while
selecting one-to-many aggregate fields.
EXPLAIN-validated against the database: every dataset's widest query, its
count query, and all 319 fields individually. That run caught five columns
typed varchar rather than timestamp (companies.date_registered,
renewal_date, renewed_from, renewed_to and invoices.eims_ack_date), which
were being pushed through to_char and would have 500'd the moment anyone
ticked them; they now export verbatim.
All nine count endpoints verified equal to SELECT count(*) on their table.
A Stripe-style export dialog over the /exports catalog: searchable field
picker grouped by related entity, format choice, row scope, saved presets,
and a live row count.
The picker is what makes 77 fields usable. Groups auto-expand only when they
already hold a selection, so the dialog opens showing the default columns and
their groups rather than a wall of checkboxes; searching force-expands so a
match can't hide inside a collapsed group. Group headers carry a tri-state
checkbox and an n/total badge.
The row count comes from /exports/:key/count with the page's own filters, so
the button reads 'Export 223 rows' before anything is downloaded, and turns
into a cap warning with a one-click 'export the first N' escape when the
result is too large for the chosen format.
ExportButton takes plain params rather than a UseFilters instance —
four of the pages that need this haven't migrated to FilterBar yet, and
coupling to the hook would have blocked them. Pagination keys are stripped in
one place instead of at every call site. It renders nothing when the catalog
omits the dataset, so the catalog's permission filtering IS the UI gate.
Presets reuse useSavedViews unchanged by encoding the preset as a query
string; a preset naming a field the catalog no longer offers is dropped on
load rather than 400ing the download. Download errors go through
extractDownloadErrorMessage, without which the server's row-cap message
degrades to 'Request failed with status code 400'.
Adds a parallel export system the reports module can also draw on. A dataset
describes a table's exportable fields — including related-entity detail the
list page never shows — and the engine assembles a query from whichever fields
the caller picked.
GET /exports catalog (metadata only; select/requires never ship)
GET /exports/:key/count exact row count + per-format caps
GET /exports/:key/download csv | xlsx | pdf
Two invariants carry the design:
- Every lazy join is a LEFT join, and ExportJoin has no 'kind' field to make
anything else expressible. An inner join added because a checkbox was ticked
would change the rowset, so two exports of the same filters would disagree on
their row count.
- Because of that, the count cannot depend on field selection, so /count runs
base + alwaysJoin only and is exact rather than an estimate. Verified: count
and the delivered file both report 223 rows.
One-to-many relations (a booking's containers) aggregate in a correlated
subquery rather than joining, so a row can never multiply.
Export rides each dataset's existing view permission — no new permission keys
and no seeder change. Sensitive columns are simply never declared as fields:
raw gateway payloads, signature blobs, error dumps, raw jsonb snapshots,
internal user UUIDs and review notes are all absent by construction.
bookings ships 77 fields across 10 groups. scripts/validate-export-datasets.ts
EXPLAINs every dataset's widest query, its count query, and each field on its
own against the real database — the per-field pass is what catches a field
referencing a join it forgot to declare, which otherwise only fails when that
one field is picked alone.
The export path used one number for two different things: the format's hard
row cap, and the caller's explicit 'give me the first N rows'. Because
resolveExportCap() returned min(requested, formatCap) and runAll() then threw
when the result reached it, picking 'Records: First 100' in the export dialog
400'd on any report with more than 100 rows — the user asked to be truncated
and got an error instead.
Splits them: formatRowCap() is the hard, non-caller-controllable ceiling that
still throws when exceeded (a silently short file hides missing rows), while
resolveRowLimit() is the deliberate truncation and is honoured by slicing.
Verified against a 223-row dataset: limit=5 now returns 5 rows, and no limit
returns all 223.
Completes the writer extraction whose other half landed in fb21ad154.
reports.controller now builds a TabularDoc and calls TabularExportService,
so report-export.service.ts and report-export-request.util.ts are dead and
removed — HEAD was carrying both copies with the controller still on the old
one.
Reports gain CSV for free, and the PDF path now passes buildTabularFallbackPdf
as its fallback: previously it passed none, so a box without Chromium silently
returned PdfRenderService's ~900-character generic text dump instead of a
table. Adds a spec covering the CSV writer's quoting of embedded commas and
double quotes — the reason this uses ExcelJS's csv writer rather than a
hand-rolled join.
New endpoints:
POST invoices/eims/bulk-register { invoiceIds: [...] } — trigger
POST eims/webhook/bulk-register — MoR's callback
Fundamentally different shape from single register: bulkRegister
answers only {conversationId, status:202} immediately: MoR processes
the array asynchronously and pushes the real per-invoice results
(a mix of accepted/rejected in one array, per the collection's own
examples) to a webhook configured out of band. So this ships as two
halves that don't share a call stack — EimsBulkRegistrationService.
registerBulk() reserves a contiguous block of counters (durable
reservation, same doctrine as single register, extended to N items)
and submits; handleBulkCallback(), invoked by the new
EimsWebhookController whenever MoR gets around to it, settles.
New EimsSystemState.inFlightConversationId is the bulk equivalent of
inFlightInvoiceId — a whole batch outstanding, not one invoice — and
the two markers block each other since they share the same counter
sequence. The conversation id isn't known until MoR's 202 arrives, so
reservation stamps a locally-generated placeholder first (same
commit-before-the-network-call reasoning as single register), then
swaps it for MoR's real id right after — the only value the callback
can actually use to find the batch again.
Only the first invoice in a bulk batch chains via PreviousIrn — every
other item gets an empty string, matching the collection's own
two-invoice example exactly (MoR doesn't expect a batch to chain to
IRNs that don't exist yet at submission time).
Webhook has no auth (MoR has no JWT to send) — the conversation id
embedded in the payload is what stands between this and a forged
callback: an item only ever touches an invoice actually holding that
exact id, and an unknown id is logged and ignored, never applied.
Migration 3580000000000: eims_system_state.in_flight_conversation_id,
invoices.eims_bulk_conversation_id (tags which batch an invoice was
submitted in, so a stuck batch — webhook never arrived — can be found
and reconciled by conversation id). Applied to dev DB and recorded in
freight.migrations directly (idempotent IF NOT EXISTS DDL).
Not live-testable from this sandbox (no route to MoR's real gateway).
Signing the whole array as one envelope, the way single /v1/register
was confirmed live to need despite the collection's raw example
showing no envelope, is the reasonable extension of that confirmed
behavior, not a blind guess — but it has not itself been exercised
against the real gateway. Left for the first live bulk attempt to
confirm, same as every other MoR-facing assumption this integration
has made.
Title and subtitle grew unbounded, relying only on flex-squeeze from action
buttons to ever truncate. Caps the container at 640px and gives the
subtitle the same truncate treatment the title already had, so a long
page title or description (report descriptions in particular) always
ellipses instead of stretching the header.
Swaps ReportView's bespoke ReportFilters for the same FilterBar/useFilters
combo BookingRequestsPage uses — pills, saved-view-ready state, URL sync.
toFilterDefs() maps the report catalog's own filter vocabulary
(daterange/date/select/multiselect/text) onto FilterDef, matched against
every report definition's backend param handling. The search box only
shows for reports that actually implement `search` server-side, so it's
not a dead control on the rest. ReportFilters.tsx is now dead, removed.
DateBody always initialized its op state to DEFAULT_OP.date ("between"),
ignoring a def's `operators` restriction. A single-operator exact-date
filter (e.g. `operators: ["before"]`) opened on the range UI with no way
off it, since OperatorSelect hides itself when there's only one choice.
Default to the def's first allowed operator instead.
- Add migration for consolidation approvals table and status enum
- Create ConsolidationApprovalService to handle approval logic
- Implement repository for managing consolidation approvals
- Add entity for consolidation approval with necessary fields
- Develop frontend components for displaying and managing consolidation approvals
- Create tests for consolidation approval service to ensure correct behavior
- Add migration for consolidation approvals table and status enum
- Create ConsolidationApprovalService to handle approval logic
- Implement repository for managing consolidation approvals
- Add entity for consolidation approval with necessary fields
- Develop frontend components for displaying and managing consolidation approvals
- Create tests for consolidation approval service to ensure correct behavior
- Added support for viewing and managing consolidated bookings in BookingRequestDetailPage.
- Enhanced BookingRequestsPage to display paired bookings in a single row.
- Introduced pairedDecision method in bookings service to handle decisions for both halves of a consolidated pair.
- Updated contracts service to include methods for manual consolidation of odd-20ft bookings.
- Created new components for selecting and editing consolidation partners.
- Added tests for paired decision logic and manual consolidation scenarios.
- Updated UI to reflect changes in booking handling and provide user feedback for odd container counts.
