Every SQL tonnage in the export datasets and report definitions used
`COALESCE(b.bulk_total_weight_tons, b.cargo_total_weight_vgm)`. COALESCE
falls through on NULL, never on 0 — and the portal booking wizard stores
`cargo_total_weight_vgm = 0` for container freight on purpose, because VGM
is captured per container line, not as a booking-level figure. So every
portal-created container booking reported as weighing nothing. The
backoffice wizard does store a booking-level total, so the same table holds
both shapes and the numbers looked erratic rather than uniformly zero.
Extract the resolver the TypeScript side already has three copies of
(bookingCargoTons, cargoTonsAndItems, totalVgmTons) into one SQL helper:
NULLIF both booking-level columns, then fall back to
SUM(booking_container.total_vgm_tons). Applied to the bookings and
train-schedules export datasets, the cargo-summary, contract-utilization and
booking-status-breakdown reports, and the intercity booking list.
On dev data this recovers 116 of 154 zero-weight container bookings and
raises live booking tonnage from 42,973 t to 61,424 t.
Every revenue report excluded invoices whose booking carries
contract_kind = 'GENERAL', on the premise that such a booking is an
umbrella contract row paid once and drawn down by many orders, so
counting it alongside those orders would double-count.
That premise does not hold. On the dev database all 169 GENERAL
bookings are real shipments with origin and destination yards, warehouse
receipts and their own invoices; there is no umbrella invoice to
double-count, and no invoice source of that kind exists at all. The
predicate simply deleted 119 invoices and ETB 164.6M of billed revenue
from every Finance report, which is why revenue-by-customer reported
ETB 39.4M collected while /billing/invoices/summary reported ETB 140.1M
- a gap of exactly ETB 100,741,976, the paid total of the 80 PAID
invoices the predicate hid.
Removing it from the shared revenue and invoice ledgers brings the
reports back in line with billing (ETB 210.6M billed, ETB 140.1M paid,
163 invoices), and removing it from the overview repository restores the
same bookings to the operational KPIs.
Also fixes what that exposed in the reports that build their own query:
- GATEWAY_PAID read the booking's whole gateway total onto every invoice
sharing that booking. With 25 booking ids backing 58 invoices, the
reconciliation report claimed ETB 113.2M of receipts against ETB 43.6M
of settlement and showed ~ETB 89.4M of variance that does not exist.
Receipts are now apportioned across an invoice's siblings by settled
share, so the gateway column sums to the payments table and total
variance is the real ETB 13.1M of manual settlements.
- Aging Receivables joined companies with an INNER JOIN, dropping
shipping-line-billed arrears, and summed both currencies under a
hardcoded ETB label. Both payer joins are now LEFT and the report
takes a currency filter.
- Invoicing Pipeline summed ETB and USD invoices into one ETB total and
applied no trade-direction scope, unlike every other Finance report.
Both are now applied.
Verified against the shared dev database: every new statement passes
EXPLAIN, and the report totals reconcile with billing and with
freight.payments. Type-check and the reports and overview suites pass;
the five failures in billing.service.spec.ts are pre-existing on this
branch and untouched by this change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Four changes the business asked for on Charged and Actual Volumes:
- Split the Leg column into From and To, both sortable.
- Add 20ft and 40ft container counts beside TEU, off the marshalling
record's container items.
- Classify cargo into the revenue vocabulary rather than the operational
one, so a corridor's tonnage and its revenue read in the same buckets.
Charge-only buckets (incidental, first/last mile, customs) cannot be
emitted — no physical wagon is one.
- Carry the empty wagons as rows of their own, the way the marshalling
document lists them. `allocationLedgerQb` gains `includeEmptyWagons`,
which starts the ledger from the wagon instead of the allocation; the
empty and total wagon counts become plain group aggregates, so a
departure's wagons now add up down its rows instead of every row
repeating the train's total. Vehicle-Km lands on the empty rows and
sums across legs.
`LOADED_WAGONS_EXPR` gains a FILTER on the allocation being present — a
no-op for every allocation-grain report, and the fix at the one place
all of them route through. `SCHEDULE_EMPTY_WAGONS` had no callers left
and is deleted.
Verified: type-check clean, 50 report specs pass (incl. a new one
asserting the cargo expression only emits keys the revenue vocabulary
offers), and the report SQL EXPLAINs and runs against edr_dev — 60 empty
wagons over 4 rows, 53,539 Vehicle-Km, 20ft/40ft counts populating.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Grouped by Train, a row was a period average over the train's stops, which
threw away the one thing that grain is for: when the work actually happened.
The row is now the stop itself — logged arrival, departure, unloading and
loading times, and that stop's own durations. Grouped by Station it still
averages over every train that called there.
The two shapes need different columns, so `ReportColumn.hideWhen` names the
filter values that hide a column, and the runner resolves sort against the
visible set — sorting by an average and then switching to Train falls back to
the default sort instead of a 42703 on a column the query no longer selects.
`ReportFilterDef.defaultValue` pins the unset grain to `train`, so "no value"
is never a third shape. The table, the export field list and the chart toggle
all follow the visible set; the chart is station-grain only, since per-stop
rows have nothing to bar-chart.
Both grains EXPLAIN-validated and run against the dev database.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The OCC report publishes total loading and unloading time and the other
activity left over from a station stay, but nothing recorded when handling
started or ended — the July 2026 seed had to write the figure into a
checkpoint note. Four nullable stamps now ride the stop's arrival row, which
is the row the staying-time report builds a stay from (a turnaround's
departure belongs to a different schedule).
Handling is unloading start to loading end, so a container stop reads as one
window and a bulk station that only loads or only unloads still reports its
half; other activity is the rest of the stay. Both stay NULL where nothing
was logged rather than collapsing to zero.
- station-staying-time: + loading/unloading and other activity per stop
- turnaround-cycle: + the same, summed over the cycle's stops
- loading-unloading (new): per train per station per period, so a week or
month view is that train's average over its stops
- the stop/stay query moves to operations-classification, shared by both
Reproduces the monthly count sheet a port warehouse publishes: trains,
containers by size and laden state, wagons, TEU and bulk wagons per cargo
type, each split into export and import beside an overall total.
Two departures from the spreadsheet it replaces:
- Wagons are counted distinctly from the marshalling record rather than
derived as 20ft/2 + 40ft + bulk wagons, which overstates whenever a wagon
ran part-loaded.
- Total is counted over everything rather than summed across the direction
columns — a train carrying both an import and an export booking belongs to
both and would otherwise count twice.
Demurrage is billed on invoice lines and is left to Revenue by Category.
Adds a station filter matching either end of the corridor, so one warehouse
can report the trains it worked in both directions.
These reports resolved ctx.directions and never read it, so a user restricted
to one trade direction saw every row, and a user with no trade access — where
directions is [] and the rule is show nothing — saw all of them.
first-last-mile-bookings scopes on the booking's own direction; the other five
scope on ts.direction. global-logistics-wagons uses the fragment form so a log
row whose schedule is gone stays visible, which is the rule the other ledgers
apply to rows carrying no direction.
Also adds the missing soft-delete guards: b.deleted_at on the first/last mile
booking join, ts.deleted_at on the wagon-teu-utilization and
global-logistics-wagons schedule joins.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The cargo filter was applied only to the operated and attainment subqueries.
The plan side selected targets by metric and dimension alone, and the FULL
OUTER JOIN put every filtered-out key back as a row of zeros —
?categories=FERTILIZER returned all ten planned categories.
planKeyFilter restricts targets to the selected categories (or container
classes), reading ot.cargo_category for a station plan and ot.dimension_key
otherwise. Values are whitelisted against the vocabulary and inlined, because
the fragment is assembled into raw CTE text and the runner does not validate
multiselect values.
Two related grain leaks close with it:
- planCountryFilter narrows a station plan to the chosen country. All seven
station targets are Ethiopian, so the Djibouti view was listing 33 Ethiopian
targets as stations that moved nothing.
- planGrainFilter drops the plan entirely when origin, destination, train
number or direction is set. No target carries a route, so the plan there was
the whole corridor's target sitting beside one slice of its work, and the
implement rate read as a miss that never happened.
Fixes cargo-volume-performance, cargo-volume-by-station, trainset-performance
and teu-performance together.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Whole-file requote to double quotes plus wrapped column literals. No
behaviour change.
Kept as its own commit because it is out of step with the rest of
reports/definitions, which is single-quoted: bare prettier ignores
@edr/prettier-config, so running it on one file requotes that file alone.
Drop this commit if the directory should stay consistent.
Loaded and empty wagons were both shown but never summed, so the train's
actual consist had to be added up by hand on every row. MAX() on the empty
count for the same reason the distance uses it: the value is constant within
a group that includes ts.id and the leg.
The report priced every departure against its planned origin-to-destination
corridor, so a train that worked several station-to-station moves showed one
row and one distance. Ton/Km and Vehicle-Km were then computed against that
single corridor and understated the work actually done.
The row grain is now the leg: consecutive checkpoint events at different
yards, read with lead() over each schedule. DISTINCT because a train that
works the same pair twice in one departure is still one leg — without it the
join fans the cargo out and doubles every SUM in the group. Both ends fall
back to the schedule's own corridor, so a departure with no checkpoints
logged keeps exactly the single row it had before.
Only query() joins the legs. The KPIs stay corridor-level and would count the
same cargo once per leg if they had that join.
The bare UUID column sat next to the booking reference it duplicates, and a
reference is what anyone reading or exporting this report actually quotes. The
report is the audit trail for an export, so a column nobody can act on is
weight in every downloaded file.
Eleven report columns bucketed their timestamp to a bare day with to_char,
which is wrong for anything a user reads as an event rather than a period:
two departures on the same date, or a wagon request fulfilled hours after it
was raised, were indistinguishable in the output.
The renderer only shows the time when the value actually has one, keyed off
the string rather than a per-column flag — a genuine day bucket would
otherwise render as 12:00 AM, which reads as data rather than as absence.
Guards the two facts the report exists to get right: the cancellation
fee is never a payable, and exactly one wagon-cancellation status
(CREDIT_AVAILABLE) is a live liability. Adding a status to
WAGON_CANCELLATION_STATUSES now fails here until someone decides which
side of the ledger it lands on.
Also pins the sort expressions to the union wrapper alias — a branch
alias would resolve at build time and 42P01 at runtime, since the runner
appends ORDER BY outside the subquery.
The receivable/payable split contradicted how money actually moves, in
three ways that each changed a headline number:
- The wagon-cancellation FEE was booked as a payable. It is money the
customer owes EDR (raised ISSUED and unpaid at request time), so it
belongs on the receivable side while open. The sign was inverted.
- A whole-booking wagon cancellation was booked at the source invoice's
full paid_amount, and never cleared: the booking stays CANCELLED and
the invoice stays PAID even after the credit is rebooked. The real
liability is the ledger row's credit_amount, and only while it sits in
CREDIT_AVAILABLE — cancellation refunds no cash, it hands back
bookable credit redeemed by creating another booking.
- Shipping-line debt in UNBILLED has no invoice row at all, so an
invoice-only fact table could not see it. That is the un-batched half
of the debt, in the report whose stated purpose is shipping-line
credit.
The report is now a UNION of the three tables that hold the answer:
invoices with a balance (plus prepayments against dead bookings),
UNBILLED shipping_line_credits, and CREDIT_AVAILABLE
booking_wagon_cancellations. A booking already carried by the
cancellation ledger is excluded from the invoice branch so its money is
counted once. Fully settled invoices are dropped — zero exposure is
neither a receivable nor a payable.
Branches are re-projected through an explicit column list before being
unioned: UNION matches by position and TypeORM does not preserve
addSelect order, which silently reordered one branch into
"gross, exposure, side_key, ..." and failed with "UNION types text and
numeric cannot be matched".
Verified against Postgres with a rollback-only fixture covering every
side, plus EXPLAIN over each filter combination and every sortable
column.
A target is a quota, not a flat allowance. The Plan column spread it evenly
and kept asking for the same twelfth of a yearly figure no matter how far
behind the year had fallen, so the one number operations actually needs —
what must move per month for the rest of the year — was nowhere on the
page.
Plan now keeps its meaning and a Required column sits beside it. Plan is
the committed spread and never moves, which is the whole reason it stays:
Implement Rate is measured against it, so a month that missed still reads
as a month that missed. Required is the same target read as a quota — at
each bucket, whatever is still outstanding spread across the time still
left. A 1,200 t year 20% met by June asks 140 t of June and 960 t of
December, which is 1,200 less the 240 delivered. Over-delivery clamps to
zero rather than going negative.
Attainment is deliberately measured with the user's date bounds stripped
(attainmentCtx) and every other filter left in place. Reusing the report's
own filtered aggregate would make a July-only view read year-to-date as
nothing delivered and demand the entire year's tonnage from one month —
the failure would look like a plausible number, not an error.
Granularity gains half-year, nine-month and 90-day. Postgres has no
date_trunc for any of them, so PERIOD_UNITS entries became builders rather
than fragments to interpolate, and all eight blocks anchor to the calendar
year. Nine does not divide twelve and 90 does not divide 365: a nine-month
year is Jan-Sep plus a short Oct-Dec, and the fourth 90-day block absorbs
the remainder at 95 days. That last one is a choice — uncapped floor
division opens a five-day stub bucket every December, which is noise
rather than a period.
Two consequences of the shared unit table, both handled here:
- plannedRowsSql now generates a day at a time and groups, instead of
stepping by the bucket width. The ragged blocks restart each January, so
stepping 90 days from January 1st walks off the anchor in the second
year. Day grain also gets partial-bucket overlap for free, at the same
sub-day precision the old clipping had.
- nextPeriodOrdinalExpr asks the unit for its next block start rather than
adding its own step. revenue-by-period evaluates a regression there, and
a ragged unit's final block is shorter than its nominal width, so + step
would land past the next block and forecast at the wrong x.
Verified against Postgres 16 with the entities synchronised into it: all
272 report/granularity combinations in the registry EXPLAIN clean, and the
1,200 t drill-down sums back to 1,200 at every one of the eight grains.
A category with no invoice lines in a period simply had no row, so a
category going quiet was indistinguishable from one that never existed,
and filtering to a category that was never billed returned an empty table.
The query is now three levels. The aggregate groups as before. A grid
crosses every period that saw revenue with every category the filter
allows, and LEFT JOINs the aggregate onto it so a missing combination
lands at zero. The wrapper does the display rounding and the labelling.
Two things had to move for that to be correct:
- The lag() window is now in the wrapper. A window function only sees the
rows its own query level produces, so left on the aggregate it would
skip a category's silent periods — billed in January and March, it
would read March's prior as January and report flat growth.
- The category filter is off the aggregate and enforced by the grid's
category list. Filtering the aggregate too would make the period axis
depend on the selection, which is what left the table empty when the
selected category had never been billed.
Periods come from the data, not generate_series over the date filter: a
twelve-month range over one billed month would otherwise publish eleven
months of pure zeros, and daily granularity would multiply that by thirty.
The Categories KPI is now "Categories with revenue" — a bare count of live
categories reads as a contradiction next to a table listing all fourteen.
EXPLAIN-validated against the dev database across seven filter shapes,
including the empty-array case (hence unnest(ARRAY[...]) over VALUES,
which is a syntax error when empty).
Claude-Session: https://claude.ai/code/session_01LoY3hNWqcaAC1pYmGPN7jr
CATEGORY_LABEL_EXPR wraps the classifying CASE, so it only works where the
classification happens in the same SELECT. A report that classifies in a
subquery and labels in the wrapper has a plain key column to label instead.
CATEGORY_LABEL_OF takes that key expression; CATEGORY_LABEL_EXPR is now
defined through it, so its three existing callers are unchanged. Mirrors
CATEGORY_LABEL_OF in operations-classification.ts.
Claude-Session: https://claude.ai/code/session_01LoY3hNWqcaAC1pYmGPN7jr
plannedValueExpr matched a target only when its period_type and
period_start equalled the report's bucket exactly, so a monthly plan
vanished the moment you viewed by quarter, by year, or by day. The plan
column simply went empty and the implement rate read 0%.
plannedRowsSql replaces it with a derived table: each target is spread
evenly over the days it covers, then re-gathered into whichever bucket
the report shows. Three monthly targets add up to a quarter exactly, a
daily view gets a thirty-first of the month, and a week straddling a
month boundary draws proportionally on both. The even spread is an
assumption and the only one available — a monthly figure says nothing
about which days inside it were busier — so PLAN_GRANULARITY_NOTE says so
in each report's description.
The share is clipped to the user's date filter as well as to the bucket,
or filtering to July and viewing by year would sit a whole year's plan
next to one month's work. Reports FULL OUTER JOIN it so a category that
was planned but never ran still publishes, at 0% — dropping the row would
hide a total miss, which is the one thing a plan-versus-actual table is
for.
The export path used one number for two different things: the format's hard
row cap, and the caller's explicit 'give me the first N rows'. Because
resolveExportCap() returned min(requested, formatCap) and runAll() then threw
when the result reached it, picking 'Records: First 100' in the export dialog
400'd on any report with more than 100 rows — the user asked to be truncated
and got an error instead.
Splits them: formatRowCap() is the hard, non-caller-controllable ceiling that
still throws when exceeded (a silently short file hides missing rows), while
resolveRowLimit() is the deliberate truncation and is honoured by slicing.
Verified against a 223-row dataset: limit=5 now returns 5 rows, and no limit
returns all 223.
Completes the writer extraction whose other half landed in fb21ad154.
reports.controller now builds a TabularDoc and calls TabularExportService,
so report-export.service.ts and report-export-request.util.ts are dead and
removed — HEAD was carrying both copies with the controller still on the old
one.
Reports gain CSV for free, and the PDF path now passes buildTabularFallbackPdf
as its fallback: previously it passed none, so a box without Chromium silently
returned PdfRenderService's ~900-character generic text dump instead of a
table. Adds a spec covering the CSV writer's quoting of embedded commas and
double quotes — the reason this uses ExcelJS's csv writer rather than a
hand-rolled join.
Extracted the export route's format/cap/column-whitelist branching out
of the controller into pure functions (resolveExportFormat,
resolveExportCap, resolveExportColumns) and added a spec: unknown
format falls back to xlsx, limit clamps to the format cap and ignores
non-positive/NaN input, unknown field keys are dropped and an
all-unknown fields list falls back to every column instead of
shipping a blank sheet. Was untested branching logic before this.
- ReportPage drops its own PageHeader (and the back arrow); ReportView
now optionally renders the header itself (pageHeader prop) with
export/refresh as its actions. Embedded ReportSection usage is
unaffected (keeps the inline toolbar next to filters).
- Replace the two xlsx/pdf icon buttons with one Export button opening
a dialog: format as large icon radio cards, fields as checkboxes
(select-all toggle), record count (default all, capped per format).
Export applies the report's current filters and sort.
- Backend: export route accepts fields (whitelisted against the
report's own columns) and limit; ReportExportService takes an
optional column subset instead of always dumping every column.
- Fixed a real bug found while wiring this up: runAll() ignored the
caller's sortBy/sortOrder and always used the report's default sort,
so exports silently didn't match whatever order was on screen.
- Report daterange filters now use DatePickerInput + the shared
getDateRangePresets() (Today/Last 7 days/This month/...) instead of
two bare DateInputs, matching every other date-range filter in the
app.
- Removed the reports hub grid page. /dashboard/reports now redirects
to the first report the caller has access to, or /dashboard if they
have none.
ReportDefinition gets an optional chart {type: line|bar, x, y[]} field —
plots the same rows the table gets, no separate query. Frontend adds a
table/chart toggle (defaults to table) using the existing recharts
dependency, no new package.
Chart view fetches up to 100 rows (the API's page-size ceiling) instead
of the table's current page, so it doesn't silently plot a fraction of
the filtered set; shows a truncation note past that cap.
Wired onto 5 reports as proof: wagon-fleet-status, locomotive-fleet-
status, booking-status-breakdown, revenue-summary (bar), and
global-logistics-wagons (line). Everything else stays table-only —
charting is opt-in per report, not a default.
customer-status (company-profile roles, not Company — importer/
exporter/forwarder lives there), contract-lifecycle, customs-documents
(clearance milestones), invoicing-pipeline, first-last-mile-bookings
(one resolver, UNION ALL over first_mile/last_mile — verified the
raw-string .from() subquery against the live query builder, not just
hand-written SQL, after the join-alias bug earlier this branch),
invoices-by-status, payments-by-status, revenue-summary, cargo-summary.
payments carries no deleted_at column despite extending BaseEntity —
caught by column-checking against the live DB before shipping, dropped
the soft-delete filter for that one query.
Completes the ITLMS dashboard spec's 20-resolver dedup list (19 built,
freight-weight-variance dropped — no charged-vs-actual weight
distinction in the schema).
booking-status-breakdown (dedupes the same 'status per port/train/
cargo/contract' ask across 4 dashboards), train-schedule-status,
train-turnaround, wagon-teu-utilization, loaded-capacity,
global-logistics-wagons.
Dropped freight-weight-variance from this batch: the schema has no
'charged weight' distinct from VGM/actual, so a charged-vs-actual
variance report isn't buildable without a product decision on what
'charged' means here.
wagon-fleet-status, wagon-status-duration, wagon-requests,
locomotive-fleet-status. First batch off the ITLMS dashboard spec —
fleet data (wagons/locomotives/transfer-requests) needed no schema
work, just resolvers. No frontend changes: catalog is server-driven.
Sorting by a column with no explicit sortExpr fell back to the bare
select alias unquoted. Postgres folds unquoted identifiers to lowercase,
so any camelCase alias (utilizationPct, bookedTons) 42703'd. Quote the
fallback to match the case TypeORM's addSelect actually emitted.
Nuke the 17 hand-written raw-SQL reports (no pagination, hard LIMITs) and
the reports module built around them. Replace with a resolver contract:
a report declares columns/filters/permission and a TypeORM QueryBuilder;
ReportRunnerService applies filtering, a whitelisted sort, offset/limit
paging, and a COUNT(*) FROM (query) wrapper for the total (getCount() is
wrong for GROUP BY). ReportExportService re-runs the same resolver
unpaginated for xlsx (exceljs) and pdf (existing PdfRenderService, now
landscape-capable) exports.
Ships with 4 reports: bookings-list, revenue-by-customer,
aging-receivables, contract-utilization. Catalog + per-report permission
checks live in the controller; adding a report is one new definitions/
file plus a REPORT_KEYS entry, no frontend change.
Gates the previously open support-agent, procurement, compliance,
facilities, list-users and trade-access controllers, separates customer
from staff routes across bookings, contracts, companies, billing,
warehouses, files and train scheduling, and moves billing, overview,
reports and the settings controllers onto their own keys instead of the
blanket admin key. Drops the demo-permissions module and the untested
notification test route.
