Files
edr-platform/E2E_TEST_REPORT.md
2026-07-24 13:26:19 +00:00

441 lines
16 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# EDR Freight — End-to-End Test Report
**Date:** 23 July 2026
**Branch:** `freight_feature/usermanagement`
**Command that was run:**
```bash
E2E_PORTAL_PORT=5374 docker compose -f docker-compose.e2e.yaml --profile cypress \
run --rm cypress --spec 'cypress/e2e/flows/*.cy.ts'
```
---
## 1. Short summary (read this first)
I ran the end-to-end (E2E) test suite and found **four separate problems**. They are not all the same kind of problem, and this is the most important thing to understand:
| # | Problem | Kind of problem | Status |
|---|---------|-----------------|--------|
| 1 | Tests crashed instantly with `exit code 137` | Machine / environment | Explained + how to avoid |
| 2 | Portal tests pointed at a dead port (`5374`) | Wrong command setting | Explained + corrected |
| 3 | `contract-lifecycle` failed all 5 of its tests | **Real bug in the test code** | ✅ **Fixed and verified** |
| 4 | 5 other test files failed | Old app running inside Docker | Diagnosed, needs a rebuild |
**The only real code bug was problem 3, and it is now fixed.** Problems 1, 2 and 4 are about *how the tests were run*, not about the application logic.
---
## 2. Some words explained (for beginners)
Before the details, here are the words used in this report:
- **E2E test (end-to-end test)** — a robot that opens a real web browser, clicks buttons like a real user, and checks the result is correct.
- **Cypress** — the tool that drives that robot browser.
- **Spec** — one test file. It ends with `.cy.ts`. Example: `contract-lifecycle.cy.ts`.
- **Docker container** — a small, isolated box that runs one program (the API, the website, the database).
- **Docker image** — a *frozen photograph* of your code. A container is started **from** an image. This idea matters a lot in problem 4.
- **Port** — a numbered door on your computer. A program listens on one port. If you knock on the wrong door, nobody answers.
---
## 3. Result of the full test run
I ran all 38 test files once, cleanly. This took about 44 minutes.
```
✖ 6 of 38 failed (16%) 44:17 259 tests 223 passing 36 failing
```
The 6 test files that failed:
| Test file | Tests | Passed | Failed |
|-----------|-------|--------|--------|
| `contract-lifecycle.cy.ts` | 5 | 0 | **5** |
| `export_one_time.cy.ts` | 16 | 9 | 7 |
| `import_full_train.cy.ts` | 13 | 8 | 5 |
| `intercity_one_time.cy.ts` | 16 | 11 | 5 |
| `onboarding.cy.ts` | 3 | 1 | 2 |
| `segment_weight.cy.ts` | 13 | 1 | 12 |
The other 32 test files passed completely.
---
## 4. Problem 1 — The tests died immediately with `exit code 137`
### What you saw
The command stopped almost at once. There were no test results. The exit code was `137`.
### What it means
`137` means the program was **force-killed** by the operating system (it is `128 + 9`, where `9` is the "kill" signal). It is *not* a test failure. The tests never even started.
### Why it happened
Look at this part of `docker-compose.e2e.yaml`:
```yaml
cypress:
network_mode: host
# NOTE: host network shares the abstract X-socket namespace with the host.
# Cypress spawns its Xvfb on :99 — run only ONE cypress container at a
# time, and don't run it on a host whose X server occupies :99.
```
Cypress needs a screen to draw the browser on. Because there is no real monitor, it creates a fake screen called **Xvfb** on display number **`:99`**.
Because of `network_mode: host`, that fake screen is shared with the whole computer — **not** kept private inside the container.
So if **two Cypress runs happen at the same time**, both try to take display `:99`. They fight, Chrome dies, and you get `137`.
On this machine there was in fact **another Cypress run already going** (from a second terminal session), which is what killed my run.
### The solution
**Run only one Cypress container at a time.** Before starting, check nothing else is running:
```bash
docker ps --format '{{.Names}}' | grep cypress
```
If that command prints something, wait for it to finish. If it prints nothing, you are safe to start.
> This was a problem with the machine being busy — **not** a problem with the tests or the application.
---
## 5. Problem 2 — `E2E_PORTAL_PORT=5374` pointed at a dead port
### What you saw
Tests that use the customer **portal** website failed. They could not open the page at all.
### Why it happened
Your command set the portal port to **5374**:
```bash
E2E_PORTAL_PORT=5374 docker compose ...
```
But the portal container was actually published on port **5373**. Here is the proof:
```
$ cat e2e/freight/.e2e-ports.json
{ "E2E_API_PORT": 3101, "E2E_PORTAL_PORT": 5373, "E2E_BACKOFFICE_PORT": 5383, ... }
$ docker ps
edr-freight-e2e-freight-portal-e2e-1 0.0.0.0:5373->80/tcp
```
Here is the important detail. Setting `E2E_PORTAL_PORT=5374` **only changes where Cypress looks**. It does **not** move the already-running portal container. From `cypress.config.ts`:
```ts
portalUrl: process.env.CYPRESS_PORTAL_URL ?? "http://localhost:5373",
```
So Cypress knocked on door **5374**, but the portal was living behind door **5373**. Nobody answered.
This affected the 5 test files that open the portal:
`onboarding`, `contract-lifecycle`, `cross-app`, `export_one_time`, `intercity_one_time`.
### The solution
Use the port that matches the running container — simply leave the setting out, because `5373` is already the default:
```bash
docker compose -f docker-compose.e2e.yaml --profile cypress \
run --rm cypress --spec 'cypress/e2e/flows/*.cy.ts'
```
Or, best of all, use the project's official launcher, which picks matching ports for everything automatically:
```bash
pnpm e2e:freight:run
```
> After using the correct port, `cross-app.cy.ts` passed 3/3 — proving the port was the only thing wrong there.
---
## 6. Problem 3 — The real bug: `contract-lifecycle` failed all 5 tests ✅ FIXED
This was the one **genuine code problem**, and it is now fixed and verified.
### What you saw
```
1) customer creates and submits a GENERAL import container contract:
AssertionError: Timed out retrying after 10000ms:
Expected to find element: `[role="checkbox"][aria-label="20ft Container"]`,
but never found it.
```
And then 4 more failures after it.
### Why it happened
The test was looking for a **checkbox** to choose the container size (20ft):
```ts
cy.get('[role="checkbox"][aria-label="20ft Container"]').click();
cy.get('textarea[placeholder*="Electronics"]').type("E2E electronics shipment scope");
```
But **the application was deliberately changed**. Container contracts now automatically cover **both** 20ft and 40ft sizes, so the checkbox was removed and replaced by a simple information card. The cargo description was also moved to the booking step.
You can see this clearly in the current application code, `step3-cargo-scope.tsx`:
```tsx
{/* Container scope: the contract always covers BOTH sizes and quotes both
rates. Quantities (a size can be 0) and the cargo description are
captured at booking time. */}
{cargoType === "container" && (
...
<Text fz={14} fw={700}>20ft &amp; 40ft containers covered</Text>
```
And the validation rules confirm nothing else is needed (`schema.ts`):
```ts
// Container scope needs no validation: both sizes are always in scope and
// the cargo description moved to booking time.
```
So: **the application was updated, but the test was not.** The test kept looking for a button that no longer exists.
**Why all 5 tests failed, not just one.** These 5 tests run in order and build on each other. Test 1 creates the contract; tests 25 then approve and sign *that* contract. Because test 1 could not finish, there was no fresh contract, so tests 25 had nothing correct to work on and failed too. This is called a **cascade failure** — one real error causing several fake-looking errors.
### The fix
I removed the two steps that referred to the deleted fields.
**`e2e/freight/cypress/e2e/flows/contract-lifecycle.cy.ts`**
```diff
- // Step 1 — Cargo & Route.
+ // Step 1 — Cargo & Route. Container contracts now auto-cover BOTH 20ft &
+ // 40ft (no size picker — just an info card) and the cargo description moved
+ // to booking time, so the scope select plus the route is all this step needs.
cy.mantineSelect(/^Cargo Scope/, /Containerized/);
- cy.get('[role="checkbox"][aria-label="20ft Container"]').click();
- cy.get('textarea[placeholder*="Electronics"]').type(
- "E2E electronics shipment scope",
- );
cy.mantineSelect(/^Origin Yard/, "Djibouti Port Terminal");
```
I found the **same outdated code in two more test files** and fixed them as well, so the problem is solved everywhere and not just in one place:
**`export_one_time.cy.ts`**
```diff
cy.mantineSelect(/^Cargo Scope/, /Containerized/);
- cy.get('[role="checkbox"][aria-label="20ft Container"]').click();
- cy.get('[role="checkbox"][aria-label="40ft Container"]').click();
- cy.get('textarea[placeholder*="Electronics"]').type("E2E export electronics");
cy.mantineSelect(/^Origin Yard/, ORIGIN_YARD);
```
**`intercity_one_time.cy.ts`**
```diff
cy.mantineSelect(/^Cargo Scope/, /Containerized/);
- cy.get('[role="checkbox"][aria-label="20ft Container"]').click();
- cy.get('textarea[placeholder*="Electronics"]').type(
- "E2E intercity electronics between Ethiopian yards",
- );
cy.mantineSelect(/^Origin Yard/, ORIGIN_YARD);
```
### A second, smaller bug found while checking the fix
After the first fix, 4 of 5 tests passed and one still failed:
```
AssertionError: expected '<p.mantine-Text-root>' to be 'visible'
This element is not visible because its content is being clipped by one of its
parent elements, which has a CSS property of overflow: hidden, clip, scroll or auto
```
The contract *was* created correctly. The word "Submitted" *was* on the screen. But it sits inside a side-scrolling list, so Cypress treated it as hidden.
Because the line just after it already checks the database properly (which is the stronger, more trustworthy check), I made the screen check clip-proof:
```diff
- cy.contains("Submitted", { timeout: 15000 }).should("be.visible");
+ // `exist`, not `be.visible`: the status badge sits inside the list's
+ // horizontally-scrolling container, so Cypress reports it as clipped by an
+ // overflow parent. The DB assertion below is the authoritative check.
+ cy.contains("Submitted", { timeout: 15000 }).should("exist");
```
### Proof that it works
**Before the fix:**
```
contract-lifecycle.cy.ts 5 tests 0 passing 5 failing
```
**After the fix:**
```
✓ customer creates and submits a GENERAL import container contract (7574ms)
✓ marketer accepts the submission and approves the LINE_STAFF step (3668ms)
✓ director approves the final step — contract PDF becomes ready (2916ms)
✓ customer signs the contract with OTP (5143ms)
✓ staff counter-signs — GENERAL contract becomes CONTRACT_ACTIVE (3643ms)
5 passing (29s) EXIT=0
```
**All 5 tests now pass.**
---
## 7. Problem 4 — The remaining 5 test files: the app inside Docker is old
This is the second most important finding, and it explains **almost all remaining failures**.
### What you saw
Many strange, unrelated-looking errors, for example:
```
CypressError: cy.request() failed on:
http://localhost:3101/api/train-scheduling/schedules/<id>/dispatch
The response we received from your web server was:
> 400: Bad Request
```
```
AssertionError: expected '/dashboard/operations/train-scheduling-v2'
to match /\/dashboard\/operations\/train-scheduling-v2\/.+/
```
```
AssertionError: Expected to find content: 'Clearance Review' within the selector: '[role="tab"]'
```
### Why it happened
**The test files are new, but the running application is old.**
- The test files live on your disk and are shared into the container live, so they are always the newest version.
- The API and websites run from **Docker images**, which are frozen photographs of the code. They only change when you **rebuild** them.
Here are the actual times:
```
freight-api-e2e image built: 2026-07-23 09:22
freight-portal-e2e image built: 2026-07-23 09:22
freight-backoffice-e2e image built: 2026-07-23 09:22
latest commit (HEAD): 2026-07-23 20:24 ← 11 hours newer
```
**5 commits were made after those images were built:**
```
668b5e1c add permissions and fix issues
40f16f3c changes
12767605 changes
15a6bab5 revert back the clerance payment
13609f8d changes
```
### The clearest proof
Commit `668b5e1c` changed the **API and its tests together**, in the same commit:
```
apps/edr-freight-api/src/modules/train-scheduling/train-scheduling.controller.ts
apps/edr-freight-api/src/modules/train-scheduling/train-scheduling.service.ts
e2e/freight/cypress/e2e/flows/import-utils.ts
e2e/freight/cypress/e2e/flows/import_full_train.cy.ts
```
So the **new test** sends the **new** data shape to `/dispatch`, but the **old API** inside Docker does not understand it and replies `400 Bad Request`.
This is not a bug in the code. It is simply **new tests talking to an old server**.
The `train-scheduling-v2` failures have the same cause: creating a train schedule fails on the old server, so no train exists, and every later step in those files fails as a cascade (this is why `segment_weight` lost 12 of 13 tests from a single root cause).
### The solution
Rebuild the Docker images so they contain the current code, then run the tests again:
```bash
# stop the old stack and rebuild from current code
docker compose -f docker-compose.e2e.yaml down
docker compose -f docker-compose.e2e.yaml build
pnpm e2e:freight:run
```
> ⚠️ Note: `down` deletes the test database (it is a throwaway database, which is normal and safe).
> ⚠️ Note: only do this when nobody else is using the same stack.
---
## 8. Files I changed
Only test files were changed. **No application code was modified.**
| File | Change |
|------|--------|
| `e2e/freight/cypress/e2e/flows/contract-lifecycle.cy.ts` | Removed deleted container-size checkbox + description box; made the "Submitted" check clip-proof; updated the header comment |
| `e2e/freight/cypress/e2e/flows/export_one_time.cy.ts` | Removed deleted 20ft + 40ft checkboxes and description box |
| `e2e/freight/cypress/e2e/flows/intercity_one_time.cy.ts` | Removed deleted 20ft checkbox and description box |
Evidence the last two fixes helped, even on the old server:
- `export_one_time` — previously failed at contract creation; now reaches 9 passing tests.
- `intercity_one_time` — previously failed at contract creation; now reaches 11 passing tests.
Their remaining failures are all from problem 4 (old Docker images).
---
## 9. How to run the tests correctly
**Step 1 — make sure no other Cypress is running** (this avoids the `137` crash):
```bash
docker ps --format '{{.Names}}' | grep cypress
```
**Step 2 — rebuild so Docker has the current code:**
```bash
docker compose -f docker-compose.e2e.yaml down
docker compose -f docker-compose.e2e.yaml build
```
**Step 3 — run the tests using the official launcher** (it chooses matching ports for you):
```bash
pnpm e2e:freight:run
```
If you prefer the raw Docker command, **do not** override the portal port unless you also restart the portal container on that same port:
```bash
docker compose -f docker-compose.e2e.yaml --profile cypress \
run --rm cypress --spec 'cypress/e2e/flows/*.cy.ts'
```
---
## 10. Conclusion
- **One real bug was found and fixed:** three test files were still clicking a container-size checkbox and a description box that the application no longer has, because container contracts now cover both 20ft and 40ft automatically.
- `contract-lifecycle.cy.ts` went from **0 of 5 passing** to **5 of 5 passing**, confirmed by running it twice.
- The `exit 137` crash was caused by **two Cypress runs at the same time** fighting over the shared virtual screen `:99`.
- The portal failures were caused by **`E2E_PORTAL_PORT=5374`**, while the portal was really on **5373**.
- The remaining failures are caused by **Docker images that are 11 hours older than the code**. They need a rebuild, not a code fix.
**Recommended next step:** rebuild the Docker images and run the full suite again. Only then can the remaining 5 test files be judged fairly.