Load Testing — Options & Constraints

Backend and throughput testing for the Blitz platform. Nothing here has been run yet — this page records the options, the real blockers, and the decisions needed first. Contrast with Frontend Performance, which has measured results.

Read this first: staging cannot produce a meaningful load-test number

Per-service caps in fin-infra/railway/shared/resources.ts:

• Every service: 0.5 vCPU / 0.5 GB, restart ON_FAILURE max 2
• Exceptions at 1 GB: wfw, wfwpdf, bapiproxy, blitz-api; blitz-apimgmt at 2 vCPU / 1 GB
• blitz-api was already OOM-killed at 512 MB under normal staging traffic (5 restarts in 7 days) — raised to 1 GB as mitigation, not a diagnosis; the suspected leak is still open (fin-infra#69)
• Temporal is capped at 0.5 GB. resources.ts itself flags this as "aggressive for an orchestrator and fine only under staging's light load — the most likely service to OOM under a heavier run"
• Staging auto-scales-down nightly at 10:00 PM IST; scale-up is manual only

Load-testing staging today measures the caps, not the application

You would learn "Temporal OOMs at N concurrent users," which is already known. Any meaningful backend load test needs a deliberate temporary limits raise first — with a cost and blast-radius decision attached. Note also that Postgres is not covered by the converge (it stays uncapped), so it is the one component whose real limits could be measured on staging as-is.

Option 1 — API load test (k6)

Highest signal per unit effort. blitz-api:10001 is the single front door, and k6's JS-scripted VUs suit the JWT flow.

Auth is scriptable. POST /api/auth/login with {userName|email, password} returns accessToken + refreshToken, so VUs can log in and reuse tokens. Remember tenant resolution is by Host header — see Measuring.

Where the test runs changes what you measure

All backends are private-only (.railway.internal) and unreachable from a GitHub runner or your laptop. So k6 must either:

• run inside Railway as a throwaway service (a temporary IaC change, but measures the backend directly), or
• hit the public blitz-ui nginx and traverse its /api proxy — in which case you are also measuring nginx and the 0.5 vCPU cap on it.

Option 2 — Workflow / batch throughput

For a financial-ops product this is arguably more important than HTTP RPS — reconciliation and close are batch-shaped, not request-shaped.

A real harness already exists: blitz/scripts/cashapp/onboarding-tools/08-scale-test.ts drives the cashapplication workflow at N=5/50/500 items and measures wall-clock plus per-stage signals.

Gaps to close before it can run against a deployed environment:

• It is local-only — shells into docker exec blitz-db-1 psql
• Temporal's 0.5 GB cap is exactly what would break first at scale

Option 3 — Frontend performance

Done. See Frontend Performance — the only option with no infrastructure blocker, and it produced an actionable finding on the first run.

Option 4 — Database

pgbench or query profiling against multi-schema Prisma with per-tenant DBs. Notable: Postgres is excluded from the resource converge, so unlike every other service it is uncapped on staging.

Decisions needed before running Options 1 or 2

1. Raise the limits, or not? Without a temporary raise the numbers measure resources.ts, not the app. This is a cost decision.
2. Where does the load generator run? Inside Railway (accurate, needs IaC change) or through nginx (easier, contaminated).
3. Which tenant and what data? Load-testing against a real tenant DB writes real financial rows. A throwaway tenant is wanted — but tenant provisioning is proven locally and not yet proven on staging, so that may itself be prerequisite work.
4. Nightly scale-down. Any run must fit the manual-up window before 10:00 PM IST, or the environment disappears mid-test.

Recommended sequence

1. ✅ Frontend — done, no blockers, real finding
2. Workflow/batch (Option 2) — targets the actual product shape and reuses existing code
3. API load test (Option 1) — hold until the limits question is decided, otherwise you pay for a number you cannot use

Also free and unblocked

Lighthouse CI (@lhci/cli, free) can gate PRs on score/bundle-size budgets. GitHub's ubuntu-latest runners are amd64 natively, so the Apple-Silicon Docker caveat in Measuring does not apply in CI. Worth wiring once blitz-ui#837 lands, so the 8 MB chunk cannot silently return.