LLM Compliance & Security Readiness

Owner	Classification	Version	Effective	Next review	Status
Sekhar Prakash	Internal	0.1	2026-08-26	2026-11-27	Draft

Purpose. The governance and compliance view of Finni's LLM processing — current posture, desired end state, prioritised fixes, and the regulatory checklist. Companion to the Tokenization spec (the technical de-identification design).

Related: Tokenization · DPIA · Findings Register.

Evidence tags: ✓ verified = read in code at the cited file:line. ~ unconfirmed = plausible, flagged for a dedicated check, not yet reproduced.

---

1. Executive summary

Finni (LangGraph agents, financeagents) sends real tenant financial data to an external LLM on every turn.

• The tokenization spec addresses one of four places that data lands.
• The other three — intelligence audit log · stdout logs · conversation checkpoint state — persist un-tokenized PII inside our own boundary and are not covered by the model-boundary tokenizer.

Good news, confirmed in code:

• No prompt or response text is logged (only metadata). ✓
• No external LLM tracing (no LangSmith / Langfuse / LLM-level OTEL). ✓
• The L2 LLM judges are user-triggered only, one record per call — no batch fan-out. ✓

Blocking / high-priority gaps, confirmed in code:

• Authorization role is read from the request body and fails open to the most-privileged controller — even though the JWT-loaded userInfo.roles is already present on the request. Any authenticated user can self-escalate by sending role:"controller". It is authentication done right, authorization done wrong, not a missing JWT. ✓
• Conversation state is keyed on thread_id alone in a process-global in-memory map, and HTTP thread IDs carry no tenant component → cross-tenant collision/bleed risk. ✓
• Tool-call arguments are logged verbatim to stdout. ✓
• The free AI-Studio Gemini key (trains on submitted data) remains the headline blocker — tracked separately, resolved by the enterprise-endpoint move.

---

2. The four PII sinks (the mental model)

Tokenization at the model boundary is necessary but not sufficient. The same identity data reaches four sinks; only the first is covered by the tokenization spec.

#	Sink	Covered by tokenization spec?	Evidence	Residual PII
1	LLM prompt (outbound)	✅ yes (v1 structured, v2 Presidio)	TOKENIZATION-SPEC.md §5–6	None after v1/v2 + enterprise endpoint
2	finbase.intelligencelog.layer2reasoning (durable DB)	❌ no — stores the detokenized LLM reasoning	✓ finbase.prisma:1494–1510	Names, bank accts, amounts, per decision
3	stdout logs (finni.toolNode.invoke)	❌ no — logs toolCall.args verbatim	✓ finni-agent/index.ts:207	Whatever the LLM put in tool args
4	Checkpointer (conversation state)	❌ no — in-memory, thread_id-keyed	✓ memoryCheckpointer.ts:28	Full message history incl. identifiers

Consequence: after tokenization ships, real PII still lives in our DB and logs.

• That is expected (system of record).
• But sinks 2–4 must inherit the same treatment as any other PII store: access-control · retention · DSAR/erasure.
• "We tokenized" is not, by itself, a compliance position.

---

3. Current posture — what is actually true today

3.1 L2 judge triggering (✓ verified across all call sites)

All six judges are user-triggered, inline in an HTTP request, one record per invocation. No cron / queue / worker path invokes a judge.

Judge	Endpoint / trigger	Condition	Records/call
cid-judge	/customer-identification/suggest	isAmbiguous && !skipLayer2	1
match-judge	/payment-matching/suggest	match ambiguous, not perfect tie	1
remittance-judge	/remittance-matching/suggest	remittance ambiguous	1
collection-judge	/finni/collection-customer-risk/:num	result.borderline	1
po-match-judge	/…/:id/layer2-judge	explicit "second opinion" click	1
gl-coding-judge	/…/:id/layer2-judge	explicit "second opinion" click	1

Nuance:

• Layer 1 (deterministic resolution) can run in batch — intelligencelog.source accepts 'manual' | 'batch' | 'auto'.
• But Layer 2 (the LLM judges) is only reached by an interactive request on a single ambiguous record. A bulk cash-application run does not fan out LLM calls today.
• If a "run judges across the batch" mode is ever added, egress/cost/free-tier exposure changes materially — it must be gated behind the enterprise endpoint.

3.2 Logging (✓ verified)

What	Logged?	Where	Contains PII?
Prompt text	No	—	—
Response text	No	—	—
Judge/agent metadata (action, timings, counts, model, confidence)	Yes	Pino → stdout (logger.ts, level info)	No
Tool-call arguments	Yes, verbatim	Pino → stdout finni-agent/index.ts:207	Yes
LLM reasoning (layer2reasoning)	Yes	finbase.intelligencelog (DB)	Yes
Token usage / cost	Yes	sys.llmusage (per-tenant DB)	No (counts + user email)
External LLM tracing (LangSmith/Langfuse/OTEL)	None found	—	—

• The only OpenTelemetry exporter is for Temporal workflow tracing, not prompts.
• Nothing ships prompts/responses to a third party beyond the LLM provider itself.
• Guardrail: no one enables LANGCHAIN_TRACING / a tracing vendor without a signed DPA first.

---

4. Findings & fixes

• Severity reflects impact × likelihood in a multi-tenant prod deployment.
• Effort is rough dev days.

#	Finding	Sev	Evidence	Fix	Effort
F1	Authz role read from the request body, not from the JWT-loaded userInfo.roles that is already on the request. Fails open to controller (most privileged) when the field is absent. Any authenticated user self-escalates by sending role:"controller". Affects every interactive user request — not a non-user/service edge case.	High	✓ finni.ts:57,71–72; roles present via user.service.ts:5–8 → authenticator.ts:211–234; gated in tools at collection-tools.ts:127–130, journal-tools.ts:142, reconciliation-tools.ts:158–170, closehub-tools.ts:92	Derive userRole from userInfo.roles (already present — no new lookup / no #671 dependency); map the role-set → the tools' role vocabulary; fail closed to least privilege; delete role from the request schema (keep telemetry-only if needed). #671 (per-app businessuserroles) is the richer follow-up, not required to close the escalation.	0.5–1
F2	Cross-tenant conversation bleed. HTTP threadId = finni_${Date.now()} (no tenant); checkpointer is process-global, keyed on thread_id alone.	High	✓ finni.ts:53, memoryCheckpointer.ts:28	Key state on ${tenantId}:${thread_id}; generate thread IDs with tenant + random suffix. Move to a durable, tenant-scoped checkpoint store (also survives restarts).	2–3
F3	Tool args logged verbatim to stdout.	High	✓ finni-agent/index.ts:207	Log arg keys/types, not values: Object.keys(args).	0.5
F4	intelligencelog.layer2reasoning = un-governed PII at rest. Durable, un-tokenized, no retention.	High	✓ finbase.prisma:1496	Treat as PII store: access control, retention TTL, include in DSAR/erasure. Do not delete — it is the AI-Act explainability record.	2
F5	Free AI-Studio Gemini key trains on submitted data; gemini-flash-latest on free quota.	High	(tracked separately)	Enterprise endpoint (Vertex/Bedrock/Azure), no-train + ZDR, signed DPA + SCCs, region-pinned, model pinned.	— (infra)
F6	Rate limiting is global (~1/s), not per-tenant/user.	Med	rateLimiter.ts	Per-(tenant,user,endpoint) sliding window (Redis). Return 429 over threshold.	1–2
F7	API key in a static in-memory registry.	Med	modelConfig.ts	Load secrets from a manager (AWS Secrets Manager / Vault) at the AWS/prod cutover; not a static object.	1
F8	No Gemini safetySettings; no LLM-call timeout; model unpinned (*-latest).	Med	modelConfig.ts	Add safetySettings, a 30s timeout, and pin the model version once billing is funded.	1
F9	Prompt injection via tool-output string interpolation (user query / DB free text echoed into LLM-facing markdown).	Med	~ cashapp-tools.ts	Return structured data from tools (let the LLM render), or escape markdown. Cover DB free text (remittance/journal notes) in Presidio v2 + consider LLM Guard.	2–3
F10	Journal-draft grounding not re-verified server-side (tenant-ownership of grounding.accounts).	Med	~ journal-draft.ts	Re-fetch grounding server-side under the caller's auth context, or verify tenant ownership before the LLM call.	1

---

4.1 F1 in depth — the JWT reaches Finni; only authorization is wrong

A reasonable objection: "the user's JWT goes to Finni, so where's the gap?"

• The JWT does reach Finni and is used correctly for identity.
• The gap: authorization scope is decoupled from it.

The authoritative roles are already on the request. The auth chain loads them:

• UserService.getInstanceById includes roles: true + permissions: true ✓ user.service.ts:5–8.
• The authenticator verifies the JWT and returns that full instance as userInfo ✓ authenticator.ts:211–234.

So in the Finni controller, userInfo.roles is populated. But the controller uses userInfo only for the email and takes the role from the body:

const userEmail = userInfo?.email ?? "";   // ✓ from JWT — authoritative
const userRole  = role;                      // ✗ from body — caller-supplied   (finni.ts:71–72)
// ...and role defaults to "controller" when omitted                            (finni.ts:57)

userRole is forwarded into config.configurable.userRole, which is what the tools gate on — e.g. an elevated role drops the own-queue filter and returns the whole portfolio:

const assigneduseremail = isElevatedRole(userRole) ? undefined : userEmail;   // collection-tools.ts:127–130
const role = (config?.configurable?.userRole ?? "controller").toLowerCase();  // closehub-tools.ts:92 — fail-open

Two problems, both mainline (not service/non-user paths):

1. Spoofable — an analyst (valid JWT, userInfo.roles = {analyst}) sends role:"controller" and is scoped as a controller.
2. Fail-open — omitting role yields "controller", so absence grants the most access, not the least.

Fix is local and small (0.5–1 day):

• Derive userRole from userInfo.roles.
• Map that set to the tools' role vocabulary.
• Fail closed to least privilege.
• Remove role from the request schema.

The internal-service-token path already returns a no-permissions identity (userInfo:{id:"system"}, authenticator.ts:185–189) and is not on Finni's routes, so it is unaffected.

5. Desired end state

5.1 Target data-flow (all four sinks governed)

                    ┌──────────────── trust boundary ─────────────────┐
 user input ──► tokenize(in) ──►                                       │
 tool output ─► tokenize(in) ──►  LLM prompt (tokens + semantics) ──►  │──► Enterprise LLM
                    │  token↔real map — encrypted, access-controlled,  │    (Vertex/Bedrock/Azure)
                    │  audit-logged, per-conversation                  │    no-train · ZDR · region-pinned · DPA
 UI ◄── detokenize(out) ◄──────  LLM response (tokens) ◄──────────────│◄── LLM
                    │                                                  │
                    ├─ intelligencelog: reasoning stored WITH access   │
                    │    control + retention TTL + DSAR reachability   │
                    ├─ stdout: arg KEYS only, never values             │
                    └─ checkpointer: tenant-scoped key, durable store  │
                    └──────────────────────────────────────────────────┘

5.2 End-state acceptance criteria

Isolation & authz

• [ ] Role is derived from the JWT-loaded userInfo.roles; the request-body role cannot influence data scope, and missing/invalid role fails closed to least privilege. (F1)
• [ ] Every conversation/checkpoint key includes tenantId; no two tenants can collide. (F2)
• [ ] Checkpoint store is durable and survives service restarts.

Data minimisation & logging

• [ ] No prompt/response text logged (holds today — regression-test it).
• [ ] Tool-arg values never reach stdout. (F3)
• [ ] No LLM tracing vendor enabled without a DPA on file.

De-identification (per tokenization spec)

• [ ] cid-judge: identical verdicts tokenized vs not; outbound prompt has no real name/bank/email.
• [ ] Collections narration: user sees real names; LLM payload is de-identified.
• [ ] Token map stored encrypted, access-controlled, audit-logged (re-identification key).

Provider & residency

• [ ] Enterprise endpoint with no-train + ZDR; DPA + SCCs signed; subprocessor listed; region pinned; model version pinned. (F5)

AI governance

• [ ] Human-in-the-loop on any decision with legal/significant effect (collections, GL postings).
• [ ] intelligencelog retained under a defined TTL and reachable by DSAR/erasure. (F4)
• [ ] DPIA completed; RoPA updated.

---

6. Compliance checklist

6.1 GDPR / UK-GDPR

• [ ] Lawful basis documented for LLM processing of tenant/personal data.
• [ ] Art 4(5) pseudonymisation — token↔real map treated as separately-kept re-identification info: encrypted, access-controlled, audited. Pseudonymised data is still personal data.
• [ ] Art 28 processor terms — DPA with the LLM provider; SCCs for any non-adequate-country transfer; subprocessor list maintained.
• [ ] Art 32 security — encryption at rest/in transit for the map, intelligencelog, checkpoints.
• [ ] Art 22 automated decision-making — human oversight on collections/GL actions; intelligencelog trace provides the explanation record.
• [ ] DSAR / erasure path covers: tenant DB, intelligencelog (incl. layer2reasoning), checkpointer state, and confirmed zero provider retention.
• [ ] Retention — TTL on intelligencelog; no indefinite storage of AI reasoning.
• [ ] RoPA — Finni LLM processing recorded, including data categories, provider, region, retention.
• [ ] DPIA — completed for the AI processing (inputs: tokenization spec + this doc).

6.2 EU AI Act

• [ ] Risk classification recorded. Financial decisioning may exceed "limited-risk"; assess collections/matching.
• [ ] Transparency — users informed they are interacting with / assisted by AI.
• [ ] Human oversight — judges remain decision-support (user-triggered "second opinion"), not autonomous action. Preserve this.
• [ ] Explainability / logging — intelligencelog trace retained as the decision record.

6.3 SOC2 (Security / Confidentiality)

• [ ] Access control + audit logging on the token map and intelligencelog.
• [ ] Per-tenant rate limiting / abuse controls. (F6)
• [ ] Secrets in a manager, rotated; not in static config or plaintext at prod. (F7)
• [ ] Durable audit log of LLM requests (metadata: user, tenant, model, sizes, latency, tools) — not stdout-only.

---

7. Recommended sequencing

1. Isolation & authz correctness (F1, F2, F3) — cheap, independent of the LLM-provider move, and the items an auditor treats as material. Do first.
2. Govern the reasoning store (F4) — retention TTL + access control + DSAR reachability on intelligencelog.
3. Enterprise ZDR endpoint + DPA (F5) — unblocks the free-key issue and most of §6.1/§6.2.
4. Tokenizer v1 (structured, per spec) — defense-in-depth on top of the endpoint.
5. Hardening (F6–F10) — rate limiting, secrets manager, safety settings/timeouts, injection handling (Presidio v2), grounding verification.

---

8. Out of scope / tracked elsewhere

• The free AI-Studio key (F5) — being resolved by the AWS/prod endpoint move.
• API-layer RBAC/pagination beyond Finni — SECURITY-AND-PERFORMANCE-GAPS.md.
• Financial figures may be confidential even de-identified — covered by the endpoint contract, not tokenization.

Revision history

Version	Date	Author	Change
0.1	2026-08-26	Sekhar Prakash	Initial compliance-readiness assessment (findings, end state, checklist).

Finaisse Internal — Confidential. Not for external distribution.