LLM PII Tokenization — Design & Implementation Spec

Owner	Classification	Version	Effective	Next review	Status
Sekhar Prakash	Internal	0.1	2026-08-26	2026-11-27	Draft

Purpose. The technical de-identification design for Finni's LLM data flows — tokenise identity, retain semantics, scrub free text, at the model boundary. It remedies the "no sanitisation of user input before LLM calls" finding and extends it to outbound data egress.

Related: Compliance Readiness (governance companion) · blitz SECURITY-POSTURE (2026-03-10) · Findings Register.

---

1. Problem

Finni (LangGraph agents, financeagents) currently runs on Google Gemini via getModel.

• The live .env overrides the template's local granite4 with DEFAULT_FINNI_MODEL=gemini-flash on a free AI-Studio key (GOOGLE_API_KEY).
• That key's terms permit Google to use submitted content to improve their models.

Every Finni turn therefore ships real tenant financial data to an external processor:

• Narration tools (journal/collections/cashapp/recon/closehub) — the tool's formatted summary becomes a ToolMessage sent to the LLM. Bounded (caps: take:10, slice(0,5/15), top-N) but includes names, entities, amounts, statuses.
• L2 judges (cid-judge, match-judge, remittance-judge, collection-judge, po-match-judge, gl-coding-judge) — a single decision context, but higher sensitivity: cid-judge.buildPrompt sends payer name, bank account, email, plus candidate customers' payment/invoice history.

Nothing sends bulk data — but the identifier fields are exactly what we must not leak.

2. Core principle

Tokenize identity. Keep semantics + numbers. Scrub free text. One stable bidirectional session map.

Finni's job is to narrate structure/numbers, and the judges match on correlation/equality, not name-meaning. So replacing identifiers with reversible tokens costs ~nothing in quality — provided:

• semantics stay readable, and
• the mapping is consistent.

Identity vs Attribute

• Identity (tokenize): customer/vendor name & number, person names (assignee/approver/preparer), entity/legal-entity names, email, bank account, external references where sensitive.
• Attribute (keep): amounts, status, category, risk band, exposure, days-past-due, dunning reasons, dependency/slack/impact, account type/name (chart-of-accounts semantics), dates.
• Free text (scrub, don't blank): journal narratives, task titles, the user's typed prompt — run PII detection over the string, replacing only detected identifiers.

3. Per-module classification (from live code)

Module	Tokenize (identity)	Keep (semantics/numbers)	Free-text to scrub
Collections	customer name/number	risk, exposure, days-past-due, reasons, action	notes
CashApp	customer/payer, email, bank acct, cust number	amounts, status, references	remittance text
Journal (Q&A)	assignee/approver, entity	JV#, status, category, amount	—
Journal (draft)	entity, customer	account names/types (keep!)	narratives
Close / CDG	preparer name (+ team, optional)	task titles, DAG, slack, impact-hours	title mentions
L2 judges	name, bank acct, email, customer name	amounts, refs, dates, ageing, history	—

Draft is the only "keep semantics" special case — the model needs "Utilities Expense" to pick GL6101; those are COA structure, not customer identity.

4. Why the judges tokenize losslessly (equality preservation)

cid-judge matches a payment to a customer by:

• correlation — amount/ref/date align with open invoices, and
• identifier equality — same bank account / reference.

If the same real value maps to the same token on both the payment and the candidate:

payment.bank = ⟦BANK_1⟧   candidate.bank = ⟦BANK_1⟧   → still equal → same verdict

The verdict is identical; Google never sees the real bank account/email/name. Consistent tokenization preserves equality by construction. This makes the judges the #1 target and the safest to tokenize.

5. Architecture — a middleware at the model boundary

A session-scoped Tokenizer wraps the LLM handoff, model-agnostic — works for Gemini · Vertex · local granite:

                    ┌─────────────── your trust boundary ───────────────┐
 user input ──►  tokenize(in)  ──►                                       │
 tool output ─►  tokenize(in)  ──►  LLM prompt (tokens + semantics) ──►  │──► LLM (external)
                    │  token↔real map (session-scoped, secured)          │
 UI  ◄── detokenize(out) ◄────────  LLM response (tokens) ◄──────────────│◄── LLM
                    └────────────────────────────────────────────────────┘

• Outbound: structured-tokenize known identifier fields (deterministic) + scrub free text (Presidio) → send.
• Inbound: de-tokenize the LLM's response before it reaches the UI/user.
• Map: per-conversation, 1:1, bidirectional, held server-side. Same real value → same token everywhere (tool outputs and user input) — this is what preserves cross-turn/cross-tool references and judge equality.

Two data types → hybrid (don't over-engineer)

1. Structured identifier fields — the tools already funnel identity through named helpers (userLabel, businessentityname, customername…). Tokenize these deterministically in TS at the tool-output boundary. No NER guessing → reliable. This is v1 and needs no new infra.
2. Free text (prompt, narratives, task titles) — Microsoft Presidio (local Docker REST service; NER + custom recognizers for finance entities). This is v2.

6. Implementation plan

v1 — pure-TS structured tokenizer (no new dependencies)

1. Tokenizer class: tokenize(kind, value) → token (memoized per session), detokenize(text) → text. Token format paraphrase-resistant, e.g. ⟦CUST_1⟧, ⟦PERSON_3⟧, ⟦ENTITY_1⟧, ⟦BANK_1⟧.
2. Hold the map on the LangGraph thread state (keyed by threadId) so it persists across turns and is scoped per conversation.
3. Wrap the two boundaries:
   • Tool outputs: the tools already isolate identity in helpers (userLabel, entity fields) → route those through tokenize() before the ToolMessage is built.
   • User input: tokenize known entities present in the session map; leave the rest for v2 (Presidio).
   • Judges: tokenize name/bankaccount/email/customername in buildPrompt, consistently across payment + candidates.
4. De-tokenize every LLM response (freeform narration + any surfaced strings) before returning to the FAB.
5. System-prompt line: "Tokens like ⟦CUST_1⟧ are opaque handles — reproduce them verbatim, never invent or alter them."
6. Start with one judge (cid-judge) + collections narration — the two highest-value cases.

v2 — Presidio for free text

• Stand up presidio-analyzer + presidio-anonymizer as a local Docker service (on-prem, language-agnostic REST).
• Custom recognizers for finance entities (tenant entity list, GL account formats, customer registry).
• Apply to: user prompt, journal narratives, task titles, remittance text.
• Optional: LLM Guard (built on Presidio) if we also want prompt-injection/output scanning in the same layer — note SECURITY-POSTURE.md flags prompt injection separately.

v3 — hardening

• Move the map to a secured store with access control + audit logging (it is a re-identification key — GDPR Art 4(5) "additional information kept separately"; see compliance spec).
• Metrics: % of turns tokenized, tokens/turn, de-tokenization mismatch rate.

7. Hard cases & mitigations

Case	Risk	Mitigation
User types a name ("why is Acme late?")	mention must map to the same token the tool used	(a) string-match against the session map if already present; (b) Presidio NER + entity-resolution to the canonical customer (v2)
Name carries reasoning meaning	model needs the semantic	those signals must come from attributes (risk band, terms, segment), not the name — add a structured attribute rather than leak the name
LLM paraphrases a token → de-tokenize misses	broken restore	paraphrase-resistant token format + verbatim-reproduction prompt line + fuzzy fallback match
Draft account semantics	tokenizing "Utilities Expense" blinds the model	never tokenize account names/types — COA structure, not identity

8. What this does NOT solve (scope boundary)

• Pseudonymized data is still personal data (reversible) — does not remove the need for a DPA/SCCs/subprocessor listing. See compliance spec.
• The free AI-Studio key is the real blocker — tokenization + an enterprise endpoint (Vertex/Bedrock/Azure, no-train, ZDR) is the defensible pairing.
• Financial figures may still be confidential even de-identified — covered by the endpoint contract, not tokenization.

9. Acceptance criteria (POC)

• cid-judge: identical verdicts with tokenization on vs off (equality preserved), and the outbound prompt contains no real name/bank/email.
• Collections narration: the FAB reply reads identically to the user (real names restored), while the LLM-facing payload is de-identified.
• Close/CDG: tokenization is a near-no-op (validates the low-risk end).

Revision history

Version	Date	Author	Change
0.1	2026-08-26	Sekhar Prakash	Initial design and implementation spec.

Finaisse Internal — Confidential. Not for external distribution.