How we built a PII masking layer for LLM APIs — local detection, reversible tokens, one line to integrate

If you're building LLM features on top of OpenAI or Anthropic, you're almost certainly sending raw user data to a third-party model provider. Names, emails, phone numbers, tax IDs, health records — whatever your users type, it goes straight to the API.

Here's the uncomfortable part: every attempt to fix this problem seems to make it worse. The most obvious fix — sending your text to a cloud anonymisation service first — means you're solving a data privacy problem by sending your sensitive data to another third party.

I was talking to a healthtech team recently that had been blocked from using GPT-4 for clinical notes for months. Not because the engineers didn't want to — they did. Legal wouldn't sign off because every API call meant patient data leaving their infrastructure. The problem wasn't capability. It was the missing privacy boundary between their data and the LLM.

Armos is that boundary. A local detection and masking layer that sits between your application and the LLM API — PII never leaves your server, and real values are restored in the response automatically.

This is how it works under the hood.

The problem with the obvious approaches

Option 1: Regex scrubbing

Fast to write, breaks constantly. Email regexes miss edge cases. Names are impossible. You end up with a pile of patterns that need constant maintenance and still let things through.

Option 2: Send everything to a cloud anonymisation API

Same problem, different server. You haven't kept the data in-house — you've just added a hop.

Option 3: Build it yourself with Presidio

Microsoft's Presidio is excellent — it's what powers Armos's detection. But it's detection only. You still need to build the masking layer, the vault, the de-masking logic, and wire it into your SDK calls. That's a week of work for a first pass and months of edge cases.

What Armos does instead

Three steps, all local:

1. Detect

Presidio + spaCy runs on the text before it leaves your process. No network call. No data sent anywhere during detection.

2. Mask with reversible tokens

Detected entities are replaced with deterministic tokens:

"Patient John Smith, Aadhaar 2345 6789 0123"
→
"Patient [PII:NAME:c4587843], Aadhaar [PII:AADHAAR:473adcf3]"

The token format encodes the entity type and a hash of the original value. Same value always maps to the same token — so if "John Smith" appears twice, it gets the same token both times, and the LLM can reason about it consistently.

3. Restore

After the LLM responds, the library scans the output for tokens and swaps them back. Your application receives the original text. The model never saw the real values.

The token vault

Tokens need to map back to real values. The library keeps a vault — a simple key-value store — inside the process by default, with an optional Redis backend for cross-process persistence.

# In-memory (default)
client = ArmosOpenAI(OpenAI())

# Redis-backed — tokens survive across requests and processes
client = ArmosOpenAI(OpenAI(), store="redis", redis_url="redis://...")

The vault never leaves your infrastructure. Armos has no server. There's no telemetry, no cloud component.

The integration

This is the entire change to existing code:

# Before
from openai import OpenAI
client = OpenAI()

# After
from openai import OpenAI
from armos import ArmosOpenAI
client = ArmosOpenAI(OpenAI())

Everything downstream works identically — same method signatures, same response objects. The masking and de-masking happen invisibly inside the privacy layer.

What gets detected

10 entity types out of the box:

Names — via spaCy NER (en_core_web_lg)
Email, phone, credit card, IP — Presidio built-ins
Aadhaar, PAN — custom regex recognisers (Indian identifiers that no existing tool handles reliably)
SSN, IBAN — Presidio built-ins with checksum validation
API keys — custom pattern recogniser for OpenAI, AWS, GitHub key formats

Accuracy

I ran a 1,000-sample benchmark across all entity types:

Entity	Accuracy
Email	100%
Aadhaar	100%
PAN	100%
SSN	100%
IBAN	100%
Credit card	100%
Phone	100%
API keys	100%
IP address	99.8%
Person name	96.4%

The 3.6% miss rate on names is entirely Indian names — en_core_web_lg was trained predominantly on Western text. I'm working on a supplemental approach for this.