When we audited our LLM features last quarter, the most sensitive data store we found was not a database. It was the prompt log. Every support conversation, every pasted invoice and every "here is my account, why was I charged twice" message had been stored verbatim, with full request bodies, in a logging index that 60 engineers could search. Nobody planned that. It happened because logging prompts is the obvious way to debug LLM behaviour.

This guide is the pipeline we built afterwards. It covers where personal data enters, how to redact it at the right points, what to log instead of raw text, and the settings on the provider side that matter. It is written for teams shipping LLM features on top of hosted APIs, but the same ideas apply to self-hosted models.

Map where personal data enters

Start with a data flow, not a regex. In a typical feature, personal data enters in four places:

Entry point Examples Who sees it
User input Names, emails, phone numbers, addresses, order ids Model provider, your logs, traces
Retrieved context CRM records, tickets, documents pulled by RAG Model provider, logs, eval datasets
Tool results Database rows returned to an agent Model provider, logs
Model output Summaries that repeat personal data Users, other systems, logs

For each feature, write down which of these apply, where the text is stored, for how long, and who can read it. That table is what your privacy team will ask for anyway.

Decide what the model actually needs

The best redaction is not sending the data. Many features do not need identifiers at all: a sentiment classifier does not need the customer's email, and a summariser can work with "[CUSTOMER]" instead of a full name. Others need some data to work, like a support assistant that must look up an order.

For data the model needs to reason about but not see in clear text, use reversible pseudonymisation: replace values with stable placeholders before the call, and map them back in the output.

import re
import uuid

EMAIL = re.compile(r"\b[\w.+-]+@[\w-]+\.[\w.-]+\b")
PHONE = re.compile(r"\+?\d[\d\s().-]{7,}\d")

def pseudonymise(text: str) -> tuple[str, dict[str, str]]:
    mapping: dict[str, str] = {}
    def sub(kind: str):
        def repl(m: re.Match) -> str:
            value = m.group(0)
            for k, v in mapping.items():
                if v == value:
                    return k
            token = f"[{kind}_{uuid.uuid4().hex[:6]}]"
            mapping[token] = value
            return token
        return repl
    text = EMAIL.sub(sub("EMAIL"), text)
    text = PHONE.sub(sub("PHONE"), text)
    return text, mapping

def restore(text: str, mapping: dict[str, str]) -> str:
    for token, value in mapping.items():
        text = text.replace(token, value)
    return text

Regexes cover structured identifiers well: emails, phone numbers, card numbers (check the Luhn digit to cut false positives), IBANs, national id formats for the countries you serve. Names and addresses need a named entity recogniser. Microsoft Presidio is a solid open source option that combines both and supports custom recognisers for your own id formats.

Keep the mapping in memory for the request only. If you store it, you have rebuilt the original data with extra steps.

Redact before logging, always

Even if you decide to send raw data to the provider, logs should not contain it. Put redaction inside the logging layer so nobody can forget it:

import logging

class RedactingFilter(logging.Filter):
    def filter(self, record: logging.LogRecord) -> bool:
        if isinstance(record.msg, str):
            record.msg, _ = pseudonymise(record.msg)
        if record.args:
            record.args = tuple(
                pseudonymise(a)[0] if isinstance(a, str) else a for a in record.args
            )
        return True

logging.getLogger("llm").addFilter(RedactingFilter())

Then go one step further and question whether you need text in logs at all. For most operational questions, structured metadata answers them:

{
  "feature": "support_assistant",
  "prompt_version": "v14",
  "model": "large-2025-08",
  "input_tokens": 2311,
  "output_tokens": 412,
  "latency_ms": 2840,
  "tool_calls": ["get_order_status"],
  "finish_reason": "stop",
  "redactions": {"EMAIL": 1, "PHONE": 0},
  "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736"
}

When you do need the text to debug a specific bad answer, store redacted text in a separate store with short retention and named access, not in the general log index.

Traces and eval datasets are logs too

Tracing tools for LLM apps capture prompts and completions by default, and eval datasets are often built by exporting production conversations. Both inherit every piece of personal data in them. Apply the same redaction before export, and treat eval datasets as data with an owner and a retention period. If a user asks for deletion, you need to find their data in these places as well, which is much easier if it was never there.

Provider-side settings

Read the data usage terms for every model provider you use and check, for your account type:

  • Whether API inputs and outputs are used for training. For business API accounts at the major providers, the default is no, but confirm it in writing for your contract.
  • How long inputs and outputs are retained for abuse monitoring, and whether a zero or reduced retention option exists for your use case.
  • Which region processes the data, if you have residency requirements.
  • Whether features you enable (stored conversations, file storage, assistants) keep data longer than plain completions.

Record the answers next to the data flow table. They change, so review them once a year.

Output checks

Models repeat what they see. A summary for an internal dashboard can include a customer's phone number because it was in the ticket. Run the same detectors on outputs that go to other audiences, and either redact or block. For user-facing chat, the main risk is the model revealing data from retrieved context that belongs to someone else; that is an access control problem in retrieval, not something output filters can fully fix.

Common mistakes

Mistake Why it hurts Do this instead
Logging full request bodies at INFO Builds a searchable PII store Log metadata; keep redacted text in a restricted store
Redacting only user input Retrieved docs and tool results carry PII too Redact at the logging and export layer for all text
Storing the pseudonym mapping Defeats the purpose Keep mappings per request, in memory
Regex-only detection for names Low recall on free text Combine regex with an NER-based detector
Eval sets exported raw from production PII spreads to laptops and notebooks Redact on export, set retention and an owner

Key takeaways

  • Map where personal data enters each LLM feature and where it is stored.
  • Send the model only what it needs; pseudonymise identifiers it must reason about.
  • Make redaction part of the logging layer, and prefer structured metadata over raw text.
  • Treat traces and eval datasets with the same rules as logs.
  • Confirm provider retention and training settings for your account, and record them.

References