A user uploads a contract to your AI product. The system extracts the text, creates embeddings, sends relevant passages to a model, and saves the answer.
The answer takes three minutes. The data may remain for three years.
I think this deserves a place in the architecture review before the upload feature reaches production.
Deleting the File Is Only One Step
Imagine the user asks your product to summarize a supplier agreement.
The PDF goes into object storage. Extracted text goes into a database. Chunks and embeddings go into a search index. The summary stays in the conversation history.
Maybe your tracing tool captures the prompt and response. Maybe a cache stores the answer.
Now the user clicks Delete. Your application removes the PDF.
What happened to the rest?
The summary may still contain confidential pricing. The trace may contain entire paragraphs. The search index may still return sections of the agreement.
Removing the file does not automatically remove those artifacts.
Decide What You Need to Keep
Before accepting that contract, I would ask why each artifact needs to remain.
Does a one-time summary require a permanent search index? Do we need the full contract in a log to investigate a timeout? How long should the saved answer remain?
Different artifacts can have different retention periods. But each needs a reason, an expiry rule, and an owner. The product also needs to explain whether Delete removes the original file, its derived content, or both.
Include external model providers and tracing services. Their retention and deletion behavior also need to fit the product’s promise.
I made a related point in AI Cost Starts with Architecture: engineering time belongs in the cost discussion. Every additional store creates work to operate, secure, and eventually clean up.
Cheap storage does not remove that responsibility.
Build and Verify the Deletion Path
Keep a reliable connection between the source document and its retained artifacts, within the correct customer account.
Then define what happens when deletion starts. Stop serving the content. Track cleanup across the relevant stores. Retry failures, and make unfinished work visible to the engineering team responsible for it.
Background workers need to respect deletion too. An extraction job should not recreate search records after the user removes the document.
In Evaluating an LLM Is Different From Evaluating a RAG System, I argued that evaluation should focus on the behavior you need and the failure you want to catch.
Apply that here. Upload a test contract, process it, delete it, and inspect the relevant stores. Asking the model about it afterward is insufficient; retrieval can miss content that still exists.
Include backups. Define when retained copies expire and make restoration reapply recorded deletions before serving users. Recovery should not quietly bring deleted contracts back.
Before approving ingestion, understand where the data goes, how long it stays, who deletes it, and how completion is verified.
Every ingestion path needs a corresponding deletion path.
That is part of building the product.