Why does a RAG system copy your documents?
To answer quickly, the system cannot open every source file at question time. It reads documents ahead of time, extracts the text, splits it into chunks, converts each chunk into an embedding and stores both in an index. When someone asks a question, the closest chunks are retrieved and sent to the model with the question. Each step creates a copy in a new place, sometimes with a different owner, location and retention period. Some products avoid one of these copies: Microsoft says the Azure OpenAI 'on your data' feature leaves data in the data source and location you designate and does not create a duplicate data store. Ask each vendor which copies its design creates.
Are embeddings sensitive?
Embeddings look like long lists of numbers, which can make them seem harmless. Published research shows they can reveal the original text. A 2023 paper at EMNLP, a peer-reviewed natural language processing conference, described a method that recovered 92% of 32-token text inputs exactly from their embeddings, and it recovered full names from embeddings of clinical notes. Treat a vector store with the same classification as the documents it was built from: the same access controls, the same location rules and the same deletion obligations.
Which copies does a typical system create?
Walk through the pipeline from the source system to the answer on screen and list every place content lands, including places outside the main application such as monitoring tools. Your design may add or remove copies, and products differ, so confirm each one with whoever built or supplies it. The list below covers the copies that appear most often. Record each one in your data flow, with the team or supplier responsible for it.
- Sync cache or staging store: raw files pulled from SharePoint, file shares or other systems.
- Extracted text and chunks: plain text, often with metadata such as title, author and permissions.
- Vector index: embeddings plus the chunk text returned at query time.
- Conversation history: questions and answers, which quote retrieved passages.
- Logs: application logs, provider abuse-monitoring logs and observability tools.
- Backups and exports: snapshots of each store above, and evaluation datasets built from real questions.
Where do provider logs and stored conversations sit?
Hosted model providers may keep prompts, which include the retrieved chunks, for a period. OpenAI says its abuse monitoring logs may contain prompts and responses and are retained for up to 30 days by default, unless longer retention is required by law, and eligible customers can be approved for controls that exclude their content. Microsoft says some Azure features, such as the Responses API, Assistants threads and stored completions, store data in your resource within the same geography until you delete it. These copies sit outside your vector store, so a deletion process that only touches the index will miss them.
How do you delete a document from every copy?
Deletion needs to flow downstream automatically. When a source document is deleted or withdrawn, the sync should remove its chunks and embeddings from the index, and the system should record when that happened. Conversation history and logs that quoted the document are harder: decide whether they are kept for audit or purged on a schedule, and write that into your retention policy. For personal information, APP 11 requires covered organisations to take reasonable steps to destroy or de-identify information they no longer need, so check whether your indexes and logs keep it beyond that point. Our guide on removing retired documents covers retrieval withdrawal in detail.
How do you present this to a security reviewer?
Produce a one-page data flow: source systems on the left, each copy in the middle, and the model and users on the right. For each copy, record the location (country and service), who can access it, the retention period, encryption and how deletion reaches it. Attach provider statements for the hosted parts. This page covers many of the data-handling questions in a vendor security questionnaire and gives your privacy officer the inputs for an assessment. Hypothetical example: a system with three source collections, one vector index, conversation history kept for 90 days and provider logs kept for 30 days would show each of those stores with its own retention line; the figures are illustrative.