Your data does not need to visit the AI. The AI can move in.

Illustration: AI-generated, art-directed by the author.

Remember how the Samsung story ends. Not with the ban.

In 2023, after engineers pasted confidential code into ChatGPT, Samsung banned public AI tools on company devices. That part everyone quotes. Six months later came the part almost nobody quotes: Samsung unveiled Gauss, its own in-house generative AI, and rolled it out to employees (CNBC, November 2023). The company that wrote the most famous AI ban in corporate history concluded that the answer was not "no AI." It was "our AI."

The rest of the market followed the same arc. Cisco has asked the same question every year, counting organizations with a generative-AI ban in force: 27% in its 2024 benchmark, 28% in 2025, and just 7% by the 2026 edition. That is a one-year fall, not a slow drift. (The larger 75% figure from the shelfware story counted organizations banning *or considering* a ban; Cisco counts bans actually in force, which is why the two numbers sit so far apart.) The study does not editorialize about why, but the reason is the least surprising one in this whole subject: blanket bans proved impossible to enforce. I wrote about that failure in shadow AI. This one is about the thing that works instead.

What "bring the AI to your data" actually means

Strip the vendor language away and a private AI setup is three pieces:

The model is a rented brain. It reasons, writes, and answers. The critical thing to understand is that in an enterprise setup the model is stateless: it does not remember your questions, and it does not absorb your documents into itself. It processes what you hand it and returns an answer.

The retrieval layer is a librarian. When someone asks "what did we agree with the Riverside client about payment terms," the system does not hope the model somehow knows. It searches your document store, finds the three relevant passages, hands them to the model, and says: answer from these. That technique has a clumsy name, retrieval-augmented generation, but the idea is just open-book exam instead of memory test.

The data store is where your knowledge lives, and this is where the term "vector database" comes in. Here is the whole concept in plain language: a vector database stores your documents as coordinates on a giant map of meaning. Two sentences that mean similar things sit close together on the map, even if they share no words. "The client is unhappy about the invoice" and "Riverside disputed the billing" land near each other. So when a question comes in, the system finds neighbors by meaning, not by keyword matching. That is the entire trick. It is why the assistant can find the right passage in fractions of a second, even across millions of documents.

Put together: your documents stay in your store, the librarian fetches only what a question needs, and the rented brain answers from what it was handed. Nothing about this requires your data to live inside someone else's product.

This is not exotic anymore. In a Snowflake-sponsored survey of organizations already using generative AI, 71% were augmenting models with their own data through retrieval. Among adopters, this is simply how it is done.

What the providers actually promise

The major platforms have converged on the same contractual core, and it is worth knowing it is contract, not vibes:

• AWS Bedrock: your inputs and outputs "are never shared with model providers or used to train base models."
• Microsoft Azure OpenAI: prompts and outputs "are NOT available to OpenAI" and are not used to train models.
• Google Vertex: customer data is not used to train foundation models by default; the models are frozen.
• Anthropic's commercial terms: no training on customer content, flat out.

Two honest footnotes belong next to that list. First, "not trained on" is not the same as "never seen by anyone": Azure, for example, runs abuse monitoring by default, meaning flagged prompts can be reviewed by authorized humans unless you request the modified tier. Second, defaults matter: some platforms cache inputs briefly for performance unless you configure zero retention. Neither footnote is a scandal. Both are the kind of thing your rollout should check on purpose rather than discover later.

Turn on company AI and the first thing it audits is who could already see what.

Now the part vendors will not lead with

A vector database is not magic security, and pretending otherwise is how the next generation of leaks happens.

Three things the research says out loud:

Embeddings are the text. A 2023 Cornell paper showed that short text (32-token inputs) can be reconstructed from its embeddings with 92% exact recovery; the same method also recovered full personal names from clinical notes. Those coordinates on the meaning map are not anonymized data. Treat the vector store with the same seriousness as the documents themselves.

Retrieval can be attacked. Research on enterprise RAG systems has demonstrated prompt injection through planted documents and leakage through retrieval caches. If a document can talk to your AI, a malicious document can too.

And the failure you will actually meet is neither of those. The typical real-world incident in corporate AI search is mundane: the assistant surfaces a document the employee technically had access to all along, through some overshared folder or ancient sharing link, and now it is one question away instead of buried. The AI did not break your permissions. It revealed that your permissions were already broken.

Which is why the real safeguarding checklist is boring, and boring is a compliment in security:

1. 01Mirror your permissions into retrieval. The librarian must check who is asking before fetching. If access control does not travel with the documents into the AI layer, you have built a leak with a chat interface.
2. 02Log every query and every retrieval. You cannot coach or investigate what you cannot see. This is also, quietly, the end of shadow AI: sanctioned convenience plus visibility.
3. 03Clean the house before the housewarming. An AI rollout is the best permissions audit your company will ever run. Fix the overshared folders first.
4. 04Set retention deliberately. Zero where you can, short where you cannot, documented either way.
5. 05Pilot one workflow with a number attached. Same rule as every AI rollout: pre-registered success metric, a date, and a public kill switch.

The AI did not break your permissions. It revealed they were already broken.

What it costs, roughly

For scale: the big off-the-shelf suites run about $20 to $30 per user per month before you customize anything. A private retrieval deployment for a mid-sized company is not a data center project anymore; it is a managed cloud project, typically standing on the same platforms listed above, sized to your document volume and seat count. The honest cost driver is rarely the technology. It is the permissions cleanup and the training, the same two line items every AI project tries to skip.

The difference between the companies that banned AI and the companies that built something worth using is not budget. It is that one group treated "where does our data go" as a reason to stop, and the other treated it as the first requirement to engineer for. That is the whole argument for the specialist: a generic assistant handed to everybody cannot answer that question per department, because nobody scoped it to a department in the first place.

So here is the test, and it costs nothing to run: if your team could ask an assistant that already knows your documents, inside your walls, would anyone still be pasting your contracts into a public chatbot at midnight? The honest answer tells you whether you have a data problem or a product problem.