Sovereign AI: running RAG on regulated data without OpenAI exposure

The RAG promise — and the leakage problem

Retrieval-augmented generation (RAG) is the dominant pattern for enterprise AI: take a user query, retrieve the most relevant documents from a private corpus, and feed them to a large language model alongside the query. The model produces an answer grounded in your data.

For regulated organisations — healthcare, government, defence, legal — this is the only AI pattern that's actually deployable. Off-the-shelf LLMs hallucinate; RAG-grounded systems cite their sources. The problem: most RAG stacks involve sending your private corpus — at embedding time, query time, or both — to OpenAI, Anthropic, Google, or Azure OpenAI.

For data covered by PIPEDA, GDPR, HIPAA, or PHIPA, that transfer is the breach. The architecture has to keep the data inside the sovereignty envelope from end to end.

Where exposure happens in a typical RAG stack

A standard production RAG pipeline has three exposure points:

1. Embedding at ingest. Each document is chunked and embedded into vectors. If you use OpenAI's embedding API, every document chunk is transmitted to OpenAI. Even though OpenAI's terms now exclude embedding traffic from model training (as of late 2023), the data is still in their infrastructure and reachable by US legal process under the CLOUD Act.
2. Query embedding. Every user query is also embedded — same exposure.
3. Generation. The retrieved chunks are sent to the LLM along with the query. This is the most consequential leak: actual regulated content is in the LLM provider's environment.

Most "private RAG" deployments solve points 1 and 2 by running embeddings locally; they then solve point 3 by routing to Azure OpenAI in a "private" deployment. But Azure OpenAI is still Azure — still US-controlled, still in CLOUD Act reach.

What sovereign RAG actually requires

For a truly sovereign RAG, all three exposure points must be addressed:

• Embeddings computed on customer-controlled hardware
• Vector storage in customer-controlled databases (or sovereign storage)
• Generation either on customer-controlled hardware, or routed only to LLM providers whose entire operational plane is in the customer's sovereignty envelope

How SkyeGXU handles embeddings

SkyeGXU is SkyeConnex's sovereign on-prem AI compute subsystem. It exposes an OpenAI-compatible /v1/embeddings API — so existing RAG stacks can swap their endpoint and inherit sovereignty with zero code change. Underneath, embeddings run on customer GPUs via DirectML (Windows) or ROCm/CUDA (Linux planned), using ONNX-format open models: all-MiniLM-L6-v2 (384-dim) at 1.0, with pluggable support for Phi-mini, BGE, and equivalents.

The model lives on customer infrastructure. The documents being embedded live on customer infrastructure. The vectors live on customer infrastructure. There is no point in the embedding flow where data crosses out of the customer's network.

Coordination — model selection, scheduling, billing — happens through a Coordinator service that talks to app.skyeconnex.com only for licensing heartbeats (60-min cadence with 24-hour grace). Customer data never leaves the customer's network.

The Barrista assistant: end-to-end

On top of SkyeGXU, SkyeConnex ships an in-app AI assistant called Barrista. It answers natural-language questions grounded in the user's own corpus — documents stored in CloudRAID, embedded by SkyeGXU, retrieved via pgvector. On Sovereign tier, the entire path from document upload to answered question runs without data leaving the sovereignty envelope.

This is the only sovereign-storage product on the market that ships a turnkey on-prem embedding service. Healthcare, defence, and legal customers cite it as the capability that lets them deploy AI at all.

What about the generation step?

For generation, customers have three options:

1. Self-host an open-weights model (Llama 3, Mistral, Qwen). Highest sovereignty, highest infrastructure cost.
2. Use a sovereign LLM provider (Mistral on European infrastructure, etc.). High sovereignty if the provider's entire operational plane is in your jurisdiction.
3. Route only de-identified prompts to a hyperscaler LLM, using SkyeGXU embeddings to find the relevant chunks and stripping PII before prompt assembly. Lowest infrastructure cost, partial sovereignty.

SkyeConnex doesn't dictate the choice. The architecture provides the sovereign embedding tier; you choose the generation tier that fits your residency mandate and budget.

The procurement clarification

If procurement asks "do you have AI features," the honest answer with SkyeConnex is: yes, and on Sovereign tier the entire path runs without data ever leaving your network. That answer changes the conversation. Book a briefing to see SkyeGXU running on your own GPU, embedding your own documents, in 30 minutes.

---

Published April 30, 2026 · Written by SkyeConnex Inc. · More from the SkyeConnex blog