PathweysAI Engineer Track · Enterprise Deployment
Private/VPC, On-Prem & Air-Gapped Deployment
Private/VPC, On-Prem & Air-Gapped Deployment
Private, on-premises, and air-gapped deployment means running an AI system inside a customer's own network boundary rather than reaching out to the open internet: private network endpoints so traffic never leaves their environment, self-hosted open models running on their own hardware, or, for the most restrictive clients, a fully isolated environment with no internet access at all. It is the deployment mode compliance-heavy industries specifically require.
Why this matters
Many enterprise clients simply will not adopt an AI system unless their data stays inside their own network boundary, so knowing how to deliver under these constraints is not optional expertise, without it, certain deals do not happen at all.
Private/VPC, on-prem, and air-gapped deployment
When data genuinely cannot leave a boundary, an LLM system has to be deployed inside a cloud private network, fully on-premises, or fully air-gapped with no internet access at all. The core constraint shaping every option is that the model itself must run inside that boundary, which means either a self-hosted open model or a dedicated, genuinely private endpoint offered directly by a vendor. Self-hosted models become mandatory for air-gapped environments, since no outbound vendor call is possible there at all. Private endpoints work for VPC deployments that do not require full self-hosting. And in every case, retrieval, embeddings, and generation all have to stay entirely inside the boundary rather than touching any external service.
VPC, on-prem, and air-gapped deployments all keep data inside a defined boundary, through either self-hosted models or genuinely private endpoints. The real tradeoff is operational burden traded directly for data control.
Self-hosting models in a VPC
Self-hosting an open model inside a VPC grants full data control, typically served through vLLM or a similar engine running behind a gateway with an OpenAI-compatible API. Three concerns shape this path in practice. GPU capacity has to be right-sized for actual throughput, with GPU workers autoscaled to match real demand. Model choice is constrained to open models in a practical size range, since a frontier-scale model is generally not practical to self-host at all. And operations shift entirely onto the team running it, who now own model updates, security patching, and ongoing capacity planning that a managed API would otherwise have handled.
Self-hosting trades real operational burden for genuine data control. A well-chosen open model served through something like vLLM behind your own gateway is the shape of that trade; owning patching and capacity planning is the price.
Worked example
A healthcare client whose compliance team requires that patient data never leave their own network cannot use a managed model API at all, regardless of how it is connected, and instead needs an open model self-hosted entirely on GPUs inside their own VPC, accepting a real quality tradeoff against a frontier model in exchange for a deployment their compliance team can actually approve.
Practice
Design the deployment for a client requiring zero data egress: private endpoints for managed models, a self-hosted open model inside the VPC, or both together. Produce a deployment plan covering network, model, and data flow that proves no request body ever leaves the VPC, along with a note on the resulting quality-versus-cost tradeoff against a public API.
Interview checks
- Why does a 'private endpoint' not automatically mean a managed model is actually running inside your own network?
- What specifically forces a client toward self-hosting rather than any form of private managed endpoint?
- What operational burden does self-hosting an open model in a VPC actually add on top of using a managed API?
Common failure modes
- Assuming a private endpoint means the model itself runs inside a client's VPC, when a managed model still processes data at the provider, the endpoint is private transit, not private processing.
- Underestimating air-gapped operations, offline package management and no automatic updates, as a genuine ongoing burden.
- Never actually verifying egress controls, assuming they hold rather than directly testing that no traffic leaves the boundary.
Resources
- AWS PrivateLink — The mechanism for reaching services, including managed model APIs, over private endpoints rather than the public internet.
- AWS Bedrock User Guide — Covers Bedrock over VPC endpoints directly, the concrete 'managed model reachable inside your VPC' pattern.
Expert signals
Knowing the real difference between private transit and private processing, being able to deliver across the full spectrum from private endpoints to on-premises to fully air-gapped, and actually verifying egress controls directly rather than assuming they hold, is what separates a deployment a compliance team can genuinely approve from one that merely sounds compliant.
Confidence
medium