Self-hosting (AI models)
Self-hosting an AI model means running it on hardware you control, such as your own servers, rented GPUs or a laptop, instead of calling a provider's API. You choose the model and keep the data, but you run the infrastructure.
Also called: self-hosted, self-hosted LLM, on-premises AI
By Divya Sharma · Checked Nov 15, 2026 · How we review
With a hosted API, a provider runs the model and you send it your data. Self-hosting reverses that: you download model weights, usually an open-weight model, and serve them yourself.
Why teams do it
The main reasons are control and privacy. Prompts and outputs never leave your environment, which helps with data residency and with data you are not allowed to send to a third party. You also pin the exact model version, so it does not change under you, and you can fine-tune it. At high, steady volume, renting or owning GPUs can cost less than paying per token.
How it is done
On a single machine, Ollama, llama.cpp and LM Studio run local LLMs and offer an API on localhost. For production traffic, servers such as vLLM and SGLang add continuous batching and an OpenAI-compatible API, so existing client code keeps working. The hardware can be your own or rented from a GPU cloud. The GPU calculator estimates what a model needs.
The costs
You own uptime, scaling, security patches, monitoring and capacity planning. GPUs sit idle and still cost money when traffic is low. The best open models may trail the leading closed ones on some tasks, and some closed models are not available to self-host at all. Each model's licence also sets terms on use.
Middle ground
Dedicated deployments from inference providers, or managed platforms like Modal, run a model of your choice on reserved hardware without you managing servers. They give some of the control at less operational cost, but the data still leaves your network.
Related terms
- Model serving
- Model serving is running a trained model as a service that applications call over a network, with software that loads the weights onto GPUs, queues and batches requests, manages memory and returns results.
- Data residency
- Data residency is a control that keeps a customer's data stored, and often processed, in a chosen geographic region such as the EU, so a team can meet its own legal, contractual or customer rules on where data lives.
- Local LLM
- A local LLM is a language model that runs on your own computer, phone or server instead of a provider's cloud API, so prompts and data stay on your hardware and it works without an internet connection.
- Open-weight model
- An open-weight model is a model whose trained weights are published for anyone to download and run, often without the training code or full training data, so it is not necessarily open source.
- GPU cloud
- A GPU cloud is a service that rents out GPU machines by the hour or second for AI training, fine-tuning and inference. Specialist providers, sometimes called neoclouds, focus on GPUs and often charge less than the large general clouds.