Skip to content
ReviewRamble

The weekly brief

AI, reviewed, in your inbox.

  • New reviews of tools, models, MCP servers and repos, with the verdict up front
  • Re-checked prices and ratings when something changes
  • New analysis pieces as they are published

Models, agents, MCP servers and the tools actually worth your time.

One email a week. Unsubscribe with the link in any brief. How we use your email

Self-hosting (AI models)

Self-hosting an AI model means running it on hardware you control, such as your own servers, rented GPUs or a laptop, instead of calling a provider's API. You choose the model and keep the data, but you run the infrastructure.

Also called: self-hosted, self-hosted LLM, on-premises AI

By Divya Sharma · Checked Nov 15, 2026 · How we review

With a hosted API, a provider runs the model and you send it your data. Self-hosting reverses that: you download model weights, usually an open-weight model, and serve them yourself.

Why teams do it

The main reasons are control and privacy. Prompts and outputs never leave your environment, which helps with data residency and with data you are not allowed to send to a third party. You also pin the exact model version, so it does not change under you, and you can fine-tune it. At high, steady volume, renting or owning GPUs can cost less than paying per token.

How it is done

On a single machine, Ollama, llama.cpp and LM Studio run local LLMs and offer an API on localhost. For production traffic, servers such as vLLM and SGLang add continuous batching and an OpenAI-compatible API, so existing client code keeps working. The hardware can be your own or rented from a GPU cloud. The GPU calculator estimates what a model needs.

The costs

You own uptime, scaling, security patches, monitoring and capacity planning. GPUs sit idle and still cost money when traffic is low. The best open models may trail the leading closed ones on some tasks, and some closed models are not available to self-host at all. Each model's licence also sets terms on use.

Middle ground

Dedicated deployments from inference providers, or managed platforms like Modal, run a model of your choice on reserved hardware without you managing servers. They give some of the control at less operational cost, but the data still leaves your network.

Related terms

Model serving
Model serving is running a trained model as a service that applications call over a network, with software that loads the weights onto GPUs, queues and batches requests, manages memory and returns results.
Data residency
Data residency is a control that keeps a customer's data stored, and often processed, in a chosen geographic region such as the EU, so a team can meet its own legal, contractual or customer rules on where data lives.
Local LLM
A local LLM is a language model that runs on your own computer, phone or server instead of a provider's cloud API, so prompts and data stay on your hardware and it works without an internet connection.
Open-weight model
An open-weight model is a model whose trained weights are published for anyone to download and run, often without the training code or full training data, so it is not necessarily open source.
GPU cloud
A GPU cloud is a service that rents out GPU machines by the hour or second for AI training, fine-tuning and inference. Specialist providers, sometimes called neoclouds, focus on GPUs and often charge less than the large general clouds.

Sources