Skip to content
ReviewRamble

The weekly brief

AI, reviewed, in your inbox.

  • New reviews of tools, models, MCP servers and repos, with the verdict up front
  • Re-checked prices and ratings when something changes
  • New analysis pieces as they are published

Models, agents, MCP servers and the tools actually worth your time.

One email a week. Unsubscribe with the link in any brief. How we use your email

GPU cloud

A GPU cloud is a service that rents out GPU machines by the hour or second for AI training, fine-tuning and inference. Specialist providers, sometimes called neoclouds, focus on GPUs and often charge less than the large general clouds.

Also called: neocloud, GPU rental, GPU cloud provider

By Divya Sharma · Checked Nov 15, 2026 · How we review

Buying data-centre GPUs is expensive and slow, so most teams rent. The large clouds (AWS, Google Cloud, Azure) offer GPU instances alongside everything else. A newer group of providers, often called neoclouds, offer little besides GPUs. RunPod is one example: it offers on-demand GPU instances billed by the second, in both certified data centres and a vetted community network of individual providers.

What you get

Usually a virtual machine or container with one or more GPUs, a choice of GPU models, attached storage and SSH or notebook access. You pick a template or your own container image and run whatever you like: fine-tuning jobs, a vLLM server, image generation. For multi-GPU work, check the interconnect, since tensor parallelism needs fast links between cards.

Ways to pay

On-demand instances can be started and stopped at will. Reserved capacity is cheaper per hour but commits you for months. Spot or interruptible instances are cheapest but can be taken back at short notice, so save checkpoints. Serverless GPU platforms such as Modal bill only while your code runs and scale to zero. Our GPU calculator compares hourly prices for the cards a model needs.

GPU cloud versus inference provider

A GPU cloud sells compute and you run the software. An inference provider sells model output per token and runs everything. Renting GPUs makes sense when you need a custom or fine-tuned model, steady high volume, or full control of the data path.

What to check

Availability of the GPU you want, egress and storage fees, where the data centre is for data residency, and the security and compliance commitments of the provider and, for community hardware, of whoever owns the machine.

Related terms

GPU
A GPU (graphics processing unit) is a processor with many small cores built to run many calculations in parallel. Its speed at matrix maths makes it the standard hardware for training and running AI models.
Serverless GPU
Serverless GPU is a cloud model where you deploy code or a model and the platform starts GPU workers only when requests arrive, billing for the time they run and scaling to zero when idle.
Inference provider
An inference provider is a company that runs AI models on its own hardware and sells access to them through an API, usually charged per token. You send prompts and get outputs without managing GPUs yourself.
Self-hosting (AI models)
Self-hosting an AI model means running it on hardware you control, such as your own servers, rented GPUs or a laptop, instead of calling a provider's API. You choose the model and keep the data, but you run the infrastructure.
VRAM
VRAM is the memory on a graphics card. For AI work it holds the model weights, the KV cache and working data, and whether a model fits in VRAM usually decides which GPU you need to run it.

Sources