GPU cloud
A GPU cloud is a service that rents out GPU machines by the hour or second for AI training, fine-tuning and inference. Specialist providers, sometimes called neoclouds, focus on GPUs and often charge less than the large general clouds.
Also called: neocloud, GPU rental, GPU cloud provider
By Divya Sharma · Checked Nov 15, 2026 · How we review
Buying data-centre GPUs is expensive and slow, so most teams rent. The large clouds (AWS, Google Cloud, Azure) offer GPU instances alongside everything else. A newer group of providers, often called neoclouds, offer little besides GPUs. RunPod is one example: it offers on-demand GPU instances billed by the second, in both certified data centres and a vetted community network of individual providers.
What you get
Usually a virtual machine or container with one or more GPUs, a choice of GPU models, attached storage and SSH or notebook access. You pick a template or your own container image and run whatever you like: fine-tuning jobs, a vLLM server, image generation. For multi-GPU work, check the interconnect, since tensor parallelism needs fast links between cards.
Ways to pay
On-demand instances can be started and stopped at will. Reserved capacity is cheaper per hour but commits you for months. Spot or interruptible instances are cheapest but can be taken back at short notice, so save checkpoints. Serverless GPU platforms such as Modal bill only while your code runs and scale to zero. Our GPU calculator compares hourly prices for the cards a model needs.
GPU cloud versus inference provider
A GPU cloud sells compute and you run the software. An inference provider sells model output per token and runs everything. Renting GPUs makes sense when you need a custom or fine-tuned model, steady high volume, or full control of the data path.
What to check
Availability of the GPU you want, egress and storage fees, where the data centre is for data residency, and the security and compliance commitments of the provider and, for community hardware, of whoever owns the machine.
Related terms
- GPU
- A GPU (graphics processing unit) is a processor with many small cores built to run many calculations in parallel. Its speed at matrix maths makes it the standard hardware for training and running AI models.
- Serverless GPU
- Serverless GPU is a cloud model where you deploy code or a model and the platform starts GPU workers only when requests arrive, billing for the time they run and scaling to zero when idle.
- Inference provider
- An inference provider is a company that runs AI models on its own hardware and sells access to them through an API, usually charged per token. You send prompts and get outputs without managing GPUs yourself.
- Self-hosting (AI models)
- Self-hosting an AI model means running it on hardware you control, such as your own servers, rented GPUs or a laptop, instead of calling a provider's API. You choose the model and keep the data, but you run the infrastructure.
- VRAM
- VRAM is the memory on a graphics card. For AI work it holds the model weights, the KV cache and working data, and whether a model fits in VRAM usually decides which GPU you need to run it.