What is Cog?
Cog packages a machine-learning model as a Docker container. You describe the environment in cog.yaml (Python version, system packages, requirements, whether it needs a GPU) and write a Python class whose setup() loads the weights and whose run() takes typed inputs. Cog picks a compatible CUDA, cuDNN and framework base image, builds the container, generates an OpenAPI schema from your types and serves it through coglet, a Rust HTTP server, on port 5000.
It is made by Replicate, and it is the format Replicate's hosted platform runs: cog push sends an image to Replicate's registry or to any OCI registry. The same image runs anywhere Docker does, so you are not tied to Replicate. Replicate itself was bought by Cloudflare, a deal that closed on 1 December 2025.
The project is written mainly in Go, licensed Apache 2.0 and has about 9,500 stars. It ships often: v0.20.0 in May 2026 renamed cog predict to cog run and predict() to run(), v0.22.0 in August changed local servers to bind only to 127.0.0.1, and v0.23.1 on 20 October 2026 is the latest release.
Is Cog worth it?
Cog removes the part of shipping a model that researchers hate most: matching CUDA, cuDNN, PyTorch and Python versions in a Dockerfile until the image finally runs on a GPU. You list the Python version and packages in cog.yaml, write a class with setup() and run(), and cog build produces an image that answers POST /predictions with inputs checked against an OpenAPI schema generated from your type hints.
It is a packaging tool, not a fast serving engine. vLLM or SGLang batch requests across users to squeeze throughput from a GPU running a language model; Cog wraps whatever code you write, and runs one prediction at a time unless you raise the limit with @cog.concurrent. That suits image, audio and video models and custom pipelines more than chat models with many simultaneous users; when every slot is busy the server answers 409 Conflict. Recent releases added Server-Sent Event streaming in June 2026, a browser playground in August and opt-in OpenTelemetry tracing in September.
The catch: the HTTP docs describe no authentication for the prediction API. Version 0.22.0 made cog serve bind to 127.0.0.1 rather than every network interface, a sensible fix, but a built image still serves on port 5000 to whoever can reach it, so a proxy with auth belongs in front of any shared deployment. The interface also churns: v0.20.0 renamed predict to run (the old names still work as aliases), the cog weights commands print a warning that they are experimental, and a bug fixed only in v0.23.1 could mark a prediction as succeeded after its output upload failed.
Ownership is the long-term question. Cog exists to feed Replicate, which Cloudflare now owns and is moving onto its own infrastructure. The licence is Apache 2.0, so the tool would survive a change of heart, but its roadmap follows Replicate's needs.
Our take: Cog is to a model what a shipping container is to cargo. It will not make the contents faster, but nobody at the other end needs to know what is inside to move it.
Where it earns its place
A computer-vision researcher with a working repo and no DevOps help
Publish a model so others can call it through an API
Write cog.yaml and a run.py around the existing inference code, test it with cog run -i image=@input.jpg, then cog push to Replicate. Cog is free; Replicate bills per second of hardware while the model runs.
A platform team at a media company running models on its own Kubernetes cluster
Give every model the same container shape and HTTP contract
Require each model team to ship a Cog image, push it to the internal OCI registry with cog push --json for digest-pinned references, and put an authenticating gateway in front of port 5000. No licence cost; the saving is not maintaining one Dockerfile per model.
A two-person startup selling an audio-cleanup API
Stream progress and partial output to the customer's app
Mark run() with @cog.streaming and have clients request text/event-stream to receive start, output, log and completed events, with reconnecting clients able to replay in-flight events. Host it on Replicate or a rented GPU.
Before you commit
- Setup
- Install Docker (plus Buildx with Docker Engine), then brew install replicate/tap/cog on macOS or the install script on Linux; Windows 11 needs WSL 2. A first image is cog.yaml, run.py and cog build; cog doctor warns when the chosen PyTorch and CUDA wheels have no kernels for your GPU.
- Integrations
- Docker and any OCI registry; Replicate's registry and hosting; OpenAPI schema for clients; Server-Sent Events streaming; webhooks; OTLP export for OpenTelemetry traces; a local playground web UI.
- Data and privacy
- Cog itself sends nothing anywhere; data stays wherever you run the image. The HTTP API docs describe no authentication, and local servers bind to 127.0.0.1 by default since v0.22.0. Tracing is off by default. Apache 2.0.
- Cost at scale
- Cog is free; the cost is the GPU. On Replicate that is per-second hardware billing; self-hosted, it is your own instances, and default single-prediction concurrency means more replicas for high request volumes.
- Limits
- One prediction at a time unless @cog.concurrent(max=N) is set; JSON responses do not support polling for status; training endpoints do not support SSE; cog weights is experimental; about 100 open issues and pull requests.
What’s good
- Picks compatible CUDA, cuDNN and framework versions for you
- OpenAPI schema and input validation generated from Python type hints
- Same image runs on Replicate, your own Docker hosts or any OCI registry
- Active releases: streaming, playground and OpenTelemetry tracing in 2026
What’s not
- No authentication documented for the prediction API
- One prediction at a time by default; not a batching LLM server
- Interface churn: predict renamed to run in v0.20.0; weights commands experimental
- No native Windows support; needs WSL 2
Best for
ML engineers and researchers who need a model to run reliably in a GPU container, especially those publishing to Replicate.
Cog alternatives
Cog: common questions
- Is Cog worth it?
- Cog is the shortest road from a research repo to a container with a typed HTTP API, and the obvious choice if Replicate is where you will host, though its API documents no authentication of its own and its interface is still moving. We rate it 4.1 out of 5.
- How much does Cog cost?
- Free and open source under Apache 2.0. Running images on Replicate is billed separately on Replicate's per-second hardware or per-output pricing.
- What are the downsides of Cog?
- No authentication documented for the prediction API; one prediction at a time by default; not a batching LLM server; interface churn: predict renamed to run in v0.20.0; weights commands experimental; no native Windows support; needs WSL 2.
- Who is Cog best for?
- ML engineers and researchers who need a model to run reliably in a GPU container, especially those publishing to Replicate.
- How much does Cog cost for a team or in production?
- Cog is free; the cost is the GPU. On Replicate that is per-second hardware billing; self-hosted, it is your own instances, and default single-prediction concurrency means more replicas for high request volumes.
- Is Cog safe for company data?
- Cog itself sends nothing anywhere; data stays wherever you run the image. The HTTP API docs describe no authentication, and local servers bind to 127.0.0.1 by default since v0.22.0. Tracing is off by default. Apache 2.0.
- What are the best Cog alternatives?
- The closest alternatives we have reviewed are vLLM and SGLang.
Answers checked Nov 14, 2026.
Make Cog? Show this rating on your site
