Skip to content
ReviewRamble

The weekly brief

AI, reviewed, in your inbox.

  • New reviews of tools, models, MCP servers and repos, with the verdict up front
  • Re-checked prices and ratings when something changes
  • New analysis pieces as they are published

Models, agents, MCP servers and the tools actually worth your time.

One email a week. Unsubscribe with the link in any brief. How we use your email

ControlNet

ControlNet is a neural network add-on that lets a text-to-image diffusion model follow a structural guide, such as an edge map, depth map or human pose, in addition to the text prompt.

Also called: control net, spatial conditioning

By Divya Sharma · Checked Nov 15, 2026 · How we review

A prompt alone cannot say exactly where things go. ControlNet adds a second input that pins down the layout. Give it a stick-figure pose and the generated person stands that way; give it the outlines of a building and the image keeps those lines while the prompt decides the style.

How it works

Lvmin Zhang and colleagues introduced ControlNet in 2023. It freezes the original diffusion model and trains a copy of its encoder on the new condition. The copy connects back through zero-initialised convolution layers, so at the start of training it has no effect and cannot damage what the base model already knows. Separate ControlNets exist for edges, depth, segmentation, pose and more, and several can be combined.

Where you meet it

ControlNet is a staple of node-based workflows in ComfyUI and is supported in Diffusers and InvokeAI. Designers use it to keep a product's shape, architects to restyle sketches, and animators to keep a character's pose consistent.

How it differs from image-to-image

Image-to-image starts from a noised copy of the whole picture, so colours and textures carry over. ControlNet carries over only the structure you extract, leaving the look to the prompt. It is also different from a LoRA, which changes what the model draws, such as a style or a character, rather than where it draws it.

Related terms

Diffusion model
A diffusion model is a generative model that learns to turn random noise into data, such as an image, by removing a little noise at a time over many steps.
Image-to-image
Image-to-image is the task of generating a new image from an existing one plus a prompt, keeping some of the original's structure while changing style, detail or content.
Text-to-image
Text-to-image is the task of generating a new picture from a written description, called a prompt. Most current text-to-image models are diffusion models conditioned on the text.
LoRA
LoRA (low-rank adaptation) is a fine-tuning method that freezes a model's original weights and trains small low-rank matrices added to its layers, so far fewer parameters change and much less GPU memory is needed.
Inpainting and outpainting
Inpainting is filling a masked region of an image with new generated content that fits the rest of the picture. Outpainting is the same idea applied outside the edges, extending the image beyond its frame.
Guidance scale (CFG)
The guidance scale, or CFG scale, is a setting in diffusion image generators that controls how strongly the output follows the prompt. Higher values follow the prompt more closely but reduce variety.

Sources