Skip to content
ReviewRamble

The weekly brief

AI, reviewed, in your inbox.

  • New reviews of tools, models, MCP servers and repos, with the verdict up front
  • Re-checked prices and ratings when something changes
  • New analysis pieces as they are published

Models, agents, MCP servers and the tools actually worth your time.

One email a week. Unsubscribe with the link in any brief. How we use your email

SceneMosaic

Research code that turns one photo of a room into six editable, physics-checked 3D scene variants, using SAM 3D and VLM agents.

Reviewed by ReviewRamble editors · Updated

View on GitHub Open source

Share this page:XLinkedInFacebookRedditWhatsAppEmail

Our rating

3.3/5

SceneMosaic is a clever way to get many physically valid room layouts out of a single photo, but it is a one-push research release that needs a 32 GB GPU, two Conda environments and a paid model API.

What is SceneMosaic?

SceneMosaic is the code release for a paper from The University of Hong Kong and the University of Electronic Science and Technology of China. Given a single image of an indoor space, it segments the objects with SAM3, rebuilds each one as a mesh with SAM 3D Objects, works out which objects rest on or sit inside which, and then lets vision-language model agents repair and vary the layout. The output is a set of representative scenes as .glb files, ready for a simulator.

The project page puts a base scene at about 8 minutes and each further layout variant at about 2 minutes. The repository was created on 3 September 2026, is MIT licensed and has about 110 stars and two commits; there are no tagged releases and the paper has no arXiv listing, only a PDF on the project page.

Is SceneMosaic worth it?

SceneMosaic sits between two approaches that both disappoint people building training worlds for robots. Agentic text-to-3D systems produce careful scenes but, by the authors' account, often need hours of tool calls; image-to-3D models are fast but leave furniture floating, colliding or clipping through walls. SceneMosaic takes the fast route for a first draft and then spends its agent effort on repair and variety.

The idea that earns attention is how variety is made. The scene is split into local groups around an anchor object, a table with its chairs, a shelf with its books, and agents produce variants of each group in parallel. The groups are then combined in every possible way, invalid combinations are thrown out, and a max-min search keeps a small set of layouts that differ most in position, distance and rotation. A memory stops agents undoing and redoing the same edit. On the project page that yields a base scene in about 8 minutes and each extra variant in about 2.

The weak point is everything around the method. The README asks for Linux, an NVIDIA GPU with at least 32 GB of VRAM (it observed a peak of about 38 GB), approval for two gated Meta model repositories on Hugging Face, and two separate Conda environments because SAM3 and SAM 3D Objects cannot share one. Perception, relation extraction and evolution all call an OpenAI-compatible chat endpoint, so each scene costs API tokens and sends your image to that provider. The project page publishes no benchmark or comparison with named baselines.

The repository has two commits, both from 3 September 2026, and no issues filed. The README warns not to update the pinned SAM3 submodule, since upstream has moved to SAM 3.1. Treat it as a reference implementation to read and reproduce, not a maintained tool.

Where it earns its place

  • A robotics PhD student training a household manipulation policy

    Generate dozens of different kitchen layouts from a handful of photos

    Run the end-to-end script on each photo and keep the six representative .glb variants per scene for the simulator. Each extra variant takes about 2 minutes on the project page's figures; the cost is GPU time and API tokens.

  • A computer-vision lab reproducing scene-generation work

    Compare a new layout method against a hybrid agentic baseline

    Clone with submodules, build both Conda environments and run the four bundled example images. The stage-level resume keeps finished perception and meshes, so only the stage under study needs rerunning.

Before you commit

Setup
Clone with submodules, run two installer scripts for the scenemosaic and sam3 Conda environments, request access to SAM3 and SAM 3D Objects on Hugging Face, download checkpoints, set OPENAI_API_KEY and run the verify command. Expect an afternoon.
Integrations
SAM3, SAM 3D Objects, MoGe depth and DINOv2 checkpoints; any OpenAI-compatible chat endpoint via OPENAI_BASE_URL; optional ambientCG PBR materials; .glb output for simulators.
Data and privacy
Runs locally apart from the agent stages, which send your image and scene descriptions to the OpenAI endpoint or the gateway you configure. The runtime config stores environment variable names, not key values; the README warns never to commit weights or Hugging Face tokens.
Cost at scale
No licence cost. Each scene needs a 32 GB-plus GPU (about 38 GB observed peak) and API calls for perception, relations and evolution; token cost per scene is not published.
Limits
Linux x86-64 only; SAM3 submodule pinned and must not be updated to SAM 3.1; no benchmarks, releases or issue history; failed attempts are kept on disk for inspection.

What’s good

  • Turns one photo into several distinct, simulation-ready .glb scenes
  • Explicit support and containment relations, with gravity and collision checks
  • Stage-level resume reuses perception, meshes and relations after a failure
  • MIT licence; installer scripts and a verify command

What’s not

  • Needs an NVIDIA GPU with at least 32 GB of VRAM
  • Every run calls a paid OpenAI-compatible API and sends it the image
  • Two commits, no releases, no published benchmarks
  • Depends on gated Meta models pinned to an older SAM3

Best for

Embodied-AI and robotics researchers who need many varied, physically plausible indoor scenes and can run large GPU jobs.

SceneMosaic: common questions

Is SceneMosaic worth it?
SceneMosaic is a clever way to get many physically valid room layouts out of a single photo, but it is a one-push research release that needs a 32 GB GPU, two Conda environments and a paid model API. We rate it 3.3 out of 5.
How much does SceneMosaic cost?
Free and open source (MIT). You need an NVIDIA GPU with at least 32 GB of VRAM and pay your own OpenAI-compatible API usage for the agent stages.
What are the downsides of SceneMosaic?
Needs an NVIDIA GPU with at least 32 GB of VRAM; every run calls a paid OpenAI-compatible API and sends it the image; two commits, no releases, no published benchmarks; depends on gated Meta models pinned to an older SAM3.
Who is SceneMosaic best for?
Embodied-AI and robotics researchers who need many varied, physically plausible indoor scenes and can run large GPU jobs.
How much does SceneMosaic cost for a team or in production?
No licence cost. Each scene needs a 32 GB-plus GPU (about 38 GB observed peak) and API calls for perception, relations and evolution; token cost per scene is not published.
Is SceneMosaic safe for company data?
Runs locally apart from the agent stages, which send your image and scene descriptions to the OpenAI endpoint or the gateway you configure. The runtime config stores environment variable names, not key values; the README warns never to commit weights or Hugging Face tokens.

Answers checked Nov 21, 2026.