General-purpose 3D generators are good at producing objects that look right at a glance and wrong in the details that matter. Hugging Face has published a step-by-step guide to fixing that for TRELLIS.2, a popular open model that turns a single image into a textured 3D mesh, using small add-on weights trained on your own objects. Its test case is shoes: the base model got the uppers about right and the soles wrong.
The method is LoRA, short for low-rank adaptation, which trains small adapter matrices while leaving the original model frozen. The interesting decision is where to put them. TRELLIS.2 works in stages: it removes the background, encodes the image with DINOv3, predicts a coarse 3D occupancy grid, generates detailed geometry and finally adds materials. The authors added adapters only to the two stages that control shape, about 9.3 million trainable parameters in total, and froze the materials stage. Their training scans had lighting baked into the colours, so training on them could teach the model to paint shadows on as surface colour and to forget what it already did well.
The data was small. From the 1,030 objects in Google's Scanned Objects collection, they took 188 shoes, held back nine for testing and trained on 179, rendering 16 views of each in Blender. Preprocessing took about an hour on an A100. Each adapter was trained on one A100 80 GB through Hugging Face Jobs: 7.2 hours for the coarse-structure stage and 3.6 hours for the shape stage. Docker images and commands are provided for running the same thing on your own GPU.
The results on the nine held-out shoes are large. Mean F-score, a measure of how closely the generated surface matches the scan, rose from 0.276 to 0.803. Chamfer distance, where lower is better, fell from 0.174 to 0.033, and normal consistency rose from 0.530 to 0.804. The adapted model beat the base on every one of the nine. For teams already using Hugging Face for models and datasets, this is a practical recipe for any narrow product category, from furniture to parts.
The limits are clear. Nine test objects is a very small sample, and the training curves for the shape stage barely moved, so the authors rely on the held-out comparisons rather than loss. Results are reported for one category only, and the guide does not show whether shoe adapters harm other objects. The comparison also involved a rotation adjustment to the base output before scoring.