Biology labs have spent heavily on combining tissue images with gene-expression data in the hope that two views of a cell beat one. A paper in Nature Machine Intelligence argues that the most common way of doing this often adds nothing the stronger single source did not already hold, and offers a test to tell when fusion is worth the effort.
The setting is what the authors call compositional foundation models: two pretrained encoders, here UNI-2 for histology slides and Nicheformer for spatial transcriptomics, kept frozen and joined through a small learned interface. This design is popular because paired data, where the same tissue is both imaged and sequenced, is scarce while single-modality data is plentiful. The authors benchmark ten such interfaces, from simple concatenation to CCA, VICReg and a method called CoMM, on lung fibrosis, developing thymus and breast cancer datasets, across five tasks such as classifying tissue niches and predicting a patch's neighbours.
Their new measure, the Synergistic Information Score, asks how much better a fused representation does than the best single one, using simple linear probes. Their theory shows that alignment objectives with frozen encoders mostly recover the linear overlap the two encoders already share, which the authors call a spectral ceiling. In high-resolution data where image and expression match closely, the score sits near zero or below, and fusion brings little. In a scaling test, fine-tuning the gene-expression encoder on as little as 1% of the labelled data improved results quickly and matched or beat the fused models at every data fraction tried.
Fusion does earn its place in some cases. Where resolution differs a lot, as with coarse spot-based sequencing beside fine histology in the thymus data, the score rises and methods built to model interactions, such as CoMM, become competitive. The practical lesson is to measure before building: if a single modality already carries the signal, a second encoder adds parameters and cost without new information.
The findings have clear limits. The ceiling applies to frozen encoders judged by linear probes; end-to-end training in the style of CLIP may behave differently. The study covers spatial histology, with only a pilot on other multi-omics pairings. And the score measures what a linear probe can reach, not every way a richer model might use combined data. As a peer-reviewed paper it carries more weight than a preprint, but it is one group's benchmark on three datasets.