MGM2 as a Multimodal Foundation Model for Microbial Community States and Responses
Code · Tutorial · Preprint · Feature atlas · Interpretation agent
Microbial General Model 2 (MGM2) is a multimodal foundation model that represents microbial communities through sequence identity, quantitative abundance and community semantics. Pretrained on a corpus of 1.82 million microbiomes, it produces community-level representations and contextual microbial features for studying community states and responses to intervention.
This bundle brings together four encoder scales, their pretraining heads,
sequence and taxonomic embedding banks, and a 4,096-feature sparse autoencoder.
MGM2 v2.0 trains community representations through CLS-conditioned abundance
reconstruction and semantic alignment. The matching release is available at
revision v2.1.1.
The v2.1.1 code release standardizes code comments and docstrings in English. The pretrained MGM2 weights, embedding resources and SAE are unchanged.
Checkpoints
Browse the complete MGM2 v2.1.1 model bundle, or open an individual checkpoint below.
| Model | Encoder parameters | Hidden dimension | Directory |
|---|---|---|---|
| MGM2-Small | 4,076,352 | 224 | models/small |
| MGM2-Medium | 8,096,640 | 320 | models/medium |
| MGM2-Large | 15,083,520 | 384 | models/large |
| MGM2-XLarge | 32,838,656 | 512 | models/xlarge |
Each model directory contains model.safetensors and config.json. Parameters
refer to the community encoder; configurations record architecture, selection
epoch and source-checkpoint SHA256. Each checkpoint is selected by its full
pretraining validation loss.
Usage
Install the accompanying package from PyPI:
python -m pip install 'microformer-mgm2[tables,hub]'
See the installation guide
for source installation and GPU setup.
The notebook extra adds JupyterLab and the interactive tutorial.
With this model bundle as the working directory, the following example
embeds a community constructed from three OTUs in the sequence embedding bank:
import torch
from mgm2 import MGM2, EmbeddingBank, prepare_sample, collate_samples
model = MGM2.from_pretrained("models/xlarge", device="cpu")
with EmbeddingBank("embeddings/otus_99_embeddings.h5") as bank:
sample = prepare_sample(bank.ids[:3], [20, 5, 1], bank)
batch = collate_samples([sample], device="cpu")
with torch.inference_mode():
outputs = model(batch)
print(outputs["cls"].shape) # (1, 512): community representation
print(outputs["hidden"].shape) # (1, 4, 512): CLS followed by three OTUs
The loader enters evaluation mode. Outputs include CLS, contextual token states,
mean-pooled OTU states and a 4,096-dimensional semantic projection. The
code README
describes abundance-table embedding through mgm2-embed, including
sample-level sequence and taxonomy resolution counts.
Encode your own sequences
Install the code package with .[ntv3] to encode representative OTU or ASV
sequences using the legacy
NTv3_650M_pre:
mgm2-encode-fasta --fasta representative_sequences.fasta \
--output outputs/asv_ntv3.h5 --device cuda:0 --trust-remote-code
Use the resulting bank with mgm2-embed --embeddings outputs/asv_ntv3.h5.
FASTA IDs must match abundance-table feature IDs. See the
sequence guide
for model access and local checkpoints, or start with the
sequence notebook.
Sparse features and interpretation
The SAE in sae/otu_l10_k64 maps MGM2-XLarge layer-10 OTU states into 4,096
sparse features. It uses BatchTopK training with an average activation budget
of 64 and a fixed threshold for inference. Using the outputs above:
from mgm2 import load_sae
sae = load_sae("sae/otu_l10_k64", device="cpu")
valid = (batch["input_ids"] >= 0) & batch["attention_mask"].bool()
with torch.inference_mode():
features = sae.encode(outputs["hidden"][valid])
print(features.shape) # (3, 4096): one sparse feature vector per OTU
The interpretation agent connects feature-level measurements with literature evidence through interpretation, on-demand research and independent review. Reusable evidence caches and compact structured handoffs preserve source references while reducing repeated context transfer between roles.
Data and pretraining
The corpus contains 1,821,291 communities: 1,639,420 for training and 181,871 for project-disjoint pretraining validation. The HDF5 resources contain NTv3-derived embeddings for 225,066 OTUs and taxonomic fallback embeddings for 5,087 groups.
Inputs use up to 768 observed OTUs per community. Each complete sample is
normalized before top-768 selection; positive values below 1e-4 are floored to
1e-4 and transformed with clip(log10(x) + 2, -2, 2). Retained values preserve
this transformed scale after selection and flooring. Per-community quantiles
define five positive-abundance bins, indexed 0–4.
Masked-abundance classification, regression for the highest two bins and Qwen3-Embedding-8B semantic alignment jointly train the model. Reconstruction combines the CLS state with the queried OTU's static identity. Inference uses the trained MGM2 encoder and its projection head.
Applications
MGM2 supports microbiome representation research, including hierarchical biome classification, transplantation-associated community and response modelling, drug perturbation prediction, and sparse-feature interpretation. The code repository includes aggregate article benchmark tables and their evaluation definitions.
Acknowledgements
We thank the authors and maintainers of NTv3, Qwen3 Embedding, PyTorch, Hugging Face Transformers and Hugging Face Hub.
Citation
The MGM2 preprint is available on bioRxiv:
Zhang, H. et al. MGM2 as a Multimodal Foundation Model for Microbial Community States and Responses. bioRxiv (2026).
@article{zhang2026mgm2,
title = {{MGM2} as a Multimodal Foundation Model for Microbial Community States and Responses},
author = {Zhang, Haohong and Zhang, Yuli and Qi, Yuxuan and Liu, Tianao and Zhao, Linfeng and Yang, Ronghua and Ning, Kang},
journal = {bioRxiv},
year = {2026},
doi = {10.64898/2026.07.20.739063},
url = {https://doi.org/10.64898/2026.07.20.739063}
}