MGM2 as a Multimodal Foundation Model for Microbial Community States and Responses

PyPI version PyPI downloads Python 3.10+ PyTorch 2.2+ Hugging Face models bioRxiv preprint

Code · Tutorial · Preprint · Feature atlas · Interpretation agent

Microbial General Model 2 (MGM2) is a multimodal foundation model that represents microbial communities through sequence identity, quantitative abundance and community semantics. Pretrained on a corpus of 1.82 million microbiomes, it produces community-level representations and contextual microbial features for studying community states and responses to intervention.

This bundle brings together four encoder scales, their pretraining heads, sequence and taxonomic embedding banks, and a 4,096-feature sparse autoencoder. MGM2 v2.0 trains community representations through CLS-conditioned abundance reconstruction and semantic alignment. The matching release is available at revision v2.1.1.

The v2.1.1 code release standardizes code comments and docstrings in English. The pretrained MGM2 weights, embedding resources and SAE are unchanged.

MGM2 workflow

Checkpoints

Browse the complete MGM2 v2.1.1 model bundle, or open an individual checkpoint below.

Model Encoder parameters Hidden dimension Directory
MGM2-Small 4,076,352 224 models/small
MGM2-Medium 8,096,640 320 models/medium
MGM2-Large 15,083,520 384 models/large
MGM2-XLarge 32,838,656 512 models/xlarge

Each model directory contains model.safetensors and config.json. Parameters refer to the community encoder; configurations record architecture, selection epoch and source-checkpoint SHA256. Each checkpoint is selected by its full pretraining validation loss.

Usage

Install the accompanying package from PyPI:

python -m pip install 'microformer-mgm2[tables,hub]'

See the installation guide for source installation and GPU setup. The notebook extra adds JupyterLab and the interactive tutorial. With this model bundle as the working directory, the following example embeds a community constructed from three OTUs in the sequence embedding bank:

import torch
from mgm2 import MGM2, EmbeddingBank, prepare_sample, collate_samples

model = MGM2.from_pretrained("models/xlarge", device="cpu")
with EmbeddingBank("embeddings/otus_99_embeddings.h5") as bank:
    sample = prepare_sample(bank.ids[:3], [20, 5, 1], bank)

batch = collate_samples([sample], device="cpu")
with torch.inference_mode():
    outputs = model(batch)

print(outputs["cls"].shape)     # (1, 512): community representation
print(outputs["hidden"].shape)  # (1, 4, 512): CLS followed by three OTUs

The loader enters evaluation mode. Outputs include CLS, contextual token states, mean-pooled OTU states and a 4,096-dimensional semantic projection. The code README describes abundance-table embedding through mgm2-embed, including sample-level sequence and taxonomy resolution counts.

Encode your own sequences

Install the code package with .[ntv3] to encode representative OTU or ASV sequences using the legacy NTv3_650M_pre:

mgm2-encode-fasta --fasta representative_sequences.fasta \
  --output outputs/asv_ntv3.h5 --device cuda:0 --trust-remote-code

Use the resulting bank with mgm2-embed --embeddings outputs/asv_ntv3.h5. FASTA IDs must match abundance-table feature IDs. See the sequence guide for model access and local checkpoints, or start with the sequence notebook.

Sparse features and interpretation

The SAE in sae/otu_l10_k64 maps MGM2-XLarge layer-10 OTU states into 4,096 sparse features. It uses BatchTopK training with an average activation budget of 64 and a fixed threshold for inference. Using the outputs above:

from mgm2 import load_sae

sae = load_sae("sae/otu_l10_k64", device="cpu")
valid = (batch["input_ids"] >= 0) & batch["attention_mask"].bool()
with torch.inference_mode():
    features = sae.encode(outputs["hidden"][valid])

print(features.shape)  # (3, 4096): one sparse feature vector per OTU

SAE agent workflow

The interpretation agent connects feature-level measurements with literature evidence through interpretation, on-demand research and independent review. Reusable evidence caches and compact structured handoffs preserve source references while reducing repeated context transfer between roles.

Data and pretraining

The corpus contains 1,821,291 communities: 1,639,420 for training and 181,871 for project-disjoint pretraining validation. The HDF5 resources contain NTv3-derived embeddings for 225,066 OTUs and taxonomic fallback embeddings for 5,087 groups.

Inputs use up to 768 observed OTUs per community. Each complete sample is normalized before top-768 selection; positive values below 1e-4 are floored to 1e-4 and transformed with clip(log10(x) + 2, -2, 2). Retained values preserve this transformed scale after selection and flooring. Per-community quantiles define five positive-abundance bins, indexed 0–4.

Masked-abundance classification, regression for the highest two bins and Qwen3-Embedding-8B semantic alignment jointly train the model. Reconstruction combines the CLS state with the queried OTU's static identity. Inference uses the trained MGM2 encoder and its projection head.

Applications

MGM2 supports microbiome representation research, including hierarchical biome classification, transplantation-associated community and response modelling, drug perturbation prediction, and sparse-feature interpretation. The code repository includes aggregate article benchmark tables and their evaluation definitions.

Acknowledgements

We thank the authors and maintainers of NTv3, Qwen3 Embedding, PyTorch, Hugging Face Transformers and Hugging Face Hub.

Citation

The MGM2 preprint is available on bioRxiv:

Zhang, H. et al. MGM2 as a Multimodal Foundation Model for Microbial Community States and Responses. bioRxiv (2026).

@article{zhang2026mgm2,
  title   = {{MGM2} as a Multimodal Foundation Model for Microbial Community States and Responses},
  author  = {Zhang, Haohong and Zhang, Yuli and Qi, Yuxuan and Liu, Tianao and Zhao, Linfeng and Yang, Ronghua and Ning, Kang},
  journal = {bioRxiv},
  year    = {2026},
  doi     = {10.64898/2026.07.20.739063},
  url     = {https://doi.org/10.64898/2026.07.20.739063}
}
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support