ZurichNLP/romansh-nllb-1.3b-2026-10-ct2

Romansh NLLB-200 1.3B CT2 Model Card

Romansh NLLB 2026-10 is an update to the original Romansh NLLB model (https://huggingface.co/ZurichNLP/romansh-nllb-1.3b-ct2). The model is a multilingual NMT model finetuned from the NLLB-200-Distilled-1.3B model (https://huggingface.co/facebook/nllb-200-distilled-1.3B). It can translate between German and six Romansh varieties, which makes it the first model to generate fluent translations in those individual Romansh varieties. This new model was trained on a slightly larger dataset with modified preprocessing.

The data used for the finetuning are monolingual data in all Romansh varieties, which we back-translated into German using Gemini 3.8 Flash, as well as a smaller amount of natively parallel data. A more extensive overview over the data used can be found further below.

The model has been extended by six language tokens, representing each of the Romansh varieties. The exact usage of the model is described below.

How to use

Before using the model, make sure that ctranslate2 is installed. This library allows for efficient inference. This model is already converted to work with ctranslate2.

Installation:

pip install ctranslate2

Further information on ctranslate2 can be found under this link: https://github.com/OpenNMT/CTranslate2

---

Follow the below steps to translate a sentence.

1. Load model and tokenizer from repo.

import ctranslate2
from huggingface_hub import snapshot_download
from transformers import NllbTokenizer

# Download the model from the HF hub. The model is already converted to CT2 format
model_path = snapshot_download(repo_id="ZurichNLP/romansh-nllb-1.3b-2026-10-ct2")

translator = ctranslate2.Translator(model_path, device="cpu")
tokenizer = NllbTokenizer.from_pretrained("ZurichNLP/romansh-nllb-1.3b-2026-10-ct2")

2. Prepare model inputs.

source_lang_code = "deu_Latn" # German
target_lang_code = "rm-puter" # Puter

source_sentence = "Gestern hat es geschneit."

tokenizer.set_src_lang_special_tokens(source_lang_code)

source_token_ids = tokenizer.encode(source_sentence, add_special_tokens=True)
source_tokens = tokenizer.convert_ids_to_tokens(source_token_ids)

3. Run the model as below.

results = translator.translate_batch(
    [source_tokens],
    target_prefix=[[target_lang_code]],
    beam_size=4,
)

4. Postprocess translation as described below.

result = results[0]
target_tokens = result.hypotheses[0]
translation = tokenizer.decode(tokenizer.convert_tokens_to_ids(target_tokens), skip_special_tokens=True)
translation
# Her ho que navieu.

The overview over the six varieties and their respective language codes can be found in the below table.

Romansh variety	Language Code
Rumantsch Grischun	rm-rumgr
Sursilvan	rm-sursilv
Sutsilvan	rm-sutsilv
Surmiran	rm-surmiran
Puter	rm-puter
Vallader	rm-vallader

Training Data

Parallel data

Compared to the previous model, this version applies additional preprocessing filters (slightly reducing token counts across existing datasets), adds "Laws of Canton Grisons" as a new dataset, and drops the RTR-contributed parallel data used previously, which was removed due to noise.

Name	Romansh varieties	URL	RM tokens	DE tokens
Dictionary data	Rumantsch Grischun, Sursilvan, Sutsilvan, Surmiran, Puter, Vallader	https://pledarigrond.ch/rumantschgrischun	3,298,051	6,530,231
Mediomatix	Sursilvan, Sutsilvan, Surmiran, Puter, Vallader	https://huggingface.co/datasets/ZurichNLP/mediomatix	5,946,856	-
Press releases of Canton Grisons	Rumantsch Grischun	https://www.gr.ch/RM/medias/communicaziuns/MMStaka/Seiten/AktuelleMeldungen.aspx	1,825,339	1,340,454
Laws of Canton Grisons 🆕	Rumantsch Grischun	https://huggingface.co/datasets/ZurichNLP/romansh-canton-laws	467,970	329,642
SwissLawTranslations	Rumantsch Grischun	https://huggingface.co/datasets/joelniklaus/SwissLawTranslations	592,136	483,176
Storyweaver	Sursilvan, Sutsilvan, Surmiran, Puter, Vallader	-	44,941	7,928
Total			12,175,293	8,691,431

Monolingual data

As with the parallel data, additional preprocessing filters slightly reduce token counts across existing datasets. Two new datasets were also added: "La Pagina da Surmeir" (articles from 2000-2014, https://paginadasurmeir.ch/home.html) and a collection of miscellaneous Sutsilvan texts.

Name	Romansh varieties	URL	Tokens
FineWeb2	Rumantsch Grischun, Sursilvan, Sutsilvan, Surmiran, Puter, Vallader	https://huggingface.co/datasets/HuggingFaceFW/fineweb-2	43,357,760
La Quotidiana (1997–2008, 2021–2025)	Rumantsch Grischun, Sursilvan, Sutsilvan, Surmiran, Puter, Vallader	https://huggingface.co/datasets/ZurichNLP/quotidiana	40,218,002
News articles by RTR	Rumantsch Grischun, Sursilvan, Sutsilvan, Surmiran, Puter, Vallader	https://rtr.ch/	22,587,779
FinePDFs	Rumantsch Grischun, Sursilvan, Sutsilvan, Surmiran, Puter, Vallader	https://huggingface.co/datasets/HuggingFaceFW/finepdfs	18,923,766
La Pagina da Surmeir 🆕	Surmiran (98%), Rumantsch Grischun (2%)*	-	14,919,414
Mediomatix (unaligned)	Rumantsch Grischun, Sursilvan, Sutsilvan, Surmiran, Puter, Vallader	https://huggingface.co/datasets/ZurichNLP/mediomatix-raw	3,548,414
FineWiki	Rumantsch Grischun, Sursilvan, Sutsilvan, Surmiran, Puter, Vallader	https://huggingface.co/datasets/HuggingFaceFW/finewiki	2,610,012
Audio transcriptions by RTR	Rumantsch Grischun, Sursilvan, Sutsilvan, Surmiran, Puter, Vallader	https://developer.srgssr.ch/en/apis/rtr-linguistic	1,511,333
Theater plays	Rumantsch Grischun, Sursilvan, Sutsilvan, Surmiran, Puter, Vallader	https://huggingface.co/datasets/ZurichNLP/romansh_theater_plays	1,051,608
Sutsilvan Miscellanea 🆕	Sutsilvan	-	373,194
Municipal documents	Rumantsch Grischun, Sursilvan, Sutsilvan, Surmiran, Puter, Vallader	https://huggingface.co/datasets/ZurichNLP/romansh-municipal-text-corpus	302,312
Historical Dictionary of Switzerland	Rumantsch Grischun, Puter	https://hls-dhs-dss.ch/rm/	235,737
Revista digl noss Sulom	Rumantsch Grischun, Sursilvan, Sutsilvan, Surmiran	-	189,095
Babulins	Rumantsch Grischun, Sursilvan, Sutsilvan, Surmiran, Puter, Vallader	-	50,883
Total			149,879,309

* Although La Pagina da Surmeir is a Surmiran newspaper, it also carries a small share of cantonal and other public-sector press releases that are published in Rumantsch Grischun.

Evaluation Results

German to Romansh BLEU:

System	Rumantsch Grischun	Sursilvan	Sutsilvan	Surmiran	Puter	Vallader
Gemini 2.5 Flash	40.3	28.0	10.6	16.9	22.0	27.3
Gemini 3 Flash (preview)	42.1	32.8	12.7	21.3	26.4	29.8
Gemini 3 Pro (preview)	45.3	37.1	17.3	27.1	34.7	36.1
Romansh NLLB-1.3b-ct2	48.4	44.5	40.5	43.0	44.9	44.6
Romansh NLLB-1.3b-2026-10-ct2	49.3	44.8	42.0	43.4	46.2	45.9

Romansh to German BLEU:

System	Rumantsch Grischun	Sursilvan	Sutsilvan	Surmiran	Puter	Vallader
Gemini 3 Flash (preview)	55.2	50.1	47.4	52.2	53.0	60.7
Gemini 3 Pro (preview)	55.8	50.8	48.7	52.2	53.8	60.8
Gemini 3.8 Flash	57.6	53.4	52.3	55.4	56.3	63.6
Romansh NLLB-1.3b-ct2	50.5	46.3	44.2	47.3	49.5	54.8
Romansh NLLB-1.3b-2026-10-ct2	49.8	45.6	45.2	47.9	49.0	55.0

Romansh to German COMET:

System	Rumantsch Grischun	Sursilvan	Sutsilvan	Surmiran	Puter	Vallader
Gemini 2.5 Flash	93.7	93.1	89.8	91.7	92.3	92.7
Gemini 3 Flash (preview)	93.9	94.0	92.7	93.4	92.8	93.9
Gemini 3 Pro (preview)	93.8	93.8	92.5	93.4	93.0	93.8
Gemini 3.8 Flash	95.2	94.6	94.7	94.9	94.5	95.0
Romansh NLLB-1.3b-ct2	92.5	92.4	91.1	91.4	91.6	91.5
Romansh NLLB-1.3b-2026-10-ct2	93.6	93.4	92.6	93.3	92.9	93.1

Ablation models

Model	Repo
No data augmentation	https://hf.co/ZurichNLP/romansh-nllb-1.3b-ablation-no-augmentation-ct2
HR→LR augmentation (based on Europarl)	https://hf.co/ZurichNLP/romansh-nllb-1.3b-ablation-hr-lr-augmentation-ct2
LR→HR augmentation without dictionary prompting	https://hf.co/ZurichNLP/romansh-nllb-1.3b-ablation-lr-hr-augmentation-ct2

Acknowledgements

We thank RTR and Fundaziun Patrimoni Cultural RTR for their support. We are grateful to Zachary Hopton, Diana Merkle, Anna Rutkiewicz and Sudehsna Sivakumar for help with data curation, Uniun dals Grischs for contributing dictionary data for Puter and Vallader, and Giuanna Caviezel, Not Soliva and their seminar participants for helpful feedback. We also acknowledge the contribution of the native speakers who participated in the human evaluation study. For this publication, use was made of media data made available via Swissdox@LiRI by the Linguistic Research Infrastructure of the University of Zurich (see (https://www.liri.uzh.ch/en/services/swissdox.html) for more information).

License

CC-BY-NC-4.0

Citation

Paper describing the original model:

@misc{vamvas-et-al-2026-translation,
      title={Translation Asymmetry in LLMs as a Data Augmentation Factor: A Case Study for 6 Romansh Language Varieties},
      author={Jannis Vamvas and Ignacio Pérez Prat and Angela Heldstab and Dominic P. Fischer and Sina Ahmadi and Rico Sennrich},
      year={2026},
      eprint={2603.25489},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2603.25489},
}