Fallovski/french-serer-nllb-lora

French–Serer NLLB LoRA

Part of a benchmark of six configurations for French→Serer neural machine translation. Serer is a critically low-resource Niger-Congo language (~1.2M speakers, Senegal/Gambia), phylogenetically close to the well-resourced Wolof.

Model summary

• Experiment ID: C
• Kind: final_translation_model
• Direction: French → Serer
• Base model: facebook/nllb-200-distilled-600M (revision: main)
• Best checkpoint: french_serer_nllb_lora-epoch=09-val_bleu=19.1677.ckpt
• Random seed: 42

Hyperparameters

• LR: 0.0003
• NUM_EPOCHS: 10
• LORA_R: 16
• LORA_ALPHA: 32
• LORA_DROPOUT: 0.1
• WARMUP_STEPS: 500

Critical limitation — Wolof decoding proxy

Serer (srr_Latn) is not a supported NLLB-200 target. This model decodes under the wol_Latn (Wolof) language tag as a proxy, fine-tuned on French–Serer data. Automatic metrics (especially BLEU) can be partly inflated by lexical/orthographic overlap with Wolof; see the companion proximity-probe calibration for this corpus (Fallovski/french-serer-nllb-wolof-proximity-probe). Human review by a qualified Serer speaker is strongly recommended before any downstream use.

Intended use

Research on French-to-Serer machine translation on a corpus that is ~90% religious (Bible) register, ~10% educational glossaries, primarily Siin dialect. Not validated for legal, medical, emergency, or fully autonomous publication use. Private repository — not intended for public deployment in its current state.

Evaluation

Metric	Value
BLEU (test, beam=5)	17.9291
chrF	38.7191
ROUGE-1	0.4343
ROUGE-L	0.391
BERTScore-F1	0.8764
Test loss	1.9768
BLEU (mean ± std, 3 seeds)	19.1615 ± 1.044

Evaluated on the held-out test split (2890 sentence pairs, SHA-256 of the split: 01d14d982a3c0cce172b5099e2db064d05bdef89677fe72a45f56715d6364ee2). Metrics were computed with the project's own evaluation scripts (not copied from the manuscript without independent reproduction); the training and evaluation code is kept in a private repository, available on request.

Training data and rights

Parallel corpus of 23113 train / 2889 val / 2890 test French–Serer sentence pairs, built primarily from religious texts (Bible, 90%) and educational glossaries (10%), predominantly Siin dialect. Preprocessing: Unicode normalization, exact-duplicate removal, length-ratio filtering (1:3–3:1). Document-level splitting was not possible (no document identifiers available); the split is at the sentence level with a fixed seed. Full provenance, licensing, and consent documentation are kept in a private dataset card, available on request, prior to any public release.