File size: 1,899 Bytes
67a9f2b c481a4b 67a9f2b c481a4b 51c7163 c481a4b |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 |
---
language:
- cy
license: llama3.1
base_model: meta-llama/Llama-3.1-8B
tags:
- subnetwork-enhancement
- low-resource
- language-specific
pipeline_tag: text-generation
---
# Llama 3.1 8B cym_latn
Language-enhanced LLaMA-3.1-8B model for **Welsh** using sparse subnetwork fine-tuning.
## Method
- Training approach: Language-specific neuron identification + subnetwork fine-tuning
- Parameters trained: <1% of total model parameters
- Framework: [Language Subnetwork Enhancement](https://github.com/d-gurgurov/Language-Subnetwork-Enhancement-LLMs)
## Performance
Enhanced monolingual capabilities in Welsh while preserving multilingual performance.
## Usage
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("DGurgurov/llama-3.1-8b-cym_latn")
tokenizer = AutoTokenizer.from_pretrained("DGurgurov/llama-3.1-8b-cym_latn")
prompt = "Your prompt here"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_length=100)
print(tokenizer.decode(outputs[0]))
```
## Citation
```bibtex
@misc{gurgurov2025sparsesubnetworkenhancement,
title={Sparse Subnetwork Enhancement for Underrepresented Languages in Large Language Models},
author={Daniil Gurgurov and Josef van Genabith and Simon Ostermann},
year={2025},
eprint={2510.13580},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2510.13580}
}
@misc{gurgurov2025languagearithmeticssystematiclanguage,
title={Language Arithmetics: Towards Systematic Language Neuron Identification and Manipulation},
author={Daniil Gurgurov and Katharina Trinley and Yusser Al Ghussin and Tanja Baeumel and Josef van Genabith and Simon Ostermann},
year={2025},
eprint={2507.22608},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2507.22608},
}
```
|