---
license: apache-2.0
datasets:
- agentlans/fineweb-200-weighted
language:
- en
- multilingual
base_model:
- sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
tags:
- translation
- semantic-similarity
---
# Multilingual MiniLM-L12-v2 (Harrier-Distilled)

This model is a high-performance, lightweight sentence encoder aligned via embedding distillation to a top-performing cross-lingual teacher model from the MTEB leaderboard. Designed for high-throughput multilingual tasks, it delivers strong semantic retrieval capabilities while maintaining the speed and efficiency of a compact student architecture.

## Model Details

| Parameter | Value |
| --- | --- |
| **Student Backbone** | `sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2` |
| **Teacher Model** | `microsoft/harrier-oss-v1-270m` |
| **Best Epoch** | 10 |
| **Validation Cosine Similarity** | 0.9377 |
| **Best Validation MSE** | 0.000195 |
| **Learning Rate** | `2e-05` |
| **Batch Size** | 16 |
| **Max Sequence Length** | 256 |
| **License** | Apache 2.0 |

<details>
<summary>Training results</summary>
  
## Validation History
| Epoch | Train Loss | Val MSE | Val Cosine Similarity |
|-------|------------|---------|-----------------------|
| 1 | 0.000493 | 0.000277 | 0.9114 |
| 2 | 0.000249 | 0.000234 | 0.9250 |
| 3 | 0.000224 | 0.000220 | 0.9297 |
| 4 | 0.000212 | 0.000211 | 0.9325 |
| 5 | 0.000203 | 0.000204 | 0.9346 |
| 6 | 0.000197 | 0.000200 | 0.9361 |
| 7 | 0.000193 | 0.000197 | 0.9370 |
| 8 | 0.000190 | 0.000195 | 0.9374 |
| 9 | 0.000188 | 0.000195 | 0.9377 |
| 10 | 0.000187 | 0.000195 | 0.9377 **(Best)** |

</details>

## Installation

Because this model couples a transformer backbone with a trained projection layer, it requires the custom helper package:

```bash
pip install git+https://github.com/agentlans/proj-embed.git

```

## Quick Start

```python
from proj_embed import ProjEmbed

# Initialize the model (loads backbone and projection weights automatically)
model = ProjEmbed("agentlans/multilingual-MiniLM-L12-v2-harrier")

# Generate L2-normalized embeddings for a single text or a batch
embeddings = model.embed(["Hello world", "Greetings from ProjEmbed"])
print(embeddings.shape)  # torch.Size([2, 640])

# Compute cosine similarity between two texts
similarity_score = model.similarity("Machine learning is fascinating.", "AI and deep learning are amazing.")
print(f"Similarity: {similarity_score:.4f}") # Similarity: 0.7600

```

## Intended Use

* **Cross-Lingual Retrieval:** Fast semantic search across multiple languages with reduced compute overhead.
* **Dataset Filtering:** Efficient processing, cleaning, and filtering of large-scale translation and parallel corpora.
* **Semantic Similarity:** Rapid text clustering, deduplication, and ranking.

## Limitations

* **Dimension & Architecture Mismatch:** Does not share the same hidden dimensions or architecture as the larger teacher model.
* **Multilingual Nuance:** Performance may vary on low-resource dialects or highly domain-specific terminology compared to major-language benchmarks.

## License

Apache 2.0 like the base model