---
license: apache-2.0
language:
- uz
base_model:
- google/flan-t5-large
pipeline_tag: summarization
---

# 📝 Paraphraser Model - Inference Documentation

## Overview

This script loads a fine-tuned `FLAN-T5` model (based on `MT5ForConditionalGeneration`) to generate fluent paraphrases of input text. It uses HuggingFace's Transformers library and supports GPU acceleration.

---

## 🔧 Requirements

### Python Packages

```bash
pip install torch transformers
```

### Model Checkpoint

Ensure you have a fine-tuned checkpoint directory (e.g., from training with HuggingFace `Trainer`) available at:

```bash
/home/deploy/second_disk/projects/paraphrase_train/flan-t5-paraphrase-v2/checkpoint-7029
```

---

## 🚀 Usage

### Run the Script

```bash
python paraphrase_infer.py
```

### Example Interaction

```shell
Enter text to paraphrase (Ctrl+C to exit):
> I want to understand this model better.
→ I would like to gain a deeper understanding of this model.
```

---

## 🧠 Functionality Breakdown

### 1. Load Model and Tokenizer

```python
tokenizer = MT5Tokenizer.from_pretrained(model_path)
model = MT5ForConditionalGeneration.from_pretrained(model_path)
```

* Loads tokenizer and model weights from the specified checkpoint directory.

### 2. Device Configuration

```python
device = "cuda" if torch.cuda.is_available() else "cpu"
model = model.to(device)
model.eval()
```

* Uses GPU if available. Sets model to evaluation mode.

### 3. Paraphrasing Function

```python
def paraphrase(text, max_length=128, num_beams=4):
```

#### Parameters:

* `text` (str): Input sentence to be paraphrased.
* `max_length` (int): Max tokens in output (default: 128).
* `num_beams` (int): Beam width for decoding (default: 4).

#### Inside:

* Prefixes input with `"paraphrase:"` (T5-style prompt tuning).
* Applies `top-p` sampling (`top_p=0.95`) and slight randomness (`temperature=0.8`) to encourage variation.

#### Returns:

* A paraphrased version of the input string.

---

## 📁 File Structure Suggestion

```
paraphraser/
├── paraphrase_infer.py  # Your script
├── README.md            # This doc
└── checkpoint/          # Your trained model
```

--

## ⚙️ Customization Ideas

* Swap model to `T5ForConditionalGeneration` if using original T5.
* Deploy as API with FastAPI/Flask.
* Add batch processing support.

---