Dc-4nderson/transcript_summarizer_model

🧠 Mistral LoRA Transcript Chunking Model

Model Overview

This LoRA adapter was trained on a custom dataset of 1,000 English transcript examples to teach a Mistral-7B-v0.2 model how to segment long transcripts into topic-based chunks using 'section #:' as delimiters.
It enables automated topic boundary detection in conversation, meeting, and podcast transcripts — ideal for preprocessing before summarization, classification, or retrieval.

---

🧩 Training Objective

The model learns to:

• Detect topic changes in unstructured transcripts
• Insert -- where those shifts occur
• Preserve the original flow of speech

Example:

---

⚙️ Training Configuration

• Base Model: mistralai/Mistral-7B-v0.2
• Adapter Type: LoRA
• PEFT Library: peft==0.10.0
• Training Framework: Hugging Face Transformers
• Epochs: 2
• Optimizer: AdamW
• Learning Rate: 2e-4
• Batch Size: 8
• Sequence Length: 512

---

📊 Training Metrics

Step	Training Loss	Validation Loss	Entropy	Num Tokens	Mean Token Accuracy
100	0.2961	0.1603	0.1644	204,800	0.9594
200	0.1362	0.1502	0.1609	409,600	0.9603
300	0.1360	0.1451	0.1391	612,864	0.9572
400	0.0951	0.1351	0.1279	817,664	0.9635
500	0.0947	0.1297	0.0892	1,022,464	0.9657

Summary:
Loss steadily decreased during training, and accuracy remained consistently above 95%, indicating the model effectively learned transcript reconstruction and accurate delimiter placement.

---

🧰 Usage Example

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base = "mistralai/Mistral-7B-Instruct-v0.2"
adapter = "Dc-4nderson/transcript_summarizer_model"

tokenizer = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base)
model = PeftModel.from_pretrained(model, adapter)

text = (
    "Break this transcript wherever a new topic begins. Use 'section #:' as a delimiter.\n"
    "Transcript: Let's start with last week's performance metrics. "
    "Next, we’ll review upcoming campaign deadlines."
)

inputs = tokenizer(text, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=30000)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

🧾 License

Released under the MIT License — free for research and commercial use with attribution.

🙌 Credits

Developed by Dequan Anderson for automated transcript segmentation and chunked text preprocessing tasks. Built using Hugging Face Transformers, PEFT, and Mistral 7B for efficient LoRA fine-tuning.