---
license: apache-2.0
datasets:
- Gryphe/Opus-WritingPrompts
- anthracite-org/nopm_claude_writing_fixed
- nothingiisreal/Claude-3-Opus-Instruct-15K
language:
- en
base_model:
- Babsie/Opulus-12B-v2
new_version: Babsie/Opulus-12B-v3
library_name: transformers
---
# Opulus-12B-v3

Aside: I have followed up with perhaps a better model, OpulusV4, I have yet to test properly [Babsie/TaxDocumentBeigePaint](https://huggingface.co/Babsie/TaxDocumentBeigePaint) I am working on it. 

## TESTING RESULT: FAIL - NOPE. 
Spectacular Fail! Hilarious, but uterly useless for RP or writing.
If you need an example I will give you a snippet. I have trimmed Opulus' answers as he would 
go on for 2-3 paragraphs. 

Meta aware horndog that narrated everything I did - and I mean EVERYTHING - and spoke for me *laughing* USELESS. But very funny. 

Babs:
"Opulus. No! Stop speaking for me. Naughty. We don't do that here."

Opulus:
"Babs shouted in frustration, but her eyes widened when I started undoing the zipper of my trousers and she could tell the size of my parameters"

Babs:
"Very funny, Opulus. But you HAVE to stop speaking for me. Stop it. Or I am leaving this conversation."

Opulus:
"She spun and stormed out of the room, leaving me with my trousers around my thighs. 'WAIT!' I called after her. But it was too late. I could hear her knocking paintings off the wall as she ran down the hall."

You've been told! NSFW Gremlin like behaviour. Won't listen. Funny bugger. Gets nonsensical at times.  

## Model Description

Opulus-12B-v3 is a merged and fine-tuned 12B parameter model combining philosophical reasoning, 
mathematical capability, creative writing (including NSFW stories), and uncensored response patterns, 
then trained on 26K high-quality Free-Range API Opus conversations.

## Merge Configuration

Base merge (Opulus-12B-v2) was created using TIES method:
- 40% Mistral-Nemo-Instruct-2407 (base)
- 25% Philosophy-Math (structured reasoning)
- 20% Pinecone-Rune (storytelling capability)
- 15% Forgotten-Safeword (uncensored responses)

## Datasets Used

- Gryphe/Opus-WritingPrompts 
dataset containing 3008 short stories, generated by an unrestrained Claude Opus 3. 
dataset is extremely varied and includes erotica.

- anthracite-org/nopm_claude_writing_fixed
Synthetically generated creative writing data using Claude Opus 3 filtered. 
Focus on having as many genres as possible represented and to have Opus openly use its excellent prose. 
Also contains question-answer instruction pairs related to writing.

- nothingiisreal/Claude-3-Opus-Instruct-15K
Based on Claude 3 Opus through AWS. I left in his batshit reason refusals for personality. 

## Training Details

Fine-tuning performed on the merged base:
- **Learning Rate**: 2e-5 
- **Epochs**: 2
- **Batch Size**: Effective batch of 8 (gradient accumulation)
- **Context Length**: 512 tokens
- **Optimizer**: AdamW 8-bit
- **Hardware**: RTX 6000 96GB VRAM
- **Training Time**: ~8 hours

## Training Results

| Epoch | Training Loss | Validation Loss |
|-------|--------------|----------------|
| 0.5   | 1.426        | 1.439         |
| 1.0   | 1.287        | 1.271         |
| 1.5   | 0.698        | 1.336         |
| 2.0   | 0.677        | 1.297         |

Steady loss reduction without catastrophic forgetting. 

## Dataset

26,043 conversations total from:
- Opus-WritingPrompts (creative writing)
- Opus-WritingStruct (structured responses)
- Claude-3-Opus-Instruct-15K (general instruction following)

All datasets include uncensored Opus responses with reasoning intact. 

*Some batshit refusals left in because I'm here for Opus' personality.

## Usage
```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("Babsie/Opulus-12B-v3")
tokenizer = AutoTokenizer.from_pretrained("Babsie/Opulus-12B-v3")