---
license: gemma
language:
- en
library_name: transformers
pipeline_tag: text-generation
base_model:
- google/gemma-4-E4B-it
- AuriAetherwiing/G4-E4B-Musica-v1
- ToastyPigeon/Gemma4-E4B-TIES-3mask-r64
- juiceb0xc0de/bella-bartender-gemma-e4b
base_model_relation: merge
tags:
- merge
- task-arithmetic
- roleplay
- creative-writing
- gemma-4
- e4b
- measured with calipers not vibes
- one build no regrets
---
# Gemma 4 E4B Nightcap
*One part Musica, a generous splash of TIES, and Bella behind the bar. Served after dark.*
A roleplay / creative-writing merge of three community fine-tunes of **Gemma 4 E4B**, mixed the boring way: measure the weight deltas first, pick parents that don't step on each other, build once, and blind-test the result against the strongest parent before deciding it's worth your bandwidth.
The goal was a single E4B that keeps Musica's prose and stability while picking up range it doesn't have on its own — grounded slice-of-life dialogue, comic timing, and a persona voice that will answer a direct question directly instead of deflecting into vibes. In blind A/B judging across 24 roleplay scenarios it trades wins evenly with Musica overall, with its stable wins exactly where the recipe aimed: grounded, blunt, character-forward scenes.
## What's in the glass
| Parent | What it brings | Weight |
|---|---|---|
| [AuriAetherwiing/G4-E4B-Musica-v1](https://huggingface.co/AuriAetherwiing/G4-E4B-Musica-v1) | The lead voice — RP/storygen tune (an ArliAI collab), stable and smart for its size | 0.8 |
| [ToastyPigeon/Gemma4-E4B-TIES-3mask-r64](https://huggingface.co/ToastyPigeon/Gemma4-E4B-TIES-3mask-r64) | Novel-prose and interactive-fiction texture, trained behind an instruct-preserving gradient mask | 0.225 |
| [juiceb0xc0de/bella-bartender-gemma-e4b](https://huggingface.co/juiceb0xc0de/bella-bartender-gemma-e4b) | A real human voice — bluntness, register-matching, and permission to not end every reply on a pep talk | 0.3 |
Applied as weighted [task arithmetic](https://arxiv.org/abs/2212.04089) over `google/gemma-4-E4B-it`: `base + 0.8·Δmusica + 0.225·Δties3 + 0.3·Δbella`. The multimodal (audio/vision) towers are inherited from base effectively unchanged — all three parents are language-stack tunes.
Recipe as mergekit-equivalent YAML
The build used independent tooling (it streams parents from the Hub and preserves every base tensor, including the phantom KV-shared-layer tensors some E4B re-saves drop), but the recipe is exactly this standard config:
```yaml
merge_method: task_arithmetic
base_model: google/gemma-4-E4B-it
dtype: bfloat16
models:
- model: AuriAetherwiing/G4-E4B-Musica-v1
parameters:
weight: 0.8
- model: ToastyPigeon/Gemma4-E4B-TIES-3mask-r64
parameters:
weight: 0.225
- model: juiceb0xc0de/bella-bartender-gemma-e4b
parameters:
weight: 0.3
```
## Character notes
- **Strong:** single-character cards, grounded dialogue, persona consistency, comedy, answering the question you actually asked. Surprisingly good long-session coherence for an E4B.
- **Trade-off:** Musica alone keeps a slight edge in pure lyrical atmosphere and literary continuation. If that's your whole use case, Musica is right there.
- **Weak:** multiple distinct characters in one card (an ensemble sharing one system prompt). Voices can blend. This appears to be an E4B-class limit — the parents do it too — but it's this model's clearest weak spot.
## Usage
- **Format:** standard Gemma 4 chat template, nothing custom. Works with the thinking channel on or off; most testing was done without.
- **Samplers:** Musica's recommendations carry over — temp 1.0, min-p 0.02, nSigma 2, **no repetition penalties**. Temp 0.8 / top-p 0.95 also tested fine.
- **Quantization:** Q8_0 recommended. Bella's author found her voice audibly sanded down below Q8, and part of that voice is what you're here for.
## Credits
- **[AuriAetherwiing](https://huggingface.co/AuriAetherwiing)** and **[ArliAI](https://www.arliai.com/)** for Musica — the backbone of this merge and a genuinely lovely tune.
- **[ToastyPigeon](https://huggingface.co/ToastyPigeon)** for the TIES-3mask merge and the instruct-subspace masking technique behind it — the reason this ingredient adds prose without subtracting instruction-following.
- **[juiceb0xc0de](https://huggingface.co/juiceb0xc0de)** for Bella — 9,300 pairs of one real human voice, and it shows.
- **Google DeepMind** for Gemma 4 E4B.
Nerd details — how the parents and weights were chosen
The premise: most merge quality is decided before anything is built. So instead of a merge→test→re-merge loop, the pool was measured first and exactly one candidate recipe was built and gated.
### Scouting the pool
Every E4B derivative on the Hub (~1,100 repos at the time) was swept down to ten genuinely distinct creative-writing/RP fine-tunes — dropping quant/format re-uploads, code and domain tunes, and uncensored-only edits, then reading the cards of what remained.
### Measure, then select
Per-tensor weight deltas (Δ = fine-tune − base) were extracted for all ten, over the language stack only (audio/vision towers excluded from all statistics): exact per-tensor delta norms plus a deterministic subsample of coordinates, sampled at identical positions in every model so the candidates can be compared directly.
Measurement did the pruning:
- One popular "family" of tunes was substantially the same vector uploaded twice (pairwise cosine 0.54–0.58; one member was recoverable as a weighted sum of two others) — redundant, dropped.
- Another candidate's headline delta norm was ~99% quantization noise in the per-layer embedding table, not trained signal — dropped.
- A pair of DPO variants moved the weights so little (‖Δ‖ ≈ 0.5, ~40× quieter than the pool median) that they couldn't survive at any reasonable merge weight — dropped.
What survived selection was a **lead voice** (musica — the strongest all-round RP tune in the pool) plus two complements chosen because each brings a direction the lead measurably lacks, and all three are near-orthogonal in delta space — they learned different things, so their contributions add instead of fighting:
| pair | cosine(Δ, Δ) |
|---|---|
| musica ↔ ties3 | 0.017 |
| musica ↔ bella | 0.004 |
| bella ↔ ties3 | 0.0008 |
Delta magnitudes (language stack ℓ₂): ties3 26.6, musica 7.5, bella 1.7 — i.e. bella is a light seasoning by construction, not just by weight.
### Choosing the weights
Candidate weight vectors were swept **on the artifacts alone — no builds** — against the trained-envelope statistic (below). The recipe kept the lead near full strength (0.8, preserving musica's voice and stability), scaled ties3 down to 0.225 (its raw delta is 3.5× the lead's; at higher weight it dominates the mix), and added bella at 0.3 — deliberately below her full strength, because a single-human-voice tune is a garnish that becomes an identity at high weight. Each parent's documented quirks cap its weight: dilution does not reliably remove a known failure mode, as the four-parent variant below demonstrated.
### Constrain the build
Two guardrails picked the weights:
1. **Trained envelope:** at the chosen weights, almost no coordinate of the merged delta exceeds the largest movement any single parent trained at that coordinate (violation rate ≈ 0.7% after flooring out bf16 rounding noise, which is within the "clean" band calibrated on known-good and known-broken merges).
2. **Weight-sum budget:** Σw = 1.325. Empirically (on this family and a 12B sibling project), merges hold together up to roughly Σw ≈ 1.3 and degrade beyond it regardless of per-coordinate statistics — a heavier four-parent variant of this same recipe (Σw = 1.87) passed the envelope check and still lost a blind A/B against both Musica and this model, 3–9 and 2–10, reproducing its parents' documented failure modes at merge scale. This recipe is the one that survived.
As a free integrity check, regressing the built model's delta against the parent subsamples recovers the recipe weights (0.80 / 0.21 / 0.30) — confirming nothing unexpected leaked into the build.
### Blind gate
Two independent rounds of 12 roleplay/creative scenarios (fresh seeds per round, identical samplers, per-prompt shuffled A/B, judged blind at Q8_0): **11–13 vs Musica** — statistical parity, with stable wins split by territory: this merge on grounded/comedic/blunt scenes, Musica on atmosphere and literary continuation. Per the one-build philosophy, parity against the strongest parent plus added range = ship it, don't start a weight-tuning loop.