Kquant03/Umbra-MoE-4x10.7-GGUF

Umbra-MoE-4x10.7

Umbra is an off shoot of the [Lumosia Series] with a Focus in General Knowledge and RP/ERP

This model was built around the idea someone wanted a General Assiatant that could also tell Stories/RP/ERP when wanted.

This is a very experimental model. It's a combination MoE of Solar models, the models selected are personal favorites.

base context is 4k but it stays coherent up to 16k

Please let me know how the model works for you.

Join our Discord!

Provided files

Name	Quant method	Bits	Size	Max RAM required	Use case
Q2_K Tiny	Q2_K	2	12 GB	14 GB	smallest, significant quality loss - not recommended for most purposes
Q3_K_M	Q3_K_M	3	15.7 GB	17.7 GB	very small, high quality loss
Q4_0	Q4_0	4	20.3 GB	22.3 GB	legacy; small, very high quality loss - prefer using Q3_K_M
Q4_K_M	Q4_K_M	4	20.4 GB	22.4 GB	medium, balanced quality - recommended
Q5_0	Q5_0	5	24.8 GB	26.8 GB	legacy; large, balanced quality
Q5_K_M	Q5_K_M	5	24.9 GB	26.9 GB	large, balanced quality - recommended
Q6 XL	Q6_K	6	29.6 GB	31.6 GB	very large, extremely minor degradation
Q8 XXL	Q8_0	8	38.4 GB	40.4 GB	very large, extremely minor degradation - not recommended
Template: ChatML

### System:

### USER:{prompt}

### Assistant:

Settings:

Temp: 1.0
min-p: 0.02-0.1

Evals:

• Avg: 73.4
• ARC: 70.31
• HellaSwag: 87.81
• MMLU: 66.42
• T-QA: 67.82
• Winogrande: 83.27
• GSM8K: 64.75

Examples:

User:
Tell me an interesting fact.

Umbra:
The Amazon Rainforest is home to an astounding variety of life forms.

It encompasses over five million square kilometers across nine South American countries and holds approximately one-third of all known species on Earth.

This biodiversity hotspot supports more than 390 billion individual trees representing 16,000 species.

Over two million insect species live here alongside thousands of bird, mammal, amphibian, and reptile species. In this dense forest, indigenous communities have developed intricate knowledge systems that allow them to thrive sustainably within their surroundings.

User:
Tell me a story about a red haired fox

Umbra:
In the enchanted forest of Ardenwood, a vibrant world teeming with life and magic, lived a curious creature known as Vixie, whose russet fur glowed against the verdure that surrounded her. Her emerald eyes twinkled with mischief and wonder as she roamed the lush landscape, befriending all who crossed her path.

One sunlit afternoon, Vixie found herself weaving amongst a sea of tall grasses swaying gently in the breeze. Ahead lay a shimmering riverbank, where a gathering of animals was assembled around a mystifying sight - a radiant stone pulsing with an ethereal glow.

A majestic stag approached Vixie, his voice resonating with authority. He introduced himself as Elthon, leader of this assembly. They had gathered to seek guidance from the celestial gem, said to hold the secrets of the universe within its sparkling depths.

Vixie, filled with curiosity, asked if she could partake in their quest for knowledge. Elthon agreed, and they each took turns gazing into the jewel. As Vixie's turn came, she felt a tingling sensation course through her body, her senses heightened as the stone revealed a vision before her.

In the dreamlike tableau, Vixie saw herself standing atop a hill overlooking Ardenwood. She watched as the seasons changed, the land thrived, and creatures played out their destinies. It appeared the fates of these beings were intertwined, a tapestry woven together by love, friendship, sacrifice, and growth.

When the trance subsided, Vixie returned to the present moment, feeling humbled and grateful. This revelation emboldened her desire to cherish those bonds she shared with others, knowing how vital they were to the beauty and harmony of the enchanted realm. From then on, Vixie continued her adventures with renewed purpose, nurturing friendships and embracing the mysteries of Ardenwood.

Umbra-MoE-4x10.7 is a Mixure of Experts (MoE) made with the following models using:

• kodonho/SolarM-SakuraSolar-SLERP
• Sao10K/Sensualize-Solar-10.7B
• NousResearch/Nous-Hermes-2-SOLAR-10.7B
• fblgit/UNA-SOLAR-10.7B-Instruct-v1.0

🧩 Configuration

base_model: kodonho/SolarM-SakuraSolar-SLERP
gate_mode: hidden
dtype: bfloat16
experts:
  - source_model: kodonho/SolarM-SakuraSolar-SLERP
    positive_prompts:
    - "versatile"
    - "helpful"
    - "factual"
    - "integrated"
    - "adaptive"
    - "comprehensive"
    - "balanced"
    negative_prompts:
    - "specialized"
    - "narrow"
    - "focused"
    - "limited"
    - "specific"

  - source_model: Sao10K/Sensualize-Solar-10.7B
    positive_prompts:
    - "creative"
    - "chat"
    - "discuss"
    - "culture"
    - "world"
    - "expressive"
    - "detailed"
    - "imaginative"
    - "engaging"
    negative_prompts:
    - "sorry"
    - "cannot"
    - "factual"
    - "concise"
    - "straightforward"
    - "objective"
    - "dry"

  - source_model: NousResearch/Nous-Hermes-2-SOLAR-10.7B
    positive_prompts:
    - "analytical"
    - "accurate"
    - "logical"
    - "knowledgeable"
    - "precise"
    - "calculate"
    - "compute"
    - "solve"
    - "work"
    - "python"
    - "javascript"
    - "programming"
    - "algorithm"
    - "tell me"
    - "assistant"
    negative_prompts:
    - "creative"
    - "abstract"
    - "imaginative"
    - "artistic"
    - "emotional"
    - "mistake"
    - "inaccurate"

  - source_model: fblgit/UNA-SOLAR-10.7B-Instruct-v1.0
    positive_prompts:
    - "instructive"
    - "clear"
    - "directive"
    - "helpful"
    - "informative"
    negative_prompts:
    - "exploratory"
    - "open-ended"
    - "narrative"
    - "speculative"
    - "artistic"

💻 Usage

!pip install -qU transformers bitsandbytes accelerate

from transformers import AutoTokenizer
import transformers
import torch

model = "Steelskull/Umbra-MoE-4x10.7"

tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True},
)

messages = [{"role": "user", "content": "Explain what a Mixture of Experts is in less than 100 words."}]
prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])