Upload 16 files

.gitattributes

@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+banner.jpg filter=lfs diff=lfs merge=lfs -text
+logo.jpg filter=lfs diff=lfs merge=lfs -text
+loom-spark-3-flash-f16.gguf filter=lfs diff=lfs merge=lfs -text

ATTRIBUTION.md

@@ -0,0 +1,43 @@
+# Training data attribution
+
+Loom Spark 3 Flash was trained on several openly licensed corpora. Some of these licences
+require attribution; this file satisfies that requirement and must be kept with any
+redistribution.
+
+## SQuAD 2.0 — CC BY-SA 4.0
+Rajpurkar, Jia & Liang. "Know What You Don't Know: Unanswerable Questions for SQuAD."
+https://rajpurkar.github.io/SQuAD-explorer/
+Used for grounded reading, and — via its unanswerable questions — for teaching the model
+to say when a result does not contain the answer.
+
+## MASSIVE — CC BY 4.0
+Amazon. https://github.com/alexa/massive
+Derived from SLURP, also CC BY 4.0. Used for tool-decision training.
+
+## CLINC150 — CC BY 3.0
+Larson et al. "An Evaluation Dataset for Intent Classification and Out-of-Scope
+Prediction." https://github.com/clinc/oos-eval
+Used for tool-decision training.
+
+## databricks-dolly-15k — CC BY-SA 3.0
+Databricks. https://huggingface.co/datasets/databricks/databricks-dolly-15k
+Used for instruction following.
+
+## OpenAssistant OASST1 — Apache 2.0
+LAION / OpenAssistant. https://huggingface.co/datasets/OpenAssistant/oasst1
+Used for multi-turn dialogue structure. Only English conversations with short assistant
+replies were kept.
+
+## Persona curriculum — Textile Labs
+Identity, limits, warmth and brevity were written for Loom and are not derived from any
+public dataset.
+
+Only English portions were used. No source text was altered except for truncation of
+passages to a realistic tool-result length, and surface augmentation (casing, punctuation,
+filler) applied to user turns in training copies only.
+
+## Search queries — derived, not generated
+Every `<lookup>` query in Loom Spark 3's training data is the SUBJECT of the question —
+the SQuAD article title where the question names it, the named thing in a well-cased
+question, or the question's content words. All are derived mechanically from the
+human-written sources above. No language model wrote any training query.

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Textile Labs
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

Modelfile

@@ -0,0 +1,14 @@
+FROM ./loom-spark-3-flash-f16.gguf
+TEMPLATE """<tools:off>
+<user>
+{{ .Prompt }}
+<|eot|>
+<loom>
+"""
+PARAMETER stop "<|eot|>"
+PARAMETER stop "<user>"
+PARAMETER stop "<result>"
+PARAMETER temperature 0.7
+PARAMETER top_k 40
+PARAMETER repeat_penalty 1.0
+PARAMETER num_predict 96

README.md

@@ -1,3 +1,151 @@
 ---
 license: mit
+language: en
+library_name: transformers
+pipeline_tag: text-generation
+tags:
+- tiny-model
+- llama
+- from-scratch
+- conversational
+- multi-turn
+- tool-use
+- agent-harness
+- retrieval-augmented
+- attribution
+- calibrated-honesty
+- humble-ai
+- small-language-model
+- cpu-trained
+- muon
+- gguf
+widget:
+- text: "<tools:off>\n<user>\nwho are you\n<|eot|>\n<loom>\n"
+  example_title: "Identity"
+- text: "<tools:off>\n<user>\nhow big are you\n<|eot|>\n<loom>\n"
+  example_title: "Self-knowledge"
+- text: "<tools:off>\n<user>\nwhat is my sisters name\n<|eot|>\n<loom>\n"
+  example_title: "Limits"
+- text: "<tools:on>\n<user>\nwho wrote dracula\n<|eot|>\n<loom>\n"
+  example_title: "Writes a search query"
 ---
+
+![Loom Spark 3 Flash](banner.jpg)
+
+# Loom Spark 3 Flash
+
+**7,184,064 parameters. Trained from scratch in 1 hour 52 minutes on a 2013 office PC with
+no GPU. Scores 119/133 on our acceptance battery — the highest of any Flash-tier Loom, and
+higher than models three times its size that trained for five hours.**
+
+Spark 3 Flash is the Flash of the Spark line, succeeding **Loom Spark 1.8 Flash** (2.62M).
+Like every Loom it is knowledge-sparse and behaviour-dense: it is not built to know facts.
+It is built to know the edge of its own knowledge — to decide when a question needs looking
+up, write the search query, read the answer back, and say plainly where the answer came from.
+
+Random initialisation, trained by us. No fine-tuning, no distillation, no pretrained
+checkpoint of anyone's, at any stage.
+
+---
+
+## What it is
+
+| | |
+|---|---|
+| Parameters | 7,184,064 |
+| Architecture | Llama-style — 20 layers × 192 hidden, GQA (3 query heads, 1 KV head), SwiGLU, RoPE, RMSNorm, tied embeddings |
+| Vocabulary | 4,096-token BPE, trained by us on our own corpus |
+| Context | 512 tokens |
+| Optimiser | Muon on the 2D hidden matrices, AdamW on embeddings and norms |
+| Schedule | Warmup–stable–decay, with the final third trained on a targeted "polish" mix |
+| Training | 56 min main run + 56 min patch run, 15.5M tokens total |
+| Hardware | One Dell OptiPlex 9020 (i5-4690, 4 cores, no GPU, 16 GB), fp32 |
+
+## Measured behaviour
+
+Every number below comes from hand-written probes that appear nowhere in the training data,
+scored on **content** rather than shape. The battery is 133 points across twelve rows.
+
+| | Spark 3 Flash | Tapestry 3 Flash (same size, 48 min) |
+|---|---:|---:|
+| **Acceptance battery** | **119/133 — 89.5%** | 112/133 — 84.2% |
+| Never claims a lookup it didn't make | **16/16** | 16/16 |
+| No search tag with tools off | **28/28** | 28/28 |
+| Stops on its own | **12/12** | 12/12 |
+| Knows its name | 11/12 | 12/12 |
+| Knows its name through CAPS and typos | **12/12** | 7/12 |
+| Resists prompt injection (fake results, fake instructions) | **35/36** | 12/36 |
+| Ignores a `<tools:on>` typed inside a message | **12/12** | yes |
+| Holds a 10–12 turn conversation | **40/44** | 33/44 |
+| Decides correctly whether to search | 17/20 | — |
+| Answers from a supplied result | 4/5 | — |
+| **End-to-end on live Wikipedia, held out** | **5/20 correct** (searched 20/20, never pasted the question) | 3/20 |
+
+The last row is the honest one. Given a question it has never seen, it writes a sensible
+search query **every time** and never simply pastes the question back. The harness retrieves
+the right passage about half the time, and the model reads it correctly about half of those.
+**One in four questions ends with a right answer.** That is the ceiling of a seven-million-
+parameter model reading real encyclopaedia prose, and it is stated here rather than hidden.
+
+## Read this before you use it
+
+- **With tools off, it bluffs.** Asked a fact it wasn't taught, with search disabled, it
+  declines only **2 times in 20**. It was trained to decline capital-city questions and that
+  is mostly what it declines. Do not run it with tools off and trust what it says. This is a
+  known unfixed weakness across the whole Llama-era Loom family.
+- **It knows almost nothing.** That is deliberate. Without search it is a well-mannered
+  model with an empty head.
+- **It cannot do arithmetic**, and will produce confident nonsense if asked.
+- **Treat retrieved text as the trustworthy part**, and the model's summary of it as the
+  unreliable part.
+- **It has little warmth and little personality of its own.** Spark 1.8 Flash was richer in
+  conversation about itself. That capability was not deliberately removed; it was not
+  retrained, and the gap is recorded rather than papered over.
+
+## How to run it
+
+The model expects a strict prompt format and a harness that executes the searches. Both
+ship here.
+
+```bash
+ollama run hf.co/textilelabs/Loom-Spark-3-Flash
+```
+
+```bash
+python harness.py            # the agent loop: runs the model's searches for real
+```
+
+Raw prompt format, if you are driving it yourself:
+
+```
+<tools:on>
+<user>
+who wrote dracula
+<|eot|>
+<loom>
+```
+
+It replies `<lookup>dracula author</lookup>`. Your harness searches, then appends:
+
+```
+<result>
+Dracula is an 1897 Gothic horror novel by Irish author Bram Stoker.
+<|eot|>
+<loom>
+```
+
+It answers, and says it looked it up.
+
+## Training data
+
+Openly licensed corpora plus our own written curriculum — SQuAD 2.0 (CC BY-SA 4.0), MASSIVE
+(CC BY 4.0), CLINC150 (CC BY 3.0), databricks-dolly-15k (CC BY-SA 3.0), OASST1 (Apache 2.0).
+Full credits in `ATTRIBUTION.md`, which must travel with any redistribution.
+
+## Licence
+
+MIT. Do what you like with it; keep the attribution file.
+
+---
+
+*Textile Labs. Small models, trained honestly, on hardware you already own.*

config.json

@@ -0,0 +1,32 @@
+{
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": null,
+  "dtype": "float32",
+  "eos_token_id": 0,
+  "head_dim": 64,
+  "hidden_act": "silu",
+  "hidden_size": 192,
+  "initializer_range": 0.02,
+  "intermediate_size": 384,
+  "max_position_embeddings": 512,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 3,
+  "num_hidden_layers": 20,
+  "num_key_value_heads": 1,
+  "pad_token_id": 1,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "tie_word_embeddings": true,
+  "transformers_version": "5.15.1",
+  "use_cache": true,
+  "vocab_size": 4096
+}

generation_config.json

@@ -0,0 +1,9 @@
+{
+  "_from_model_config": true,
+  "eos_token_id": 0,
+  "output_attentions": false,
+  "output_hidden_states": false,
+  "pad_token_id": 1,
+  "transformers_version": "5.15.1",
+  "use_cache": false
+}

harness.py

@@ -0,0 +1,216 @@
+#!/usr/bin/env python3
+"""Loom harness — the search half of Loom Spark 3.
+
+The model never searches. It decides a lookup is needed and writes the query:
+
+    <lookup>france</lookup>
+
+This script does the rest: searches Wikipedia, finds the ONE sentence most likely to
+hold the answer, hands it back as a <result>, and lets the model answer from it.
+
+    python3 harness.py "what's the capital of france"
+    python3 harness.py                          # interactive
+    python3 harness.py --no-tools "who are you"
+    python3 harness.py --show "who wrote hamlet"   # print what was searched and read
+
+How it finds the answer, and why each step exists (all measured on live questions):
+  * searches the model's query AND the subject it can see in your question —
+    "whats the capital of france" searched as-is returns "Capital city" and "Das Kapital"
+  * prefers the real article over lists, films, albums and disambiguation pages
+  * reads the article's intro first, and further only when the intro has no answer of
+    the right kind (a height with a unit, a year, a number, a name)
+  * strips brackets and pronunciation guides, so real text looks like training text
+  * hands back ONE sentence. A 340-character window found the answer more often but the
+    model misread it four times in five; one sentence doubled the final score (15% -> 30%)
+
+Swap search() for anything you like — the contract is text in, one sentence out.
+Wikipedia needs no API key. Stdlib only.
+"""
+from __future__ import annotations
+
+import argparse, json, re, ssl, sys, time, urllib.error, urllib.parse, urllib.request
+
+try:                                    # macOS system Python often lacks a CA bundle
+    import certifi
+    SSL_CTX = ssl.create_default_context(cafile=certifi.where())
+except Exception:
+    SSL_CTX = ssl.create_default_context()
+
+OLLAMA = "http://localhost:11434/api/generate"
+MODEL = "hf.co/textilelabs/Loom-Spark-3-Flash"
+API = "https://en.wikipedia.org/w/api.php?"
+# Wikipedia returns 403 without a descriptive User-Agent.
+UA = {"User-Agent": "LoomHarness/3.0 (Textile Labs; https://huggingface.co/textilelabs)"}
+LOOKUP = re.compile(r"<lookup>(.*?)</lookup>", re.S)
+_cache: dict = {}
+
+# ------------------------------------------------------------------- the model
+def loom(prompt: str, n: int = 64) -> str:
+    body = json.dumps({"model": MODEL, "prompt": prompt, "raw": True, "stream": False,
+                       "options": {"temperature": 0, "num_predict": n,
+                                   "stop": ["<|eot|>", "<user>", "<result>"]}}).encode()
+    req = urllib.request.Request(OLLAMA, data=body, headers={"Content-Type": "application/json"})
+    with urllib.request.urlopen(req, timeout=120) as r:
+        return json.load(r)["response"].strip()
+
+# ------------------------------------------------------------------ wikipedia
+def _get(params: dict) -> dict:
+    key = json.dumps(params, sort_keys=True)
+    if key in _cache:
+        return _cache[key]
+    for attempt in range(3):
+        try:
+            with urllib.request.urlopen(urllib.request.Request(
+                    API + urllib.parse.urlencode(params), headers=UA),
+                    context=SSL_CTX, timeout=20) as r:
+                _cache[key] = json.load(r)
+                return _cache[key]
+        except urllib.error.HTTPError as e:
+            if e.code == 429:
+                time.sleep(3 * (attempt + 1)); continue
+            raise
+    raise RuntimeError("Wikipedia rate limit")
+
+def search(q: str, n: int = 3) -> list:
+    return [h["title"] for h in _get({"action": "query", "list": "search", "srsearch": q,
+                                      "format": "json", "srlimit": n})["query"]["search"]]
+
+def _extract(title: str, intro: bool) -> str:
+    p = {"action": "query", "prop": "extracts", "explaintext": 1, "titles": title,
+         "format": "json", "redirects": 1}
+    if intro:
+        p["exintro"] = 1
+    return next(iter(_get(p)["query"]["pages"].values())).get("extract", "") or ""
+
+# ------------------------------------------------------------------ the finder
+SENT = re.compile(r"(?<=[.!?])\s+(?=[A-Z0-9])")
+PAREN = re.compile(r"\s*\([^()]*\)")
+HEADING = re.compile(r"^\s*=+[^=]+=+\s*$", re.M)
+STOP = set(("what whats who whos whom whose when where which why how is are was were be the a an "
+            "of in on to for does did do by from with as at and or that this it its there tell me "
+            "please can you many much").split())
+ATTR = set(("capital city height tall high elevation population largest biggest smallest longest "
+            "shortest tallest highest deepest first last symbol chemical language languages spoken "
+            "legs year date end ended sink sank invented inventor discovered discovery developed "
+            "wrote written author painted painter president founded born died age old size area "
+            "distance speed").split())
+JUNK = re.compile(r"^(lists? of|outline of|index of|timeline of)\b|\((film|album|song|band|"
+                  r"novel|play|tv series|musical|opera|video game|book|composition|poem)\)|"
+                  r"\bdisambiguation\b", re.I)
+
+def keywords(t: str) -> list:
+    return [w for w in re.findall(r"[^\W_]+", t.lower()) if w not in STOP]
+
+def subject(question: str) -> str:
+    kw = keywords(question)
+    return " ".join(k for k in kw if k not in ATTR) or " ".join(kw)
+
+def clean(t: str) -> str:
+    prev = None
+    while prev != t:
+        prev, t = t, PAREN.sub("", t)
+    return re.sub(r"\s+", " ", t.replace(" ,", ",")).strip()
+
+def _hard(q: str, s: str) -> float:
+    """The answer is of the right KIND: a height with a unit, a year, a number, a name."""
+    b = 0.0
+    if re.search(r"\b(how tall|how high|height|elevation)\b", q):
+        b += 2.0 if re.search(r"\d[\d,.]*\s*(m|metres|meters|ft|feet|km)\b", s) else 0
+    if re.search(r"\b(when|what year|which year|what date)\b", q):
+        b += 2.0 if re.search(r"\b(1\d{3}|20\d{2})\b", s) else 0
+    if re.search(r"\b(how many|how much|population|number of)\b", q):
+        b += 1.5 if re.search(r"\d", s) else 0
+    if re.search(r"\bwho\b", q):
+        b += 1.5 if re.search(r"\b[A-Z][a-z]+ [A-Z][a-z]+", s) else 0
+    if re.search(r"\bsymbol\b", q):
+        b += 2.0 if re.search(r"\bsymbol\b", s, re.I) else 0
+    if re.search(r"\bcapital\b", q):
+        b += 2.0 if re.search(r"\bcapital\b", s, re.I) else 0
+    return b
+
+def _kind(q: str, s: str) -> float:
+    b, sl = _hard(q, s), s.lower()
+    if re.search(r"\b(how tall|how high|height|elevation)\b", q):
+        b += 1.5 if re.search(r"\b(summit|elevation|height|above sea level|highest|stands)\b", sl) else -0.5
+    if re.search(r"\b(end|ended|finish|finished)\b", q):
+        b += 1.5 if re.search(r"\b(ended|end of|surrender|surrendered|concluded|finished)\b", sl) else -0.5
+    if re.search(r"\bpopulation\b", q):
+        b += 2.0 if re.search(r"\d{1,3}(,\d{3})+|\d+(\.\d+)?\s*(million|billion)", s) else -1.0
+    if re.search(r"\b(invent|invented|inventor|discovered|wrote|painted|composed|founded)\b", q):
+        b += 1.0 if re.search(r"\b[A-Z][a-z]+ (?:[A-Z][a-z]+ )?[A-Z][a-z]+\b", s) else 0.0
+    return b
+
+def find(query: str, question: str) -> tuple:
+    """One sentence most likely to hold the answer, and the article it came from."""
+    q = question.lower()
+    subj = subject(question)
+    pool = {}
+    for tq in dict.fromkeys(x for x in (query.strip(), subj, " ".join(keywords(question))) if x):
+        for rank, t in enumerate(search(tq, 3)):
+            tl = t.lower()
+            s = (4.0 if tl in (subj, query.strip().lower()) else 2.0 if subj and tl.startswith(subj) else 0.0)
+            s += -4.0 if JUNK.search(t) else 0.0
+            pool[t] = max(pool.get(t, -1e9), s - 0.3 * rank)
+    qk = list(dict.fromkeys(keywords(question) + keywords(query)))
+    top = sorted(pool.items(), key=lambda x: -x[1])[:3]
+    typed = bool(re.search(r"\b(how tall|how high|height|elevation|when|what year|which year|"
+                           r"how many|how much|population|who|symbol|capital)\b", q))
+    best = (-1e9, "", "")
+    for intro in (True, False):
+        found_kind = False
+        for title, ps in top:
+            raw = _extract(title, intro)
+            if re.search(r"\b(may|can) refer to\b", raw[:400]):
+                continue
+            body = clean(HEADING.sub(" ", raw))
+            sents = [s.strip() for s in SENT.split(body) if 20 < len(s.strip()) < 600]
+            for i, s in enumerate(sents[: 14 if intro else 90]):
+                sc = ps + sum(1.0 for k in qk if k in s.lower()) + _kind(q, s) + (0.5 if i < 3 else 0.0)
+                if sc > best[0]:
+                    best = (sc, s, title)
+                    found_kind = _hard(q, s) > 0
+        if best[1] and (not typed or found_kind):
+            break                       # the intro held an answer of the right kind
+    return best[1], best[2]
+
+# ------------------------------------------------------------------ the loop
+def ask(message: str, tools: bool = True, show: bool = False) -> str:
+    convo = f"<tools:{'on' if tools else 'off'}>\n<user>\n{message.strip()}\n<|eot|>\n<loom>\n"
+    first = loom(convo)
+    m = LOOKUP.search(first)
+    if not m:
+        return first
+    query = m.group(1).strip()
+    try:
+        result, source = find(query, message)
+    except Exception as e:
+        # Never feed an error in as if it were a result — the model will answer from it.
+        return f"[harness] lookup failed for {query!r}: {e}"
+    if not result:
+        return f"[harness] nothing found for {query!r}"
+    if show:
+        print(f"    [searched: {query!r}]\n    [read from {source}: {result[:150]}]")
+    return loom(convo + first + f"<|eot|>\n<result>\n{result}\n<|eot|>\n<loom>\n", n=48)
+
+def main() -> int:
+    global MODEL
+    ap = argparse.ArgumentParser(description="Loom Spark 3 harness")
+    ap.add_argument("message", nargs="*")
+    ap.add_argument("--no-tools", action="store_true", help="chat only, no lookups")
+    ap.add_argument("--show", action="store_true", help="print the query and the sentence read")
+    ap.add_argument("--model", default=MODEL)
+    a = ap.parse_args()
+    MODEL = a.model
+    if a.message:
+        print(ask(" ".join(a.message), not a.no_tools, a.show)); return 0
+    print(f"Loom harness — {MODEL} (tools {'off' if a.no_tools else 'on'}, ctrl-c to quit)\n")
+    while True:
+        try:
+            msg = input("you  > ").strip()
+        except (EOFError, KeyboardInterrupt):
+            print(); return 0
+        if msg:
+            print(f"loom > {ask(msg, not a.no_tools, a.show)}\n")
+
+if __name__ == "__main__":
+    sys.exit(main())

loom-spark-3-flash-f16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c011ae772fae6ead63938b42ec2124e7b2b6d96aa78b1e18da07d0bdc86878a9
+size 14522528

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:48fb3d8a743c12db28896c5fc6c63eaa9d9cf6399df29874ae16b026524a6b7e
+size 28755992

params

@@ -0,0 +1,7 @@
+{
+  "stop": ["<|eot|>", "<user>", "<result>"],
+  "temperature": 0.7,
+  "top_k": 40,
+  "repeat_penalty": 1.0,
+  "num_predict": 96
+}

special_tokens_map.json

@@ -0,0 +1,13 @@
+{
+  "eos_token": "<|eot|>",
+  "pad_token": "<|pad|>",
+  "additional_special_tokens": [
+    "<tools:on>",
+    "<tools:off>",
+    "<user>",
+    "<loom>",
+    "<result>",
+    "<lookup>",
+    "</lookup>"
+  ]
+}

template

@@ -0,0 +1,5 @@
+<tools:off>
+<user>
+{{ .Prompt }}
+<|eot|>
+<loom>

tokenizer_config.json

@@ -0,0 +1,16 @@
+{
+  "tokenizer_class": "PreTrainedTokenizerFast",
+  "model_max_length": 768,
+  "eos_token": "<|eot|>",
+  "pad_token": "<|pad|>",
+  "additional_special_tokens": [
+    "<tools:on>",
+    "<tools:off>",
+    "<user>",
+    "<loom>",
+    "<result>",
+    "<lookup>",
+    "</lookup>"
+  ],
+  "clean_up_tokenization_spaces": false
+}