Memory Tape Pocket Lab

Verified project card

# Memory Tape Pocket

Memory Tape Pocket is a compact differentiable-memory retest inspired by the
content-addressing mechanism of Neural Turing Machines. It learns random
key-value associative recall on tapes containing two to eight slots, then faces
unseen tapes with 16 and 32 slots.

The control is a larger fixed-state GRU trained on the same batches. The
interactive Space exposes the complete external tape and the learned read
weight assigned to every slot.

## Verified result

Across three independent training seeds, the 4,673-parameter content-addressed
model achieved **100% exact recall** on 8-, 16-, and 32-slot tapes. At 32 slots,
four times the maximum training length, its read head placed **99.974%** of its
attention on the correct slot.

The larger 5,584-parameter fixed-state GRU reached 13.51% accuracy at eight
slots, 7.66% at 16 slots, and **4.60% at 32 slots**. This benchmark isolates the
inductive bias of external content addressing; it does not claim the tiny model
implements every component of a full Neural Turing Machine.

```bash
uv run python projects/memory-tape-pocket/train.py
uv run pytest tests/test_memory_tape_pocket.py
```

Evaluation snapshot

{
  "experiment": "Differentiable content addressing versus fixed-state recall",
  "training_slots": [
    2,
    8
  ],
  "results": {
    "memory": {
      "parameters": 4673,
      "runs": [
        {
          "seed": 2281,
          "slots_8": {
            "accuracy": 1.0,
            "examples": 4096,
            "mean_attention_on_correct_slot": 0.9999377218191512
          },
          "slots_16": {
            "accuracy": 1.0,
            "examples": 4096,
            "mean_attention_on_correct_slot": 0.9998665036546299
          },
          "slots_32": {
            "accuracy": 1.0,
            "examples": 4096,
            "mean_attention_on_correct_slot": 0.9997205645777285
          }
        },
        {
          "seed": 2287,
          "slots_8": {
            "accuracy": 1.0,
            "examples": 4096,
            "mean_attention_on_correct_slot": 0.9999373428727267
          },
          "slots_16": {
            "accuracy": 1.0,
            "examples": 4096,
            "mean_attention_on_correct_slot": 0.9998682647856185
          },
          "slots_32": {
            "accuracy": 1.0,
            "examples": 4096,
            "mean_attention_on_correct_slot": 0.999723744156654
          }
        },
        {
          "seed": 2293,
          "slots_8": {
            "accuracy": 1.0,
            "examples": 4096,
            "mean_attention_on_correct_slot": 0.9999510854540858
          },
          "slots_16": {
            "accuracy": 1.0,
            "examples": 4096,
            "mean_attention_on_correct_slot": 0.9998970205051592
          },
          "slots_32": {
            "accuracy": 1.0,
            "examples": 4096,
            "mean_attention_on_correct_slot": 0.9997834917012369
          }
        }
      ],
      "accuracy_mean": {
        "slots_8": 1.0,
        "slots_16": 1.0,
        "slots_32": 1.0
      },
      "correct_slot_attention_mean": {
        "slots_8": 0.9999420500486546,
        "slots_16": 0.9998772629818026,
        "slots_32": 0.9997426001452064
      }
    },
    "gru": {
      "parameters": 5584,
      "runs": [
        {
          "seed": 2281,
          "slots_8": {
            "accuracy": 0.132080078125,
            "examples": 4096
          },
          "slots_16": {
            "accuracy": 0.083984375,
            "examples": 4096
          },
          "slots_32": {
            "accuracy": 0.044677734375,
            "examples": 4096
          }
        },
        {
          "seed": 2287,
          "slots_8": {
            "accuracy": 0.135009765625,
            "examples": 4096
          },
          "slots_16": {
            "accuracy": 0.0703125,
            "examples": 4096
          },
          "slots_32": {
            "accuracy": 0.046142578125,
            "examples": 4096
          }
        },
        {
          "seed": 2293,
          "slots_8": {
            "accuracy": 0.13818359375,
            "examples": 4096
          },
          "slots_16": {
            "accuracy": 0.075439453125,
            "examples": 4096
          },
          "slots_32": {
            "accuracy": 0.047119140625,
            "examples": 4096
          }
        }
      ],
      "accuracy_mean": {
        "slots_8": 0.13509114583333334,
        "slots_16": 0.07657877604166667,
        "slots_32": 0.045979817708333336
      }
    }
  }
}

Backed-up artifact tree

• README.md
• __pycache__/app.cpython-311.pyc
• __pycache__/model.cpython-311.pyc
• __pycache__/train.cpython-311.pyc
• app.py
• artifacts/memory-tape-pocket/content_memory.safetensors
• artifacts/memory-tape-pocket/evaluation.json
• artifacts/memory-tape-pocket/fixed_gru.safetensors
• model.py
• requirements.txt
• train.py