---
license: mit
base_model: llava-hf/llava-1.5-7b-hf
library_name: peft
tags:
  - vision-language
  - lora
  - accv-2026
  - gate-closure
  - llava-1.5-7b
---

# RMS prefix, ungated (run 3) (rms_ungated_r3)

Checkpoint **`rms_ungated_r3`** from *The Gate Always Closes: On Injecting Auxiliary Signals into Frozen
Vision-Language Models* (ACCV 2026, [arXiv:2607.23335](https://arxiv.org/abs/2607.23335)).
Paper label: **RMS prefix, ungated (run 3)**. Code and the reproduction scripts:
<https://github.com/mrfarazi/the-gate-always-closes>.

## What this is

Prefix design: K=4 tokens from the hyperbolic relational graph are spliced after the image patches; `variant.json` says whether the prefix is RMS-normalised and whether it is gated.

| field | value |
|---|---|
| kind | `prefix` |
| variant (`VARIANT_CONFIGS` key) | `rms_ungated` |
| backbone | `llava-hf/llava-1.5-7b-hf` (not redistributed; downloaded from its own repo) |
| learned gate | gate_raw = 0.0000, gate_tanh = 0.0000 |
| trainer | `see variant.json` |
| files | lora, prefix_modules.pt, variant.json |

`variant.json` records the configuration the graph module must be rebuilt with; the release code
reads it (`hra.checkpoints`). The `lora/` adapter is a standard PEFT adapter for the base model.

## Load

```python
from hra import hub
from hra.checkpoints import load_for_eval
model, processor, wrapper = load_for_eval("prefix", "rms_ungated_r3", "cuda")   # downloads this repo
```

## Behavioural knock-out (from the paper's result files)

_No knock-out file for this checkpoint in `results/knockouts/`._

## Licence and citation

Code and these adapter weights: MIT. The base model's own licence applies to the base weights
(llava-hf/llava-1.5-7b-hf). Please cite the paper:

```bibtex
@InProceedings{Farazi_2026_ACCV,
    author    = {Farazi, Moshiur and Ramasinghe, Sameera and Ciftler, Bekir Sait and Turza, Mahbub Ahmed and Rahman, Shafin},
    title     = {The Gate Always Closes: On Injecting Auxiliary Signals into Frozen Vision-Language Models},
    booktitle = {Proceedings of the Asian Conference on Computer Vision (ACCV)},
    month     = {December},
    year      = {2026}
}
```