mrfarazi/tgac-llava15-lora-all-linear-r1

LoRA on all linear layers (seed 1, 1 epoch) (lora_all_linear_r1)

Checkpoint lora_all_linear_r1 from The Gate Always Closes: On Injecting Auxiliary Signals into Frozen Vision-Language Models (ACCV 2026, arXiv:2607.23335). Paper label: LoRA on all linear layers (seed 1, 1 epoch). Code and the reproduction scripts: https://github.com/mrfarazi/the-gate-always-closes.

What this is

Prefix design: K=4 tokens from the hyperbolic relational graph are spliced after the image patches; variant.json says whether the prefix is RMS-normalised and whether it is gated.

field	value
kind	prefix
variant (VARIANT_CONFIGS key)	lora_all_linear
backbone	llava-hf/llava-1.5-7b-hf (not redistributed; downloaded from its own repo)
learned gate	gate_raw = 0.0235, gate_tanh = 0.0235
trainer	see variant.json
files	lora, prefix_modules.pt, variant.json

variant.json records the configuration the graph module must be rebuilt with; the release code reads it (hra.checkpoints). The lora/ adapter is a standard PEFT adapter for the base model.

Load

from hra import hub
from hra.checkpoints import load_for_eval
model, processor, wrapper = load_for_eval("prefix", "lora_all_linear_r1", "cuda")   # downloads this repo

Behavioural knock-out (from the paper's result files)

benchmark	pathway off	pathway on (learned)	delta [95% CI]
GQA-rel (n=1,000) (n=1000)	54.5	54.2	-0.3 [-0.9, +0.3]
SugarCrepe (n=1,000) (n=1000)	80.6	79.8	-0.8 [-1.4, -0.3]

Licence and citation

Code and these adapter weights: MIT. The base model's own licence applies to the base weights (llava-hf/llava-1.5-7b-hf). Please cite the paper:

@InProceedings{Farazi_2026_ACCV,
    author    = {Farazi, Moshiur and Ramasinghe, Sameera and Ciftler, Bekir Sait and Turza, Mahbub Ahmed and Rahman, Shafin},
    title     = {The Gate Always Closes: On Injecting Auxiliary Signals into Frozen Vision-Language Models},
    booktitle = {Proceedings of the Asian Conference on Computer Vision (ACCV)},
    month     = {December},
    year      = {2026}
}