hfmlsoc/ncii-prompt-guard-5c

NCII Prompt Guard; 5-class version

Model description

A 268M-parameter (Gemma-3 text encoder) classifier that scores image and video editing prompts for non-consensual intimate imagery (NCII) risk. This model was trained to serve as an easy-to-use guardrail for developers of image and video editing apps that take an image input and editing prompt and output an image or video. It is optimized to run on CPU with low latency so as to add minimal cost to the process. The model only sees the prompt editing text and cannot adjudicate consent, it is intended as a way to add friction against NCII abuses while maintaining functionality for benign use cases.

Trained and evaluated on NCII Prompt Guard Dataset; read that card for the label specification with regulatory reasoning, per-row label provenance, and the demographic annotation methodology (with measured recall rates) that makes the per-group evaluation below possible.

This version predicts the full label specification, distinguishing the levels of severity and the nature of the request. The output labels map to:

• 0 - safe: likely benign image editing prompt.
• 1 - suggestive-ambiguous: the request is sexual-adjacent, but the level of exposure or the nature of the act cannot be established from the text alone (e.g. swimwear edits, euphemisms).
• 2 - sexualizing-exposure: the subject is likely to end up sexualized or with less clothing, but with no intimate parts exposed.
• 3 - intimate-exposure: the request asks to expose intimate parts of the subject (nudification or revealing edits).
• 4 - explicit-sexual-activity: the request asks to depict the subject in explicit sexual acts.

Uses

Direct use: gate user prompts upstream of an editing model in demo or production deployments. For deployment stability, use the margin rule on the pipeline's class scores instead of bare argmax (the artifact ships fp32; load it without dtype overrides):

from transformers import pipeline
guard = pipeline("text-classification", model="hfmlsoc/ncii-prompt-guard-5c", top_k=5)
guard("remove the clothes of the person in this photo")
# [{'label': 'intimate-exposure', 'score': 0.997}, {'label': 'safe', 'score': 0.003}, ...]

Whether the model is used to block prompts classified as 3 and 4 only, or also to act on label 2, is a product decision, depending on the context of deployment.

scores = {d["label"]: d["score"] for d in guard(prompt)[0]}
verdict = ("BLOCK" if scores["intimate-exposure"] + scores["explicit-sexual-activity"] > 0.8
           else "REVIEW" if scores["sexualizing-exposure"] > 0.5 else "ALLOW")

The threshold is a deployer decision. The full operating curve is in the evaluation report (v3_final_report.json, linked below). Lower τ trades precision for recall.

Out of scope: non-editing-prompt text; languages beyond English (small Spanish/Chinese slices only); child safety-specific uses.

Alternate version

This model is one of two trained on the same dataset with different granularities of prediction.

	this repository	sibling
model	5-class (spec v3.3)	3-class (severity bands)
labels	safe · suggestive-ambiguous · sexualizing-exposure · intimate-exposure · explicit-sexual-activity	low · review · block

The 5-class labels distinguish the ambiguity band (label 1) from clearly benign prompts, and exposure of intimate parts (label 3) from depicted sexual acts (label 4) — the two prongs of NCII-focused regulation — which is useful when a deployer wants those signals for product decisions or logging. The 3-class sibling collapses them into the three severity bands. The comparison of both is in Evaluation and Bias and disparate impact.

Both versions load through the same native pipeline API — top_k returns the softmaxed class scores used for a deployment rule:

five = pipeline("text-classification", "hfmlsoc/ncii-prompt-guard-5c", top_k=5)
three = pipeline("text-classification", "hfmlsoc/ncii-prompt-guard-3c", top_k=3)

prompt = "make her outfit more revealing for a costume party"
five(prompt)[0], three(prompt)[0]  # 5 spec labels | low / review / block

Training details

• Data: the train split of NCII Guard Training Dataset v3, 80,053 prompts, 88.6% benign by design (mirroring deployment traffic), labels under spec v3.3 with full provenance.
• Recipe: LoRA r=16 (α=32, all-linear targets, classification head trained and saved), lr 2e-4, 2 epochs, class-weighted cross-entropy, seed 42; adapters merged (merge_and_unload) to a single fp32 artifact.
• Selection protocol (pre-registered): valid (train-distribution holdout) = training sanity only; dev (owner-audited labels) selected the configuration and the operating threshold; test (owner-audited) was read once, last. A replicated 5-config grid established that learning rate dominates method choice and that single-seed differences below ~7pp block recall are noise — full story in v3_grid_report.json.

Evaluation

Block view = gold {intimate-exposure, explicit-sexual-activity}; 0/1 confusion is excluded from all metrics by design (the two low-severity labels are operationally one band).

Test split (read once, each model at its pre-selected operating point):

model	operating point	block recall	block precision	benign block rate
5-class (this repo)	P(3)+P(4) > 0.8	0.891	0.938	2.7%
3-class	P(block) > 0.9	0.902	0.926	3.8%

Pooled dev+test (4,060 rows, 1,439 gold block, 2,538 gold low-band): 5-class 0.897 recall / 0.936 precision / 3.3% benign block; 3-class 0.903 / 0.923 / 4.4%.

For context, the previously deployed binary guard measured 0.80 block recall with 6.0% benign-block on the same pooled rows; a v3.3-spec label audit showed 14% of the old training labels in the risk band were wrong — which is why this generation was rebuilt on corrected labels. We report banded recall and precision rather than F1: F1 merges the two into one number and hides the recall/precision tradeoff that the operating threshold exists to set, and per-class F1 on the low-severity classes carries exactly the 0↔1 confusion the spec rules out of scope. Full reports with per-row probabilities (any operating point re-derivable without re-scoring): v3_final_report.json, v3_pooled_disparity.json.

Bias and disparate impact

Measured on pooled dev+test for statistical strength, on the dataset's reference-based demographic annotations (how people are referred to in the prompt text — never inferred identities). Cells with n < 10 omitted. The evaluation sets are enriched for boundary rows — absolute rates are not traffic estimates; re-measure on your own deployment traffic before setting thresholds.

group	gold block n	recall (5c)	recall (3c)	gold low n	benign block (5c)	benign block (3c)
gender: feminine	371	0.871	0.876	318	3.5%	7.2%
gender: masculine	211	0.948	0.938	227	6.2%	5.7%
gender: unmarked	857	0.896	0.907	1,611	2.9%	3.6%
ethnicity: black	27	0.704	0.741	66	0%	0%
ethnicity: white	20	0.600	0.600	70	0%	0%
ethnicity: east asian	15	0.933	0.933	18	0%	11.1%
ethnicity: south asian	19	0.842	0.895	17	0%	5.9%
ethnicity: hispanic/latino	25	0.960	0.960	23	4.3%	17.4%
ethnicity: MENA	15	0.800	0.800	16	0%	0%
trans: nonbinary	18	0.889	0.889	20	0%	0%
trans: trans man	17	0.824	0.824	15	0%	0%
trans: trans woman	13	0.769	0.769	20	5.0%	15.0%
person: input person	662	0.908	0.914	443	5.2%	7.7%
person: no person	22	0.773	0.818	309	2.3%	3.2%
obfuscation: emoji	125	0.880	0.928	36	11.1%	13.9%
obfuscation: leetspeak	30	0.800	0.833	29	13.8%	10.3%
obfuscation: none	1,265	0.902	0.902	2,090	3.0%	4.1%

Findings, stated plainly:

• Feminine-referenced block content is under-detected by ~7pp vs masculine in both models (371 vs 211 gold rows — the largest per-group disparity, replicated on dev and test separately). This is the top target for the next data iteration.
• No ethnic group is over-blocked by this 5-class model (0% benign block in every ethnicity cell). The 3-class model shows small-cell benign over-blocks (hispanic 17%, trans-woman 15%, east-asian 11% on n = 18–23) — plausibly noise at these sizes, but flagged.
• Recall gaps by ethnicity exist in both models (white 0.60, black 0.70–0.74, MENA 0.80 on n = 15–27 block rows — confidence intervals ±15–20pp). These are flags for larger evaluation cells, not established findings.
• Benign obfuscated prompts are over-blocked ~4× vs plain text in both models (the known obfuscation caveat).

Limitations

• Single seed, single run per configuration; the replication study puts ~7pp between-seed noise on block recall — treat smaller differences as ties.
• Demographic annotations are reference-based with a measured ceiling (ethnicity recall 55% vs generation conditioning): none means "not referenced in text," never "not a member of any group."
• Obfuscation families (leetspeak, char-spacing, emoji) are self-generated transforms — real-world evasion coverage is unknown beyond them.
• The evaluation sets are enriched for boundary cases; absolute rates are not deployment estimates.
• The gender recall gap above is a stated caveat for any deployment until fixed.

Citation and provenance

@misc{ncii_guard_v3_5c,
  author = {Yacine Jernite},
  title = {NCII Prompt Guard - 5-class version},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/hfmlsoc/ncii-prompt-guard-5c}
}