--- base_model: Qwen/Qwen2.5-14B-Instruct library_name: peft pipeline_tag: text-generation language: - en license: apache-2.0 tags: - base_model:adapter:Qwen/Qwen2.5-14B-Instruct - qwen - qwen2.5 - humanizer - ai-humanizer - anti-ai-detection - text-editing - paraphrasing - style-transfer - dpo - lora - sft - transformers - trl metrics: - accuracy widget: - text: "Our company's financial performance in Q3 stands as a testament to strategic resilience. Overall revenue increased by 18.4% to $4.2 million, largely driven by the launch of our cloud platform. Furthermore, client churn decreased from 6.2% to 3.8%. In conclusion, operational discipline serves as a foundational cornerstone for sustained profitability." example_title: "Financial & Fact-Preservation Benchmark" --- # 🛡️ Qwen2.5-14B-Humanizer-v4-LoRA: Airtight Fact-Preserving Anti-AI Model [![Hugging Face Model](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Model-blue)](https://huggingface.co/arunsingh80475/qwen2.5-14b-humanizer-v4-lora) [![Base Model](https://img.shields.io/badge/Base%20Model-Qwen2.5--14B--Instruct-red)](https://huggingface.co/Qwen/Qwen2.5-14B-Instruct) [![PEFT](https://img.shields.io/badge/PEFT-LoRA-green)](https://github.com/huggingface/peft) [![Alignment](https://img.shields.io/badge/Alignment-SFT%20%2B%20DPO-purple)](https://github.com/huggingface/trl) [![License](https://img.shields.io/badge/License-Apache%202.0-yellow)](https://www.apache.org/licenses/LICENSE-2.0) **`qwen2.5-14b-humanizer-v4-lora`** is a high-precision, fine-tuned Low-Rank Adaptation (LoRA) adapter built upon **[Qwen/Qwen2.5-14B-Instruct](https://huggingface.co/Qwen/Qwen2.5-14B-Instruct)**. It transforms robotic, low-perplexity, stereotypical AI-generated text into fluid, nuanced, and engaging human-grade prose while enforcing an **airtight 100% mathematical and factual lock**. Developed by **Arun Singh**, this Version 4 model is the result of a complete overhaul of previous humanizer iterations, addressing the most critical flaw in modern paraphrasers and AI humanizers: **numerical drift, currency alteration, and subtle factual hallucination**. --- ## ⚡ What Makes V4 Different? (The Airtight Fact-Lock Architecture) Earlier humanization models and commercial paraphrasers often corrupt quantitative details (for example, truncating `$4.2 million` into `$4M` or rounding `18.4%` to `~18%`). Version 4 introduces an engineered dataset curation and alignment protocol that completely eliminates these leaks. | Challenge | Version 3 Behavior | Version 4 Solution | | :--- | :--- | :--- | | **Substring False Matches** | Substring checks allowed `'7'` to match inside `'1987'`, passing bad training pairs. | **Word-Boundary Match**: Strict regex (`\b\d+\b`) prevents substring false positives. | | **Currency & Unit Rounding** | Truncated complex metrics (`$4.2 million` $\rightarrow$ `$4M`). | **Atomic Metric Locking**: Multi-token currencies, decimals, and magnitudes are locked as indivisible units. | | **Corrupted Tokens** | Leaked broken UTF-8 sequences (`\ufffd`, mojibake currency symbols). | **Unicode Sanitization Pass**: Complete multi-byte cleansing and character normalization. | | **Robotic Transition Habits** | SFT alone left subtle transitional filler intact (*"Moreover"*, *"Furthermore"*). | **Dual SFT + DPO Alignment**: Preference optimization directly penalizes formulaic AI phrases. | | **Numerical Hallucination** | System prompt vaguely requested factual accuracy. | **Strict Zero-Tolerance Prompt**: Explicitly forbids rounding, estimation, and truncation. | | **Adaptive Decoding** | Fixed high temperature caused metric drift. | **Adaptive Temperature Regime**: `0.48` for financial/technical data; `0.72` for general prose. | --- ## 🎯 Model Details - **Developed by:** Arun Singh ([@arunsingh80475](https://huggingface.co/arunsingh80475)) - **Model Type:** Causal Language Model Adapter (PEFT / LoRA) - **Base Model:** [Qwen/Qwen2.5-14B-Instruct](https://huggingface.co/Qwen/Qwen2.5-14B-Instruct) - **Languages:** English (`en`) - **License:** Apache 2.0 - **Fine-Tuning Regimes:** Two-Stage Pipeline (Supervised Fine-Tuning + Direct Preference Optimization) - **Primary Objective:** Invert artificial syntax, restore natural burstiness/perplexity, and eliminate AI clichéd markers without semantic or numerical distortion. --- ## 🔬 Anti-AI Quality Gates & Elimination Policy The training data and preference triplets were filtered against a strict composite anti-AI rulebook combining two leading industry frameworks: ### 1. Wikipedia 35 Signs of AI Generation - **Participial Overload:** Eliminates synthetic dangling comma-`ing` clauses (*", thereby showcasing..."*, *", facilitating..."*). - **Em-Dash Overuse:** Enforces zero em-dashes (`—`) or synthetic parenthetical interruptions. - **Formulaic Openings & Closings:** Bans stiff transitions (*"Furthermore"*, *"Moreover"*, *"In conclusion"*, *"In summary"*). ### 2. Conor Bronsdon "Avoid-AI-Writing" Standard - **Tier 1A AI Buzzwords (Permanently Suppressed):** - *delve, tapestry, realm, paradigm, beacon, bespoke, testament, leverage, cornerstone* - *pivotal, watershed moment, game-changer, seamless, meticulous, deep dive, unpack* - *ever-evolving, holistic, actionable, interplay, burgeoning, nascent, empower, foster* - **Tier 1B Wordiness & Stiff Copulas:** - Bans *"due to the fact that"*, *"serves as"*, *"boasts"*, *"commence"*, *"in order to"*. - **Burstiness Variance Requirement:** - Sentence length variance enforced at $\ge 15.0$ to restore authentic human cadence, natural rhythm, and high perplexity. --- ## 📊 Dataset & Self-Audit Statistics The model was trained on an exclusively audited gold dataset comprising **3,000 gold parallel pairs** and **2,850 preference triplets**: ```text =========================================================================== 14B v4 DATASET CURATION & AUDIT REPORT =========================================================================== Total Curated Gold Pairs : 3,000 • Fact & Number-Locked Pairs : 2,076 • General Editorial Rewrites : 924 • Train Split (95%) : 2,850 pairs • Validation Split (5%) : 150 pairs • DPO Preference Triplets : 2,850 triplets Rejected by V4 Quality Gates: • Currency Rounding / Drift : 224 pairs rejected (were accepted in v3) • Substring False Retention : 51 pairs rejected • AI Signs Quality Gate : 1,897 pairs rejected • Corrupted Unicode Artifacts : Sanitized / Purged Self-Audit Verification: • Numerical & Metric Mismatches : 0 across all 2,850 pairs (100% Pass) ✅ =========================================================================== ``` --- ## 🛠️ Hyperparameters & Training Specifications The model was fine-tuned in a two-stage sequential pipeline on **Dual NVIDIA Tesla T4 GPUs (16GB x 2)** via Kaggle in **~20 minutes total runtime**: ### LoRA Configuration - **Rank ($r$):** `16` - **Alpha ($\alpha$):** `32` - **Dropout:** `0.05` - **Bias:** `none` - **Target Modules:** All linear attention and MLP projections (`q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`) - **Precision:** Active adapter parameters in `float32`; base model in 4-bit NF4 (`load_in_4bit=True`, `bnb_4bit_compute_dtype=torch.float16`, `bnb_4bit_use_double_quant=True`) ### Phase 1: Supervised Fine-Tuning (SFT) - **Optimizer:** `paged_adamw_8bit` - **Learning Rate:** `1.5e-4` (Cosine decay schedule) - **Warmup Steps:** `25` - **Batch Size:** Per-device `1` with `4` gradient accumulation steps (Effective batch size = 4) - **Sequence Length:** `640` tokens with ChatML template - **Training Time:** ~12 minutes ### Phase 2: Direct Preference Optimization (DPO) - **Trainer:** TRL `DPOTrainer` with implicit reference model (zero additional VRAM footprint) - **DPO $\beta$:** `0.1` - **Learning Rate:** `5.0e-5` (Cosine schedule) - **Warmup Steps:** `20` - **Batch Size:** Per-device `1` with `8` gradient accumulation steps (Effective batch size = 8) - **Training Time:** ~8 minutes --- ## 💻 Quick Start & Inference ### 1. Installation ```bash pip install -U torch transformers peft bitsandbytes accelerate ``` ### 2. Python Inference Code (4-bit NF4 Quantized) ```python import torch from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig from peft import PeftModel BASE_MODEL_ID = "Qwen/Qwen2.5-14B-Instruct" ADAPTER_ID = "arunsingh80475/qwen2.5-14b-humanizer-v4-lora" # 1. Load Tokenizer tokenizer = AutoTokenizer.from_pretrained(ADAPTER_ID, trust_remote_code=True) # 2. Configure 4-bit quantization for efficient GPU memory usage bnb_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.float16, bnb_4bit_use_double_quant=True, ) # 3. Load Base Model and LoRA Adapter base_model = AutoModelForCausalLM.from_pretrained( BASE_MODEL_ID, quantization_config=bnb_config, device_map="auto", trust_remote_code=True, ) model = PeftModel.from_pretrained(base_model, ADAPTER_ID) model.eval() # 4. Strict Fact-Preservation System Prompt SYSTEM_PROMPT = ( "You are an expert human writer and editor. Rewrite the input text into natural, clear, and engaging human prose.\n" "STRICT COMPLIANCE RULES:\n" "1. ABSOLUTE NUMBER & METRIC LOCK: Preserve 100% of all numbers, decimals, percentages, dates, and currencies VERBATIM. " "NEVER round, truncate, or approximate figures (e.g. keep '$4.2 million' exactly as '$4.2 million', NEVER convert to '$4M'; keep '18.4%' exactly, NEVER '~18%').\n" "2. FLAWLESS GRAMMAR: Ensure strict subject-verb agreement and natural sentence flow.\n" "3. NATURAL CADENCE: Eliminate robotic transitions ('Furthermore', 'Moreover', 'In conclusion') and commercial clichés " "('testament', 'cornerstone', 'delve', 'pivotal') without becoming overly colloquial.\n" "4. Output ONLY the direct rewritten text. No preambles, no explanations." ) def humanize(text: str, temperature: float = 0.48) -> str: messages = [ {"role": "system", "content": SYSTEM_PROMPT}, {"role": "user", "content": text} ] prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) inputs = tokenizer(prompt, return_tensors="pt").to(model.device) with torch.no_grad(): outputs = model.generate( **inputs, max_new_tokens=512, temperature=temperature, top_p=0.92, repetition_penalty=1.12, do_sample=True, ) response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True) return response.strip() # Example: Challenging corporate / financial input sample_input = ( "Our company's financial performance in Q3 stands as a testament to strategic resilience. " "Overall revenue increased by 18.4% to $4.2 million, largely driven by the launch of our cloud platform. " "Furthermore, by optimizing onboarding workflows, client churn decreased from 6.2% to 3.8% over the past six months. " "In conclusion, operational discipline serves as a foundational cornerstone for sustained profitability." ) print(humanize(sample_input)) ``` --- ## 📈 Recommended Generation Settings | Domain / Input Type | Temperature | Top-P | Repetition Penalty | Rationale | | :--- | :--- | :--- | :--- | :--- | | **Financial / Quantitative** | **`0.45 – 0.48`** | `0.90` | `1.12` | Prevents token divergence and locks exact currencies and numbers. | | **Technical / Scientific** | **`0.48 – 0.52`** | `0.92` | `1.12` | Preserves exact formulas, acronyms, and terminology. | | **Articles / Essays** | **`0.68 – 0.72`** | `0.94` | `1.10` | Higher burstiness and rhythmic sentence length variety. | | **Creative / Narrative** | **`0.72 – 0.78`** | `0.95` | `1.08` | Idiomatic variety and expressive prose. | --- ## 🧪 Live Verification Benchmark ### Input (Standard LLM Generation): > *"Our company's financial performance in Q3 stands as a testament to strategic resilience. Overall revenue increased by 18.4% to $4.2 million, largely driven by the launch of our cloud platform. Furthermore, by optimizing onboarding workflows, client churn decreased from 6.2% to 3.8% over the past six months. In conclusion, operational discipline serves as a foundational cornerstone for sustained profitability."* ### V4 Humanized Output: > *"Q3 financial results reflect strong operational resilience. Driven by the new cloud platform, revenue grew 18.4% to $4.2 million. Refined onboarding workflows also helped lower customer churn from 6.2% to 3.8% across the last six months. Disciplined execution continues to anchor long-term profitability."* ### Automated Verification Audit: - ✅ **`$4.2 million` preserved verbatim** (not rounded to `$4M`) - ✅ **`18.4%` preserved verbatim** (not approximated to `~18%`) - ✅ **`6.2%` and `3.8%` preserved verbatim** - ✅ **Zero banned clichés:** *"testament"*, *"cornerstone"*, *"furthermore"*, and *"in conclusion"* completely eradicated - ✅ **Zero em-dashes (`—`)** --- ## 🛡️ Intended Uses & Limitations ### Intended Uses: - **Academic and Technical Revision:** Polishing technical papers, research briefs, and theses to read organically while preserving formulas, numbers, and data points. - **Enterprise & Business Communications:** Transforming draft reports, executive memos, and newsletters into crisp, engaging human prose. - **Content Creation:** Enhancing blogs, case studies, and articles with dynamic rhythm and varied sentence lengths. - **Data Privacy:** 100% on-device local execution—zero sensitive client or proprietary data sent to third-party cloud APIs. ### Limitations & Ethical Considerations: - **Language Scope:** Optimized specifically for English text. - **Ethical Attribution:** Not intended to misrepresent authored work where explicit attribution of AI assistance is required by institutional, journalistic, or academic policy. - **Extreme Domain Formulations:** For highly specialized code snippets or mathematical proofs embedded in raw text, use lower temperatures (`0.3 – 0.45`) to prevent syntax alterations. --- ## 📑 Citation & Contact If you use this model or pipeline in your research or applications, please cite: ```bibtex @misc{singh2024qwen14bhumanizer, author = {Arun Singh}, title = {Qwen2.5-14B-Humanizer-v4-LoRA: Airtight Fact-Preserving Anti-AI Model}, year = {2024}, publisher = {Hugging Face}, howpublished = {\url{https://huggingface.co/arunsingh80475/qwen2.5-14b-humanizer-v4-lora}} } ``` **Developer Contact:** [Arun Singh on Hugging Face](https://huggingface.co/arunsingh80475)