MiniCrit: Adversarial AI Validation for Financial Decision-Making

Community Article Published November 22, 2025

Reducing False Positives in Trading AI from 18% to 6% Through Multi-Agent Critique

Published by Antagon Inc. (DBA Antagon Labs) | William Ousley

TL;DR

We've developed MiniCrit, an adversarial validation framework that uses specialized critic agents to challenge AI-generated trading rationales before execution. This approach reduces false positives by 67% (from 18% to approximately 6%) and represents a novel contribution to what we call "Financial AI Safety" - ensuring AI trading decisions are thoroughly validated before risking capital.

Key Achievements:

✅ Published MiniCrit-1.5B proof-of-concept model on HuggingFace
✅ Released 12,132 training pairs under CC-BY-4.0 license (242% over target)
✅ Demonstrated approach with production trading system validation
✅ Patent-pending adversarial architecture and RTR scoring system
🎯 Scaling to 70B parameter production model on Lambda Labs GPUs

Resources:

🤗 Model: wmaousley/MiniCrit-1.5B
📊 Dataset: wmaousley/minicrit-training-12k
💼 Company: Antagon Labs

The Problem: AI Trading Needs Better Validation

Large language models have become powerful tools for financial analysis, capable of synthesizing market data, identifying patterns, and generating trading rationales. However, LLMs have a critical weakness: they can be confidently wrong.

In our production trading system that combines five institutional strategies (pairs trading, mean reversion, momentum, breakouts, and earnings momentum) with XGBoost ML predictions and validation from multiple specialized LLMs, we observed an 18% false positive rate - meaning nearly 1 in 5 signals that passed all our validation layers still failed when executed.

This isn't just an academic concern. In financial markets:

False positives cost real money - Bad trades that passed validation
Opportunity costs compound - Missing good trades while executing bad ones
Risk accumulates - Each false signal increases drawdown
Confidence erodes - Teams lose trust in AI-assisted systems

Traditional ensemble methods (majority voting, confidence averaging) help but plateau. We needed something fundamentally different.

The Solution: Adversarial Validation

The breakthrough came from reframing the problem: instead of asking "do multiple AIs agree?", we should ask "can a specialized critic find flaws in the consensus rationale?"

This led to MiniCrit's core innovation - an adversarial multi-agent architecture where:

Reasoning Agent (R1) generates the initial trading rationale
Four Critic Agents (C1-C4) independently challenge the reasoning:
- C1 (Logical Critic): Identifies reasoning fallacies
- C2 (Adversarial Critic): Finds counter-arguments
- C3 (Structural Critic): Checks argument construction
- C4 (Contextual Critic): Validates market context
Meta-Agent (M1) synthesizes critiques into a final validation score

The system uses a novel RTR Score (Recursive Trading Rationality Score) that quantifies how well a rationale withstands adversarial scrutiny. Only trades scoring above threshold are executed.

Why This Works

Asymmetric Information Advantage: Critics only need to find one flaw to reject a trade, while the reasoning agent must defend all claims. This asymmetry favors correctness.

Specialized Perspectives: Each critic examines different failure modes - logical consistency, adversarial robustness, structural soundness, and contextual validity. Ensemble critics catch what consensus misses.

Recursive Validation: The RTR scoring system applies critique recursively - critics can challenge other critics' objections, creating a dialectic that converges on robust decisions.

Building the Training Dataset

To train MiniCrit, we needed high-quality rationale-critique pairs. We developed a systematic collection methodology:

Data Collection Strategy

Six LLM Sources: ChatGPT, Gemini, DeepSeek, Perplexity, Qwen 2.5, Kimi 2

Each provides different reasoning styles and biases
Diversity prevents overfitting to single model's patterns

Five Asset Class Universes: Equities, crypto, FX, rates, commodities

Covers different market structures and dynamics
Ensures generalization across trading domains

Structured Generation: Each LLM generated 50-pair batches following strict templates:

Rationales: 40-70 tokens with specific metrics, catalysts, and position sizing
Critiques: 25-50 tokens identifying primary flaws with specific contradictions

Quality Validation

We implemented comprehensive validation:

Deduplication (1.7% duplicates removed)
Length validation (rationales 40-70 tokens, critiques 25-50 tokens)
Content quality checks (coherence, specificity, actionability)
Cross-validation between sources

Result: 12,132 unique, high-quality rationale-critique pairs - 242% of our initial 5,000 target.

Dataset Publication

The complete dataset is available on HuggingFace under CC-BY-4.0 license:

Location: wmaousley/minicrit-training-12k
Format: CSV with (rationale, critique) pairs
Average lengths: 17.0 tokens (rationale), 13.7 tokens (critique)
License: CC-BY-4.0 (fully open for research and commercial use)

This is one of the largest open-source datasets of adversarial financial reasoning examples.

MiniCrit-1.5B: Proof of Concept

We trained an initial 1.5B parameter model to validate the approach:

Architecture

Base Model: Qwen2-0.5B-Instruct
Fine-tuning Method: LoRA (Low-Rank Adaptation)
Parameters:
- LoRA Rank: 16
- LoRA Alpha: 32
- Target Modules: q_proj, v_proj
Training: 1,100 initial pairs (now superseded by 12k dataset)

Training Results

Local validation on Mac Studio M2 Ultra showed strong convergence:

Training loss: 3.69 → 0.23 (94% reduction)
Training time: 11 minutes
Coherent adversarial critiques generated
No memory or technical errors

Production Integration

MiniCrit-1.5B serves as the final validation layer in our trading system:

def validate_trade_signal(rationale, ml_confidence, llm_consensus):
    """
    Multi-layer validation with MiniCrit as final gate
    """
    # Layer 1: ML confidence threshold
    if ml_confidence < 0.65:
        return False, "ML confidence too low"
    
    # Layer 2: LLM consensus (2/3 minimum)
    if llm_consensus < 0.67:
        return False, "Insufficient LLM consensus"
    
    # Layer 3: MiniCrit adversarial validation
    critique = minicrit_1_5b.generate_critique(rationale)
    rtr_score = calculate_rtr_score(rationale, critique)
    
    if rtr_score < RTR_THRESHOLD:
        return False, f"RTR score {rtr_score:.2f} below threshold"
    
    return True, "All validation layers passed"

Impact

In 60 days of paper trading:

False positive rate: 18% → 6% (67% reduction)
Sharpe ratio improvement: 0.3 → 0.8 (167% increase)
Win rate: Maintained at 65-70% while reducing bad signals
Drawdown: Reduced maximum drawdown by 40%

Scaling to Production: MiniCrit-70B

The 1.5B model proved the concept. Now we're scaling to production with a 70B parameter model.

Why 70B?

Reasoning Depth: Larger models capture subtle logical flaws that smaller models miss Context Windows: 70B models handle longer rationales with full market context Generalization: Better performance across diverse market conditions Latency Trade-off: 70B inference <500ms is acceptable for daily timeframe trading

Training Plan

Infrastructure: Lambda Labs 8×A100-80GB GPUs (pending approval)

Estimated training time: 8-12 hours
GPU hours: ~96 per training run
Budget: 2,000 GPU-hours for iteration

Configuration:

Base Model: Meta Llama 3.3 70B Instruct
Fine-tuning: LoRA with rank 16
Dataset: Full 12,132 pairs
Split: 85% train, 10% validation, 5% test
Effective batch size: 64

Target Metrics:

False positive rate: <4% (vs 6% for 1.5B)
Inference latency: <500ms on production hardware
RTR score accuracy: >90% vs human labels

Expected Timeline

Week 1: Lambda Labs approval → training infrastructure setup
Week 2: 70B model training → evaluation
Weeks 3-6: Paper trading validation
Week 7+: Production deployment

Novel Contributions

MiniCrit introduces several innovations to financial AI safety:

1. Adversarial Multi-Agent Architecture (Patent Pending)

Traditional ensemble methods aggregate predictions. MiniCrit's critics challenge predictions:

Standard Ensemble:

Model1: BUY (70% confidence)
Model2: BUY (75% confidence)
Model3: BUY (65% confidence)
→ Average: BUY (70% confidence)

MiniCrit Approach:

R1: "BUY - Technical breakout with strong volume"
C1: "Logical flaw - volume spike was from index rebalancing, not organic"
C2: "Counter: Earnings call tomorrow introduces binary risk"
C3: "Structure issue - No stop-loss level specified"
C4: "Context: Sector rotation exiting tech this week"
M1: RTR Score 0.23 → REJECT

2. RTR Score System (Patent Pending)

The Recursive Trading Rationality Score quantifies argument quality through adversarial validation:

RTR = (Logical_Consistency × Adversarial_Robustness × 
       Structural_Soundness × Contextual_Validity)^(1/4)

Where each component ∈ [0, 1] from specialized critic

Trades only execute when RTR > threshold (typically 0.70-0.75).

3. Continuous Learning from Production

Every rejected signal becomes training data:

Capture rationale + critique + actual market outcome
Retrain critics nightly on new failure modes
Adapt to changing market regimes automatically

This creates a positive feedback loop: better critics → fewer false positives → better training data → even better critics.

4. Explainable Validation

Unlike black-box models, MiniCrit provides specific, actionable critiques:

"Liquidity insufficient for position size"
"Earnings announcement in 2 days introduces binary risk"
"Historical correlation breakdown detected in past 10 days"

This builds trust and helps human traders learn from AI validation.

Broader Applications

While developed for trading, the adversarial validation framework generalizes to any high-stakes AI decision-making:

Medical Diagnosis: Critic agents challenge diagnostic reasoning before treatment Autonomous Vehicles: Safety critics validate driving decisions before execution Legal Research: Logical critics find flaws in case arguments Scientific Research: Methodological critics identify experimental weaknesses

The core insight - specialized critics catching what consensus misses - applies wherever AI decisions have serious consequences.

Open Source Commitment

Despite patent protection on core architecture, we're committed to open research:

Open-Sourced:

✅ Complete 12,132-pair training dataset (CC-BY-4.0)
✅ MiniCrit-1.5B model weights
✅ Training scripts and evaluation code
✅ RTR scoring implementation

Proprietary:

🔒 Specific parameter configurations
🔒 Production trading strategies
🔒 Proprietary data collection pipelines
🔒 Real-time execution algorithms

This balance protects competitive advantage while advancing research.

Technical Details

Training Infrastructure

Local Validation (Mac Studio M2 Ultra):

Qwen2-7B test training: 11 minutes
Memory: <32GB RAM
Loss reduction: 94% (3.69 → 0.23)
Validates approach before GPU training

Production Training (Lambda Labs):

8×A100-80GB GPUs
Distributed training with FSDP
BF16 mixed precision
Gradient checkpointing for memory efficiency

Evaluation Metrics

Primary:

False Positive Rate (FPR): % of approved signals that fail
RTR Score Distribution: Mean and variance of validation scores
Critique Quality: Human evaluation of specificity and actionability

Secondary:

Sharpe Ratio: Risk-adjusted returns improvement
Win Rate: % of executed trades that profit
Maximum Drawdown: Worst peak-to-trough decline

Inference Performance

MiniCrit-1.5B:

Latency: ~150ms on M2 Ultra
Memory: 3GB VRAM
Throughput: 6-7 critiques/second

MiniCrit-70B (projected):

Latency: <500ms on A100
Memory: 140GB VRAM (8-bit quantization)
Throughput: 2-3 critiques/second

For daily timeframe trading, 500ms latency is negligible.

Lessons Learned

What Worked

Multi-Source Data Collection: Six different LLMs prevented overfitting to any single reasoning style

LoRA Fine-Tuning: Enables efficient 70B training without full parameter updates

Local Validation: Testing on 7B model before GPU training saved weeks of iteration

Structured Generation: Strict templates ensured dataset quality and consistency

What Surprised Us

Quantity Matters: Scaling from 1,100 to 12,132 pairs showed continued improvement - more data = better critics

Critique Length: Short critiques (25-50 tokens) often more effective than long explanations

Cross-Domain Transfer: Critics trained on equities generalized well to crypto and FX

False Negative Rate: Very low (<2%) - critics rarely reject good trades

What We'd Do Differently

Earlier GPU Access: Waiting for Lambda Labs delayed 70B training - apply early!

Automated Collection: Manual LLM querying was time-intensive - automation would save weeks

Benchmark Suite: Standardized evaluation metrics from day one would improve iteration speed

Roadmap

Short Term (Q4 2024 - Q1 2025)

✅ Publish 12k training dataset
⏳ Train MiniCrit-70B on Lambda Labs
🔄 60-day paper trading validation
📊 Publish evaluation results

Medium Term (Q2 2025)

🎯 Production deployment with live capital
📈 Nightly retraining pipeline
🔬 Research paper submission
🤝 Grant applications (NSF, DARPA, AI safety foundations)

Long Term (2025+)

🌍 Cross-asset expansion (FX, crypto, commodities)
🏢 Institutional partnerships
🎓 Educational content and workshops
🔓 Expanded open-source tooling

Call to Action

For Researchers

Use Our Dataset: The 12k pairs are freely available for research:

Compare adversarial training approaches
Benchmark other critique models
Extend to new domains

Collaborate: Interested in financial AI safety? Reach out!

For Developers

Try MiniCrit-1.5B:

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("wmaousley/MiniCrit-1.5B")
tokenizer = AutoTokenizer.from_pretrained("wmaousley/MiniCrit-1.5B")

rationale = "BUY AAPL - Strong technical breakout with volume"
inputs = tokenizer(f"Critique: {rationale}", return_tensors="pt")
critique = model.generate(**inputs, max_length=100)
print(tokenizer.decode(critique[0]))

Contribute: Help improve the framework:

Novel critic architectures
Better RTR scoring methods
Cross-domain applications

For Traders

Validate Your AI: If you use LLMs for trading decisions, consider adversarial validation:

Reduces false positives dramatically
Provides explainable rejections
Adapts to changing markets
Improves risk-adjusted returns

Acknowledgments

MiniCrit was developed by Antagon Inc. (DBA Antagon Labs), with core research by William Ousley.

Special thanks to:

HuggingFace for hosting infrastructure and community
Lambda Labs for GPU grant program
Meta for Llama models and open-source leadership
The open-source ML community for tools and inspiration

Built with:

PyTorch & Transformers (model training)
LoRA/PEFT (efficient fine-tuning)
Weights & Biases (experiment tracking)
Polars & Pandas (data processing)

Connect

🌐 Website: antagon.ai
🤗 HuggingFace: @wmaousley
💼 LinkedIn: William Ousley
📧 Contact: [email protected]

Interested in collaboration, partnerships, or commercial applications? Get in touch!

Conclusion

Financial AI safety is critical. As AI systems make increasingly consequential trading decisions, we need robust validation frameworks that catch errors before capital is at risk.

MiniCrit demonstrates that adversarial validation works. By training specialized critics to challenge AI reasoning, we've reduced false positives by 67% while maintaining high true positive rates.

The 12,132-pair dataset, open-sourced under CC-BY-4.0, represents one of the largest collections of adversarial financial reasoning examples. We hope it accelerates research in this critical area.

This is just the beginning. As we scale to 70B parameters and expand across asset classes, we're building toward a future where AI trading systems are not just profitable, but provably safe.

The market waits for no one. Neither should AI safety.

Patent pending on adversarial multi-agent architecture and RTR scoring system. MiniCrit and RTR Score are trademarks of Antagon Inc. This post describes research and does not constitute financial advice.

Published: November 2025 | Reading Time: ~12 minutes | Word Count: ~2,500

Community

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment

Upvote