Gemma 4 Technical Report
Paper • 2607.02770 • Published • 76
AI & ML interests
Open science and open source
Recent Activity
View all activity
osanseviero
authored a
paper 29 days ago
osanseviero
submitted a
paper to Daily Papers about 1 month ago
albertvillanova
posted an update about 1 month ago
Post
3676
🎉 KTO is now part of the stable TRL API
As of Promote KTO to stable API, KTOTrainer and KTOConfig have graduated from trl.experimental to the stable trl API. https://github.com/huggingface/trl/pull/6175
This one closes out a long road. Over the past 6+ months, the "Align KTO with DPO" effort landed ~90 PRs methodically bringing KTO up to the standard we hold for stable trainers, one carefully-scoped change at a time:
- Feature parity with DPO: full VLM support (incl. multi-image), sync_ref_model, PEFT + Liger, ZeRO-3 + PEFT dtype fix, pad_to_multiple_of, activation offloading, IterableDataset and dict eval_dataset, remove_unused_columns, and reference-logprob precomputation at init.
- Consistency with DPO: aligned method order and signatures, tokenization, _prepare_dataset, PEFT handling, ref-model preparation for distributed training, and config layout — plus a new DataCollatorForKTO and output format. Metrics moved into _compute_loss and simplified to direct averages via the shared _metrics attribute.
- Removing legacy baggage: dropped encoder-decoder support, BOS/EOS handling, null_ref_context, generate_during_eval, model_init, preprocess_logits_for_metrics, model/ref adapter names, and several dead config knobs.
- Coverage: a full test suite mirroring DPO, text collator tests, VLM tests, and slow tests.
- The promotion itself: the experimental → stable move (#6175) and shim cleanup (#6287), handled so downstream users get a clean deprecation path.
Honestly, this has been one of the more complex tasks I've taken on since joining the team, not because any single change was hard, but because it demanded sustained consistency across a ~2,000-line trainer, with every branch, comment, and edge case kept in lockstep with DPO.
Huge thanks to everyone who reviewed along the way (especially @qgallouedec ), the incremental review cadence is exactly what kept this maintainable.
KTO now sits on equal footing with our other flagship trainers. 🚀
As of Promote KTO to stable API, KTOTrainer and KTOConfig have graduated from trl.experimental to the stable trl API. https://github.com/huggingface/trl/pull/6175
This one closes out a long road. Over the past 6+ months, the "Align KTO with DPO" effort landed ~90 PRs methodically bringing KTO up to the standard we hold for stable trainers, one carefully-scoped change at a time:
- Feature parity with DPO: full VLM support (incl. multi-image), sync_ref_model, PEFT + Liger, ZeRO-3 + PEFT dtype fix, pad_to_multiple_of, activation offloading, IterableDataset and dict eval_dataset, remove_unused_columns, and reference-logprob precomputation at init.
- Consistency with DPO: aligned method order and signatures, tokenization, _prepare_dataset, PEFT handling, ref-model preparation for distributed training, and config layout — plus a new DataCollatorForKTO and output format. Metrics moved into _compute_loss and simplified to direct averages via the shared _metrics attribute.
- Removing legacy baggage: dropped encoder-decoder support, BOS/EOS handling, null_ref_context, generate_during_eval, model_init, preprocess_logits_for_metrics, model/ref adapter names, and several dead config knobs.
- Coverage: a full test suite mirroring DPO, text collator tests, VLM tests, and slow tests.
- The promotion itself: the experimental → stable move (#6175) and shim cleanup (#6287), handled so downstream users get a clean deprecation path.
Honestly, this has been one of the more complex tasks I've taken on since joining the team, not because any single change was hard, but because it demanded sustained consistency across a ~2,000-line trainer, with every branch, comment, and edge case kept in lockstep with DPO.
Huge thanks to everyone who reviewed along the way (especially @qgallouedec ), the incremental review cadence is exactly what kept this maintainable.
KTO now sits on equal footing with our other flagship trainers. 🚀
- 2 replies
Shrijanagain
posted an update about 1 month ago
Post
213
Welcome Researcher and Developers!
SKT AI Labs, we are pushing the boundaries of AI architecture and research—and today, we are thrilled to open our doors to the global research community!
We warmly welcome researchers, developers, and AI enthusiasts to join us and contribute to our R&D efforts.
🧪 What You Can Explore:
We invite you to experiment with our WMF (Weight Manifold Fusion) technology. You can test this high-dimensional fusion technique on smaller models to gain a deeper understanding of its behavior and token convergence.
---------- CHECK OUT:
SPACE : SKT-NRS/RD
EXPERIMENT : https://huggingface.co/sKT-Ai-Labs/SKT-SURYA-H
DIRECT TO MAIN DISCUSSION : SKT-NRS/RD#1
🤝 Your Feedback Shapes the Future :
If it works: Fantastic! Share your results with us and contribute directly to the core vision of SKT AI Labs.
If it doesn't work: No problem at all! Your critical feedback is just as valuable to us. Every experiment and anomaly helps us refine this architecture to make it more stable and robust.
We firmly believe that true innovation stems from community collaboration and transparent testing. Let's build the future of advanced AI together. Your ideas, test results, and feedback are always welcome!
You Can Still Research and Development On WMF Only SKT-SURYA-H Model is Dismissed.
Let's innovate and build together! 💡
SKT AI Labs, we are pushing the boundaries of AI architecture and research—and today, we are thrilled to open our doors to the global research community!
We warmly welcome researchers, developers, and AI enthusiasts to join us and contribute to our R&D efforts.
🧪 What You Can Explore:
We invite you to experiment with our WMF (Weight Manifold Fusion) technology. You can test this high-dimensional fusion technique on smaller models to gain a deeper understanding of its behavior and token convergence.
---------- CHECK OUT:
SPACE : SKT-NRS/RD
EXPERIMENT : https://huggingface.co/sKT-Ai-Labs/SKT-SURYA-H
DIRECT TO MAIN DISCUSSION : SKT-NRS/RD#1
🤝 Your Feedback Shapes the Future :
If it works: Fantastic! Share your results with us and contribute directly to the core vision of SKT AI Labs.
If it doesn't work: No problem at all! Your critical feedback is just as valuable to us. Every experiment and anomaly helps us refine this architecture to make it more stable and robust.
We firmly believe that true innovation stems from community collaboration and transparent testing. Let's build the future of advanced AI together. Your ideas, test results, and feedback are always welcome!
You Can Still Research and Development On WMF Only SKT-SURYA-H Model is Dismissed.
Let's innovate and build together! 💡
Shrijanagain
posted an update about 1 month ago
Post
222
🚀 Big News for the AI Community! 🔥
We’re excited to release NRS_QWEN_MYTHOS_1M — a powerful reasoning model built on Qwen 3.5 9B!
At SKT AI LABS, we’ve supercharged this 9B model with our proprietary Neural Reasoning System (NRS) to deliver next-level performance.
🔥 Why This Model is a Game-Changer:
✅ 100x Reasoning Capacity — Exceptional deep logical thinking and complex problem-solving
✅ 1 Million Token Context — Perfect for massive codebases, long documents, and multi-turn agentic workflows
✅ Advanced Thinking Mode — Native <think> tags for true step-by-step Chain-of-Thought reasoning
✅ Tool-Use Ready — Optimized for Python execution, Web Search, and self-correction
✅ Blazing Fast — Runs smoothly on consumer GPUs like RTX 3090/4090
Technical Highlights:
Base: Qwen 3.5 9B
Tuning: NRS-specific high-quality reasoning data
Context: 1M Tokens (YaRN Scaling)
License: NRS DOCS
Whether you’re a developer building coding agents, a researcher working with long-context data, or someone who loves powerful reasoning — this model is built for you.
👉 Try it now on Hugging Face:
SKT-NRS/NRS_QWEN_MYTHOS_1M
Drop a comment: What will you build with it first? 👇
#AI #OpenSource #LLM #Qwen #ReasoningModel #HuggingFace #NewModel #AICommunity
We’re excited to release NRS_QWEN_MYTHOS_1M — a powerful reasoning model built on Qwen 3.5 9B!
At SKT AI LABS, we’ve supercharged this 9B model with our proprietary Neural Reasoning System (NRS) to deliver next-level performance.
🔥 Why This Model is a Game-Changer:
✅ 100x Reasoning Capacity — Exceptional deep logical thinking and complex problem-solving
✅ 1 Million Token Context — Perfect for massive codebases, long documents, and multi-turn agentic workflows
✅ Advanced Thinking Mode — Native <think> tags for true step-by-step Chain-of-Thought reasoning
✅ Tool-Use Ready — Optimized for Python execution, Web Search, and self-correction
✅ Blazing Fast — Runs smoothly on consumer GPUs like RTX 3090/4090
Technical Highlights:
Base: Qwen 3.5 9B
Tuning: NRS-specific high-quality reasoning data
Context: 1M Tokens (YaRN Scaling)
License: NRS DOCS
Whether you’re a developer building coding agents, a researcher working with long-context data, or someone who loves powerful reasoning — this model is built for you.
👉 Try it now on Hugging Face:
SKT-NRS/NRS_QWEN_MYTHOS_1M
Drop a comment: What will you build with it first? 👇
#AI #OpenSource #LLM #Qwen #ReasoningModel #HuggingFace #NewModel #AICommunity
Shrijanagain
posted an update 3 months ago
Post
2630
We are pleased to announce that the W-IMG Vision Dataset infrastructure is officially live.
The complete asset infrastructure is now accessible on Hugging Face for internal validation and architecture scaling targets.
Dataset Endpoint - sKT-Ai-Labs/W-IMG
#SovereignAI #ComputerVision #MachineLearning #OpenSource
The complete asset infrastructure is now accessible on Hugging Face for internal validation and architecture scaling targets.
Dataset Endpoint - sKT-Ai-Labs/W-IMG
#SovereignAI #ComputerVision #MachineLearning #OpenSource
minwoosun
authored a
paper 4 months ago
Shrijanagain
posted an update 4 months ago
Post
4322
Join fast we will soon published tokens and all join and get started because we will soon off join request button if you want you can join fast guys
- 1 reply
Shrijanagain
posted an update 4 months ago
Post
2687
🚀 Bharat AI Revolution ka Hissa Banein! 🇮🇳
Kya aap Bharat ko AI ki duniya mein ek nayi pehchan dilana chahte hain ?
SKT AI Labs sirf ek naam nahi, ek mission hai—desh ko digital shakti dene ka aur "Viksit Bharat" ke sapne ko sach karne ka.
Humse Kyun Judein?
1. Desh ka Apna AI: Hum aise models bana rahe hain jo khas taur par Bharat ki zarooraton aur bhashaon ke liye hain.
2. Open Collaboration: Hamare Hugging Face repository par hamare kaam ko dekhein, test karein aur apna yogdan dein.
3. Technological Growth: Agar aap student hain, developer hain ya tech enthusiast hain, toh hamare saath naya seekhne aur grow karne ka yeh behtareen mauka hai.
Join here
sKT-Ai-Labs
🔗 sKT-Ai-Labs
Aaiye, saath milkar Bharat AI Revolution ko aage badhate hain! 💻🔥
#SKTAILabs #DigitalIndia #AIRevolution #ViksitBharat #TechInnovation #JoinTheMission
Kya aap Bharat ko AI ki duniya mein ek nayi pehchan dilana chahte hain ?
SKT AI Labs sirf ek naam nahi, ek mission hai—desh ko digital shakti dene ka aur "Viksit Bharat" ke sapne ko sach karne ka.
Humse Kyun Judein?
1. Desh ka Apna AI: Hum aise models bana rahe hain jo khas taur par Bharat ki zarooraton aur bhashaon ke liye hain.
2. Open Collaboration: Hamare Hugging Face repository par hamare kaam ko dekhein, test karein aur apna yogdan dein.
3. Technological Growth: Agar aap student hain, developer hain ya tech enthusiast hain, toh hamare saath naya seekhne aur grow karne ka yeh behtareen mauka hai.
Join here
🔗
Aaiye, saath milkar Bharat AI Revolution ko aage badhate hain! 💻🔥
#SKTAILabs #DigitalIndia #AIRevolution #ViksitBharat #TechInnovation #JoinTheMission
Shrijanagain
posted an update 4 months ago
Post
6927
SOME NEW HINDI + ENGLISH DATASETS
🔗
- sKT-Ai-Labs/HIN
- sKT-Ai-Labs/SKT-MIX
- sKT-Ai-Labs/ST-H
Download and Use And Train Models
You Can Alsoo Use ST-x-LIGHTING Module For Faster Training
pip install ST-x-LIGHT-V11
🔗
- sKT-Ai-Labs/HIN
- sKT-Ai-Labs/SKT-MIX
- sKT-Ai-Labs/ST-H
Download and Use And Train Models
You Can Alsoo Use ST-x-LIGHTING Module For Faster Training
pip install ST-x-LIGHT-V11
- 2 replies
Shrijanagain
posted an update 5 months ago
Post
5662
We are thrilled to announce the launch of SKT-OMNI-CORPUS-2T, a massive-scale, high-quality dataset designed to power the next generation of Foundation Models (LLMs) from scratch.
Developed at SKT AI LABS, this corpus is not just a collection of data; it’s a mission to decentralize high-grade AI training for regional languages and global knowledge.
💎 Key Highlights:
•• Massive Scale: Targeting a multi-terabyte architecture for 2T-level tokenization.
•• Pure Quality: Curated from 500+ Elite Sources
•• Structured for MoE: Perfectly sharded into 3.5GB standardized units (SKT-𝕻 series) for seamless distributed training.
🤝 Open for Collaboration!
We are looking for AI researchers, CUDA engineers, and data scientists to join us in this journey of building Project Surya and the ST-X Series models. Whether it's optimization, custom tokenization, or architecture design—let’s build the future together.
Explore the Dataset on Hugging Face:
🔗 https://huggingface.co/datasets/Shrijanagain/SKT-OMNI-CORPUS-146T-V1
DSR -- 🔗 https://huggingface.co/datasets/Shrijanagain/SKT-DSRx10000
#AI #MachineLearning #OpenSource #IndicAI #SKTAILABS #LLM #BigData #HuggingFace #InnovationIndia
Keeby-smilyai
posted an update 5 months ago
Post
3180
Hello everyone!
- 1 reply
albertvillanova
posted an update 5 months ago
Post
3050
🚀 TRL v0.29.0 introduces trl-training: an agent-native training skill.
This makes the TRL CLI a structured, agent-readable capability, allowing AI agents to reliably execute training workflows such as:
- Supervised Fine-Tuning (SFT)
- Direct Preference Optimization (DPO)
- Group Relative Policy Optimization (GRPO)
We’re excited to see what the community builds on top of this.
If you’re working on AI agents, alignment research, or scalable RL training infrastructure: give TRL v0.29.0 a try! 🤗
The future of ML tooling is agent-native.
🔗 https://github.com/huggingface/trl/releases/tag/v0.29.0
This makes the TRL CLI a structured, agent-readable capability, allowing AI agents to reliably execute training workflows such as:
- Supervised Fine-Tuning (SFT)
- Direct Preference Optimization (DPO)
- Group Relative Policy Optimization (GRPO)
We’re excited to see what the community builds on top of this.
If you’re working on AI agents, alignment research, or scalable RL training infrastructure: give TRL v0.29.0 a try! 🤗
The future of ML tooling is agent-native.
🔗 https://github.com/huggingface/trl/releases/tag/v0.29.0
albertvillanova
posted an update 6 months ago
Post
2043
5 years already working in democratizing AI 🤗
Grateful to be part of such an awesome team making it happen every day.
Grateful to be part of such an awesome team making it happen every day.
himanshubeniwal
authored 6
papers 10 months ago
Cross-lingual Editing in Multilingual Language Models
Paper • 2401.10521 • Published • 2
Remember This Event That Year? Assessing Temporal Information and Reasoning in Large Language Models
Paper • 2402.11997 • Published • 1
Char-mander Use mBackdoor! A Study of Cross-lingual Backdoor Attacks in Multilingual LLMs
Paper • 2502.16901 • Published
COMI-LINGUA: Expert Annotated Large-Scale Dataset for Multitask NLP in Hindi-English Code-Mixing
Paper • 2503.21670 • Published
PolyGuard: A Multilingual Safety Moderation Tool for 17 Languages
Paper • 2504.04377 • Published • 1
COMMENTATOR: A Code-mixed Multilingual Text Annotation Framework
Paper • 2408.03125 • Published