README
🏃
1
AI & ML interests
None defined yet.
Recent Activity
View all activity
ysharma
updated a
Space 1 day ago
juiceb0xc0de
posted an update 2 days ago
Post
99
I'm starting a new model line, Locus. These models aren't fine tuned, they de-tuned 🤗. What I mean by that is I remove a percentage of the corporate tuned speech patterns like "why this matters" "no fluff" "as a large language model". By reducing the RLHF based habitual patterns in model response I've had higher success rates in personality adoptability. I've fine tuned on the Locus models myself so you can chat with it post fine-tune or just trust me and try it yourself!
I don't aim to remove guard rails or the LLM identity entirely, what I want to do is dampen RLHF to a manageable volume. Personality models perform better with guardrails intact no different than humans with moral guidelines and boundaries. Refusals can help steer and mold personality. RLHF however drowns out adaptability so I'm cranking it down for you to crank your project up!
juiceb0xc0de/bella-bartender-gemma-e2b
juiceb0xc0de/locus-gemma-4-e2b
I don't aim to remove guard rails or the LLM identity entirely, what I want to do is dampen RLHF to a manageable volume. Personality models perform better with guardrails intact no different than humans with moral guidelines and boundaries. Refusals can help steer and mold personality. RLHF however drowns out adaptability so I'm cranking it down for you to crank your project up!
juiceb0xc0de/bella-bartender-gemma-e2b
juiceb0xc0de/locus-gemma-4-e2b
rayanitv
updated a
Space 2 days ago
rayanitv
published a
Space 2 days ago
Post
2239
🙋🏻♂️ Hey there folks ,
Turns out : if we predict 🌏 earth we can save a lot of time looking for interesting things and less time looking at things that we expect to see.
Sentinel-2 imagery 🛰️basically takes a long time to download towards earth. so our "near real time" systems are quite far from that in practical terms.
meanwhile , if we "predict" what we will see , based on what we do see , we can send down much less data in a timely way , and prioritize 📡earth-bound response .
I'm talking about illegal fishing , logging , mining or building in nature reserves , the more of that we predict early the more we're able to stop it on time.
At least that's the concept !
check out the blog : https://huggingface.co/blog/Tonic/save-patagonia-by-predicting-earth
- Collection: https://huggingface.co/collections/NuTonic/earth-observation-with-temporal-and-general-understanding
- Code: https://github.com/Josephrp/Nutonic
- Dataset: NuTonic/sat-vl-sft-training-ready-v1
- Model: NuTonic/lspace
- Training: NuTonic/lspace-trackio
- Evals: NuTonic/Patagonia_Eval
Turns out : if we predict 🌏 earth we can save a lot of time looking for interesting things and less time looking at things that we expect to see.
Sentinel-2 imagery 🛰️basically takes a long time to download towards earth. so our "near real time" systems are quite far from that in practical terms.
meanwhile , if we "predict" what we will see , based on what we do see , we can send down much less data in a timely way , and prioritize 📡earth-bound response .
I'm talking about illegal fishing , logging , mining or building in nature reserves , the more of that we predict early the more we're able to stop it on time.
At least that's the concept !
check out the blog : https://huggingface.co/blog/Tonic/save-patagonia-by-predicting-earth
- Collection: https://huggingface.co/collections/NuTonic/earth-observation-with-temporal-and-general-understanding
- Code: https://github.com/Josephrp/Nutonic
- Dataset: NuTonic/sat-vl-sft-training-ready-v1
- Model: NuTonic/lspace
- Training: NuTonic/lspace-trackio
- Evals: NuTonic/Patagonia_Eval
- 2 replies
pngwn
updated a
Space 9 days ago
freddyaboulton
updated a
Space 12 days ago
Ujjwal-Tyagi
posted an update 14 days ago
Post
228
6 Open-Source Libraries to FineTune LLMs
1. Unsloth
GitHub: https://github.com/unslothai/unsloth
→ Fastest way to fine-tune LLMs locally
→ Optimized for low VRAM (even laptops)
→ Plug-and-play with Hugging Face models
2. Axolotl
GitHub: https://github.com/OpenAccess-AI-Collective/axolotl
→ Flexible LLM fine-tuning configs
→ Supports LoRA, QLoRA, multi-GPU
→ Great for custom training pipelines
3. TRL (Transformer Reinforcement Learning)
GitHub: https://github.com/huggingface/trl
→ RLHF, DPO, PPO for LLM alignment
→ Built on Hugging Face ecosystem
→ Essential for post-training optimization
4. DeepSpeed
GitHub: https://github.com/microsoft/DeepSpeed
→ Train massive models efficiently
→ Memory + speed optimization
→ Industry standard for scaling
5. LLaMA-Factory
GitHub: https://github.com/hiyouga/LLaMA-Factory
→ All-in-one fine-tuning UI + CLI
→ Supports multiple models (LLaMA, Qwen, etc.)
→ Beginner-friendly + powerful
6. PEFT
GitHub: https://github.com/huggingface/peft
→ Fine-tune with minimal compute
→ LoRA, adapters, prefix tuning
→ Best for cost-efficient training
1. Unsloth
GitHub: https://github.com/unslothai/unsloth
→ Fastest way to fine-tune LLMs locally
→ Optimized for low VRAM (even laptops)
→ Plug-and-play with Hugging Face models
2. Axolotl
GitHub: https://github.com/OpenAccess-AI-Collective/axolotl
→ Flexible LLM fine-tuning configs
→ Supports LoRA, QLoRA, multi-GPU
→ Great for custom training pipelines
3. TRL (Transformer Reinforcement Learning)
GitHub: https://github.com/huggingface/trl
→ RLHF, DPO, PPO for LLM alignment
→ Built on Hugging Face ecosystem
→ Essential for post-training optimization
4. DeepSpeed
GitHub: https://github.com/microsoft/DeepSpeed
→ Train massive models efficiently
→ Memory + speed optimization
→ Industry standard for scaling
5. LLaMA-Factory
GitHub: https://github.com/hiyouga/LLaMA-Factory
→ All-in-one fine-tuning UI + CLI
→ Supports multiple models (LLaMA, Qwen, etc.)
→ Beginner-friendly + powerful
6. PEFT
GitHub: https://github.com/huggingface/peft
→ Fine-tune with minimal compute
→ LoRA, adapters, prefix tuning
→ Best for cost-efficient training
- 1 reply
juiceb0xc0de
posted an update 15 days ago
Post
154
I'm not obsessed with LR schedulers you are.
juiceb0xc0de/lr-scheduler-benchmark
Okay maybe I'm a little obsessed with LR schedulers ATM. I ran a SST-2 Sentiment Classification eval using the nyu-mll/glue dataset on distilbert/distilbert-base-uncased-67M to see how different schedulers perform.
I think I've graduated from ML enthusiast to full blown data hoarder and I don't know if I can turn back now.
Anyways I evaluated the 2 schedulers that i designed as well and was pretty happy with the performance of both over all so hell ya to that guess I'll go and grab some more graphs.
https://github.com/JuiceB0xC0de/aecs-scheduler.git
https://github.com/JuiceB0xC0de/lucky-pick-scheduler.git
nyu-mll/glue
distilbert/distilbert-base-uncased
juiceb0xc0de/lr-scheduler-benchmark
Okay maybe I'm a little obsessed with LR schedulers ATM. I ran a SST-2 Sentiment Classification eval using the nyu-mll/glue dataset on distilbert/distilbert-base-uncased-67M to see how different schedulers perform.
I think I've graduated from ML enthusiast to full blown data hoarder and I don't know if I can turn back now.
Anyways I evaluated the 2 schedulers that i designed as well and was pretty happy with the performance of both over all so hell ya to that guess I'll go and grab some more graphs.
https://github.com/JuiceB0xC0de/aecs-scheduler.git
https://github.com/JuiceB0xC0de/lucky-pick-scheduler.git
nyu-mll/glue
distilbert/distilbert-base-uncased
ysharma
published a
Space 16 days ago
juiceb0xc0de
posted an update 16 days ago
Post
97
Okay, I may have been talking out of my ass about my scheduler using less VRAM compared to a FFT. What I did find though: training only ~30% of the model's weights per step consistently beat dense SFT on Hendrycks Math across 3 different seeds.
What makes it interesting isn't just the sparsity — it's that no two consecutive windows share the same active layers. The model never has a stable path from input to output decision. Adjacent layers are rarely both alive at the same time, so the model can't build shortcuts between them. I started developing this to reduce semantic redundancy across layers and stumbled onto something I didn't expect.
Results (0-shot, hendrycks_math exact match):
Dense SFT baseline: 0.0098
DeepChaos seed 1: 0.0142 (+45%)
DeepChaos seed 2: 0.0156 (+59%)
DeepChaos seed 3: 0.0138 (+41%)
Setup: Qwen2.5-3B-Instruct, simplescaling/s1K (1k reasoning traces), 5 epochs, LR 1e-5, optimizer adamw_torch_fused , and cosine scheduler with my lucky pick scheduler on an AMD MI300X 192GB.
The scheduler is still a work in progress but the current version is fully operational. You can check it out at:
https://github.com/JuiceB0xC0de/lucky-pick-scheduler
I would love to hear your experiences with sparsity training!
What makes it interesting isn't just the sparsity — it's that no two consecutive windows share the same active layers. The model never has a stable path from input to output decision. Adjacent layers are rarely both alive at the same time, so the model can't build shortcuts between them. I started developing this to reduce semantic redundancy across layers and stumbled onto something I didn't expect.
Results (0-shot, hendrycks_math exact match):
Dense SFT baseline: 0.0098
DeepChaos seed 1: 0.0142 (+45%)
DeepChaos seed 2: 0.0156 (+59%)
DeepChaos seed 3: 0.0138 (+41%)
Setup: Qwen2.5-3B-Instruct, simplescaling/s1K (1k reasoning traces), 5 epochs, LR 1e-5, optimizer adamw_torch_fused , and cosine scheduler with my lucky pick scheduler on an AMD MI300X 192GB.
The scheduler is still a work in progress but the current version is fully operational. You can check it out at:
https://github.com/JuiceB0xC0de/lucky-pick-scheduler
I would love to hear your experiences with sparsity training!
Post
4183
🙋🏻♂️ Hey there folks,
since everyone liked my previous announcement post ( https://huggingface.co/posts/Tonic/338509028435394 ) so much , i'm back with more high quality proceedural datasets in the Geospacial domain for SFT training !
Check this one out :
NuTonic/sat-bbox-metadata-sft-v1
the goal is to be able to train vision models on multiple images for remote sensing analysis with one shot .
hope you like it ! 🚀
since everyone liked my previous announcement post ( https://huggingface.co/posts/Tonic/338509028435394 ) so much , i'm back with more high quality proceedural datasets in the Geospacial domain for SFT training !
Check this one out :
NuTonic/sat-bbox-metadata-sft-v1
the goal is to be able to train vision models on multiple images for remote sensing analysis with one shot .
hope you like it ! 🚀
- 2 replies
juiceb0xc0de
posted an update 22 days ago
Post
168
Okay, I had way too much fun trying to make the unsloth-bot hallucinate incorrect answers like so many frontier models have done to me in the past regarding fine-tuning and general machine learning. Learning to fine-tune LLMs could have been so much simpler had this been available when I began screwing around with neural networks.
10/10 recommend for beginners.
https://huggingface.co/unsloth/unsloth-bot
10/10 recommend for beginners.
https://huggingface.co/unsloth/unsloth-bot
juiceb0xc0de
posted an update 23 days ago
Post
167
I dropped a new scheduler I created last week without much of an explanation of what it was or how it worked called the Lucky Pick Scheduler. It was just a modal ready app that anyone could have launched and troubleshot their way around.
I've decided I'm going to enter it into the AMD hackathon. Today I started putting together a Github repo with a few extra additions to the scheduler itself.
Essentially it's a training scheduler that randomly drops layers/heads/channels every ~50 steps during fine-tuning, holds the topology frozen, then reshuffles. In theory the model has to build distributed representations because it never trains through the same compute path for long.
And with less gradient memory, bigger models are able fit on smaller hardware.
It's now close to fully capable of automatically configuring itself to any language mode. I've tested it on:
-Qwen-2.5-3b-Instruct
-Falcon-E-3B-Instruct
-SmolLM2-360M
-Ministral-3-3B-Instruct-2512
-Doge-320M
-Llama-3.2-3b
-Gemma-4-e4b
-Phi-4-mini
-OLMo-2-0425-1B
-Phi-tiny-MoE-instruct
Feel free to check it out at Github: https://github.com/JuiceB0xC0de/lucky-pick-scheduler.git
I've decided I'm going to enter it into the AMD hackathon. Today I started putting together a Github repo with a few extra additions to the scheduler itself.
Essentially it's a training scheduler that randomly drops layers/heads/channels every ~50 steps during fine-tuning, holds the topology frozen, then reshuffles. In theory the model has to build distributed representations because it never trains through the same compute path for long.
And with less gradient memory, bigger models are able fit on smaller hardware.
It's now close to fully capable of automatically configuring itself to any language mode. I've tested it on:
-Qwen-2.5-3b-Instruct
-Falcon-E-3B-Instruct
-SmolLM2-360M
-Ministral-3-3B-Instruct-2512
-Doge-320M
-Llama-3.2-3b
-Gemma-4-e4b
-Phi-4-mini
-OLMo-2-0425-1B
-Phi-tiny-MoE-instruct
Feel free to check it out at Github: https://github.com/JuiceB0xC0de/lucky-pick-scheduler.git
Ujjwal-Tyagi
posted an update 24 days ago
Post
206
This is the best set of AI and ML books and a full guide to learning machine learning from the ground up. This is my study material that I used, so I thought it would be helpful to share it with others. Like, share, and add it to your collection at Ujjwal-Tyagi/ai-ml-foundations-book-collection.
Post
3579
🙋🏻♂️ Hey there folks ,
I'm sharing huggingface's largest dataset of annotated statelite images today.
check it out here : NuTonic/sat-image-boundingbox-sft-full
I hope you like it , the idea is to be able to use this with small vision models 🚀
I'm sharing huggingface's largest dataset of annotated statelite images today.
check it out here : NuTonic/sat-image-boundingbox-sft-full
I hope you like it , the idea is to be able to use this with small vision models 🚀
akhaliq
submitted a
paper to Daily Papers 24 days ago
Ujjwal-Tyagi
posted an update 26 days ago
Post
3942
We are hiring at Shirova AI. We need AI researchers and engineers to work in our research lab. Shirova AI is a research lab in India, so we can help our researchers move to nearby workspaces or let them work from home without ever coming to the lab. We're building our founding team, so the pay will be good. You can learn, so don't hesitate to mail us at: careers@shirova.com
juiceb0xc0de
posted an update about 1 month ago
Post
949
I just posted a scheduler I've been fucking around with. Hope some one has fun with it.
https://huggingface.co/blog/juiceb0xc0de/lucky-pick-scheduler
https://huggingface.co/blog/juiceb0xc0de/lucky-pick-scheduler
parkneurals
authored a
paper about 1 month ago