Koen Botermans

Koen1995

AI & ML interests

None yet

Recent Activity

replied to sherryxychen's post 4 days ago

**Training ACT on SO-101: From Woodpecker to 90% Success (All the Mistakes Included)** I spent 3 weeks training Action Chunking Transformer on SO-101 for pick-and-place. Spoiler: the first attempt trained a woodpecker that just pecked the table. 🐦 **What's Different About This Post:** Most ACT tutorials show the success. I documented every failure, hardware issue, and debugging step. If you're new to SO-101/LeRobot/ACT, hopefully my mistakes save you time. Try 1: The Woodpecker - Followed the LeRobot tutorial, collected 50 episodes - Beautiful loss curves ✅ - Robot learned to peck at table ❌ - Rookie mistakes: moving cameras, arm calibration mismatch, limited data diversity, looking at follower arm during teleop (it's cheating!) Try 2: Engineering Upgrades - Fixed hardware setup (tape + markers everywhere) - USB udev rules for camera stability - Formal task definition with stratified sampling - Built proper eval pipeline with progress scoring - Motor breakdown mid-collection (broke the gripper with excessive force 💀) - Results: 60% in-distribution success, 10% OOD (better, but not great) Try 3: More & Better Data https://huggingface.co/sherryxychen/2025-09-07_act - 125 episodes with rotation variations (https://huggingface.co/datasets/sherryxychen/2025-09-01_pick-and-place-block) - Better workspace coverage - Improved grasping technique - Results: 90% in-distribution success, 75% OOD! 🎉 Key Learnings: - Consistent hardware setup is everything - Don't look at the follower arm during teleop - Data diversity is key for generalization - Debug infrastructure matters - Real robots break in mysterious ways (buy spare motors!) Full write-up (with more videos!): https://huggingface.co/blog/sherryxychen/train-act-on-so-101 Code: https://github.com/sherrychen1120/so101_bench Happy to answer any questions! 🤗 #imitation-learning #lerobot #act #so-101

reacted to sherryxychen's post with 🤗 4 days ago

updated a model 5 days ago

Koen1995/base-S0101-env

View all activity

Organizations

replied to sherryxychen's post 4 days ago

Hi @sherryxychen ,

Thanks a lot for the writing this down, what you wrote down really helped!

But I have a question about "Built proper eval pipeline with progress scoring" What do you mean exactly with this? I assume it is from this file: https://github.com/sherrychen1120/so101_bench/blob/main/src/so101_bench/task_progress_labeler.py

But I can't really figure out what happens in that file.

Thanks again!

reacted to sherryxychen's post with 🤗 4 days ago

Post

391

**Training ACT on SO-101: From Woodpecker to 90% Success (All the Mistakes Included)**

I spent 3 weeks training Action Chunking Transformer on SO-101 for pick-and-place. Spoiler: the first attempt trained a woodpecker that just pecked the table. 🐦

**What's Different About This Post:**
Most ACT tutorials show the success. I documented every failure, hardware issue, and debugging step. If you're new to SO-101/LeRobot/ACT, hopefully my mistakes save you time.

Try 1: The Woodpecker
- Followed the LeRobot tutorial, collected 50 episodes
- Beautiful loss curves ✅
- Robot learned to peck at table ❌
- Rookie mistakes: moving cameras, arm calibration mismatch, limited data diversity, looking at follower arm during teleop (it's cheating!)

Try 2: Engineering Upgrades
- Fixed hardware setup (tape + markers everywhere)
- USB udev rules for camera stability
- Formal task definition with stratified sampling
- Built proper eval pipeline with progress scoring
- Motor breakdown mid-collection (broke the gripper with excessive force 💀)
- Results: 60% in-distribution success, 10% OOD (better, but not great)

Try 3: More & Better Data
sherryxychen/2025-09-07_act
- 125 episodes with rotation variations ( sherryxychen/2025-09-01_pick-and-place-block)
- Better workspace coverage
- Improved grasping technique
- Results: 90% in-distribution success, 75% OOD! 🎉

Key Learnings:
- Consistent hardware setup is everything
- Don't look at the follower arm during teleop
- Data diversity is key for generalization
- Debug infrastructure matters
- Real robots break in mysterious ways (buy spare motors!)

Full write-up (with more videos!): https://huggingface.co/blog/sherryxychen/train-act-on-so-101

Code: https://github.com/sherrychen1120/so101_bench

Happy to answer any questions! 🤗

#imitation-learning #lerobot #act #so-101