huggingface/DEH-image-scan-data
Viewer β’ Updated β’ 4 β’ 10k β’ 13
Alvaro Bartolome
alvarobartt
AI & ML interests
machine learning + tech lead @huggingface (inference + cloud)
Recent Activity
updated a dataset about 14 hours ago
huggingface/DEH-image-scan-data posted an update 1 day ago
Open agents on AWS SageMaker AI with open models from the Hugging Face Hub!
> Deploy an open model from the Hugging Face Hub on SageMaker AI
> Connect the deployed model to Strands Agents
> Add built-in and custom tools for tool calling
> Expose external capabilities through MCP integration
> Bonus: talk to your agent and visualize traces with Gradio
https://alvarobartt.com/agents-on-aws-sagemaker upvoted a collection 3 days ago
𧬠CarbonOrganizations
![[INTERNAL] Hugging Face + Dell's profile picture](https://cdn-avatars.huggingface.co/v1/production/uploads/60f0608166e5701b80ed3f02/otGPyK6OMlrQrkngKw4tJ.png)
updated a
dataset about 14 hours ago
posted an update 1 day ago
Post
101
Open agents on AWS SageMaker AI with open models from the Hugging Face Hub!
> Deploy an open model from the Hugging Face Hub on SageMaker AI
> Connect the deployed model to Strands Agents
> Add built-in and custom tools for tool calling
> Expose external capabilities through MCP integration
> Bonus: talk to your agent and visualize traces with Gradio
https://alvarobartt.com/agents-on-aws-sagemaker
> Deploy an open model from the Hugging Face Hub on SageMaker AI
> Connect the deployed model to Strands Agents
> Add built-in and custom tools for tool calling
> Expose external capabilities through MCP integration
> Bonus: talk to your agent and visualize traces with Gradio
https://alvarobartt.com/agents-on-aws-sagemaker
upvoted a collection 3 days ago
upvoted an article 3 days ago
Article
Software Forgets: Agent Traces Are the Memory
huggingface
β’ β’ 8 posted an update 5 days ago
Post
3199
Latest
TL; DR MoEs can be misleading to reason about from active parameters alone, since each token only activates a subset of experts, while the serving setup still needs to account for the full resident memory footprint.
π§
ποΈ Dense models usually load and use most weights every forward pass, while MoEs load many experts but only route each token to a few of them
β‘ Active params isn't the same as memory footprint, especially for sparse architectures
π¦ Runtime memory is about what is used per request/token, while loading memory also includes the expert weights that need to be resident
π KV cache can still dominate depending on context length, batch size, and concurrency
π Expert Parallelism (EP) helps shard experts across accelerators when expert weights dominate
π Data Parallelism (DP) + EP is often a good fit for throughput-oriented MoE serving
Check the repository at https://github.com/alvarobartt/hf-mem
hf-mem release added a breakdown of Mixture-of-Experts (MoE) memory usage!TL; DR MoEs can be misleading to reason about from active parameters alone, since each token only activates a subset of experts, while the serving setup still needs to account for the full resident memory footprint.
π§
hf-mem now splits MoE memory into base model weights, routed experts, and KV cacheποΈ Dense models usually load and use most weights every forward pass, while MoEs load many experts but only route each token to a few of them
β‘ Active params isn't the same as memory footprint, especially for sparse architectures
π¦ Runtime memory is about what is used per request/token, while loading memory also includes the expert weights that need to be resident
π KV cache can still dominate depending on context length, batch size, and concurrency
π Expert Parallelism (EP) helps shard experts across accelerators when expert weights dominate
π Data Parallelism (DP) + EP is often a good fit for throughput-oriented MoE serving
Check the repository at https://github.com/alvarobartt/hf-mem
liked a
model 5 days ago
Can't serve the model using TEI
6
#12 opened 3 months ago
by 
TomaszZietkiewicz
Update `experimental.png` with MoE breakdown
#2 opened 12 days ago
by alvarobartt
Upload deploy-nemotron-3-nano-omni/nemotron-3-nano-omni-workflow.png
#612 opened 19 days ago
by 
juanjucm
updated a
dataset 19 days ago
published a
dataset 19 days ago
liked a
model 25 days ago
liked 2
models about 1 month ago
upvoted a changelog about 1 month ago
Hugging Face Changelog
Introducing Kernels
β’ 187
reacted to sergiopaniego's post with π€π₯ about 1 month ago
Post
1391
Earlier this month, Apple introduced Simple Self-Distillation: a fine-tuning method that improves models on coding tasks just by sampling from the model and training on its own outputs with plain cross-entropy
Andβ¦ it's already supported in TRL, built by Kashif Rasul. you can really feel the pace of development in the team π
Paper by Ruixiang ZHANG, He Bai, Huangjie Zheng, Navdeep Jaitly, Ronan Collobert, Yizhe Zhang at Apple π
How it works: the model generates completions at a training-time temperature (T_train) with top_k/top_p truncation, then fine-tunes on them with plain cross-entropy. no labels or verifier needed
You can try it right away with this ready-to-run example (Qwen3-4B on rStar-Coder):
https://github.com/huggingface/trl/blob/main/trl/experimental/ssd/ssd.py
or benchmark a checkpoint with the eval script:
https://github.com/huggingface/trl/blob/main/trl/experimental/ssd/ssd_eval.py
One neat insight from the paper: T_train and T_eval compose into an effective T_eff = T_train Γ T_eval, so a broad band of configs works well. even very noisy samples still help
Want to dig deeper?
Paper: Embarrassingly Simple Self-Distillation Improves Code Generation (2604.01193)
Trainer docs: https://huggingface.co/docs/trl/main/en/ssd_trainer
Andβ¦ it's already supported in TRL, built by Kashif Rasul. you can really feel the pace of development in the team π
Paper by Ruixiang ZHANG, He Bai, Huangjie Zheng, Navdeep Jaitly, Ronan Collobert, Yizhe Zhang at Apple π
How it works: the model generates completions at a training-time temperature (T_train) with top_k/top_p truncation, then fine-tunes on them with plain cross-entropy. no labels or verifier needed
You can try it right away with this ready-to-run example (Qwen3-4B on rStar-Coder):
https://github.com/huggingface/trl/blob/main/trl/experimental/ssd/ssd.py
or benchmark a checkpoint with the eval script:
https://github.com/huggingface/trl/blob/main/trl/experimental/ssd/ssd_eval.py
One neat insight from the paper: T_train and T_eval compose into an effective T_eff = T_train Γ T_eval, so a broad band of configs works well. even very noisy samples still help
Want to dig deeper?
Paper: Embarrassingly Simple Self-Distillation Improves Code Generation (2604.01193)
Trainer docs: https://huggingface.co/docs/trl/main/en/ssd_trainer
updated a
dataset about 1 month ago
upvoted an article about 1 month ago
Article
Multimodal Embedding & Reranker Models with Sentence Transformers
tomaarsen
β’ β’ 59 updated a
dataset about 1 month ago