Article
Fixing Gradient Accumulation
- +4
•
63
Joseph Lee
jiosephlee
AI & ML interests
None yet
Recent Activity
updated
a model
8 days ago
jiosephlee/thera-pretrain_All_jiosephlee_Intern-S1-mini-lm_1ep_2e-05lr_512bs_ppt-with_smiles_pps-pf_ds-mede
published
a model
8 days ago
jiosephlee/thera-pretrain_All_jiosephlee_Intern-S1-mini-lm_1ep_2e-05lr_512bs_ppt-with_smiles_pps-pf_ds-mede
updated
a model
9 days ago
jiosephlee/thera-pretrain_All_jiosephlee_Intern-S1-mini-lm_1ep_8e-05lr_256bs_ppt-with_smiles_pps-cat_ds-med
Organizations
None yet
upvoted
3
articles
5 months ago
Article
Improving Hugging Face Training Efficiency Through Packing with Flash Attention 2
-
- +4
•
42
Article
Efficient LLM Pretraining: Packed Sequences and Masked Attention
•
60
upvoted
2
articles
6 months ago
Article
Good answers are not necessarily factual answers: an analysis of hallucination in leading LLMs
•
41
Article
Saving Memory Using Padding-Free Transformer Layers during Finetuning
•
20