monkt/doclang
Viewer
โข
Updated
โข
13.2k
โข
420
โข
1
AI & ML interests
Transforming documents into AI-ready Markdown or JSON
Recent Activity
s-emanuilovย
published
a
dataset
23 days ago
s-emanuilovย
updated
a
dataset
23 days ago
s-emanuilovย
published
a
model
23 days ago
s-emanuilovย
updated
a
model
23 days ago
s-emanuilovย
published
a
model
23 days ago
s-emanuilovย
updated
a
model
23 days ago
s-emanuilovย
published
a
dataset
25 days ago
s-emanuilovย
updated
a
dataset
25 days ago
s-emanuilovย
posted
an
update
about 2 months ago
Post
240
Ran MTEB evaluation on Bulgarian tasks comparing EmbeddingGemma-300M (
google/embeddinggemma-300m) vs Multilingual-E5-Large (
intfloat/multilingual-e5-large)
EmbeddingGemma-300M scored 71.6% average while E5-Large got 75.9%. Pretty solid results for EmbeddingGemma considering it's half the size and uses way less resources.
EmbeddingGemma actually beats E5-Large on sentiment analysis and natural language inference. E5-Large wins on retrieval and bitext mining tasks.
The 300M model has 4x longer context window (2048 vs 512 tokens) and lower carbon footprint which is good.
Both models work great for Bulgarian but have different strengths depending what you need.
Blog article about the usage: https://huggingface.co/blog/embeddinggemma
PS: Don't forget to use the recommended libraries versions :D
EmbeddingGemma-300M scored 71.6% average while E5-Large got 75.9%. Pretty solid results for EmbeddingGemma considering it's half the size and uses way less resources.
EmbeddingGemma actually beats E5-Large on sentiment analysis and natural language inference. E5-Large wins on retrieval and bitext mining tasks.
The 300M model has 4x longer context window (2048 vs 512 tokens) and lower carbon footprint which is good.
Both models work great for Bulgarian but have different strengths depending what you need.
Blog article about the usage: https://huggingface.co/blog/embeddinggemma
PS: Don't forget to use the recommended libraries versions :D
pip install git+https://github.com/huggingface/transformers@v4.56.0-Embedding-Gemma-preview
pip install sentence-transformers>=5.0.0
-
- 4 replies
s-emanuilovย
posted
an
update
about 2 months ago
Post
301
Embeddings are pretty useful, but mathematically limited.
Great insights from Google DeepMind: On the Theoretical Limitations of Embedding-Based Retrieval (2508.21038)
What could be the alternative? Cross-Encoders (good but can't scale), Multi-vector, Sparse models...or something new.
Hybrid retrieval is the current quick fix, imo.
Great insights from Google DeepMind: On the Theoretical Limitations of Embedding-Based Retrieval (2508.21038)
What could be the alternative? Cross-Encoders (good but can't scale), Multi-vector, Sparse models...or something new.
Hybrid retrieval is the current quick fix, imo.
s-emanuilovย
authored
a
paper
4 months ago
s-emanuilovย
posted
an
update
9 months ago
Post
5466
Tutorial ๐ฅ Training a non-English reasoning model with GRPO and Unsloth
I wanted to share my experiment with training reasoning models in languages other than English/Chinese.
Using Llama 3.1 8B as base, GRPO trainer from trl, and Unsloth optimizations, I got a working prototype in Bulgarian after ~5 hours on an L40S GPU. The approach should work for any language where the base model has some pre-training coverage.
Full code and tutorial here: https://unfoldai.com/reasoning-in-a-non-english-language/
The model itself: s-emanuilov/LLMBG-Llama-3.1-8B-BG-Reasoning-v0.1
I hope this helps anyone looking to build reasoning models in their language.
I wanted to share my experiment with training reasoning models in languages other than English/Chinese.
Using Llama 3.1 8B as base, GRPO trainer from trl, and Unsloth optimizations, I got a working prototype in Bulgarian after ~5 hours on an L40S GPU. The approach should work for any language where the base model has some pre-training coverage.
Full code and tutorial here: https://unfoldai.com/reasoning-in-a-non-english-language/
The model itself: s-emanuilov/LLMBG-Llama-3.1-8B-BG-Reasoning-v0.1
I hope this helps anyone looking to build reasoning models in their language.
-
- 4 replies
s-emanuilovย
authored
2
papers
9 months ago
s-emanuilovย
posted
an
update
10 months ago
Post
749
A new benchmark (DPAB-ฮฑ) has been released that evaluates LLM function calling in both Pythonic and JSON approaches.
It shows that Pythonic function calling often outperforms traditional JSON-based methods, especially for complex multi-step tasks.
Key findings from benchmarks:
โ Claude 3.5 Sonnet leads with 87% on Pythonic vs 45% on JSON
โ Smaller models show impressive results (Dria-Agent-ฮฑ-3B: 72% Pythonic)
โ Even larger models like DeepSeek V3 (685B) show significant gaps (63% Pythonic vs 33% JSON)
If you're building or using LLM agents, these results suggest that how you implement function calling could impact performance - might be worth reconsidering JSON-only approaches.
The benchmark: https://github.com/firstbatchxyz/function-calling-eval
Blog post: https://huggingface.co/blog/andthattoo/dpab-a
It shows that Pythonic function calling often outperforms traditional JSON-based methods, especially for complex multi-step tasks.
Key findings from benchmarks:
โ Claude 3.5 Sonnet leads with 87% on Pythonic vs 45% on JSON
โ Smaller models show impressive results (Dria-Agent-ฮฑ-3B: 72% Pythonic)
โ Even larger models like DeepSeek V3 (685B) show significant gaps (63% Pythonic vs 33% JSON)
If you're building or using LLM agents, these results suggest that how you implement function calling could impact performance - might be worth reconsidering JSON-only approaches.
The benchmark: https://github.com/firstbatchxyz/function-calling-eval
Blog post: https://huggingface.co/blog/andthattoo/dpab-a
s-emanuilovย
posted
an
update
10 months ago
Post
583
New paper from Salesforce AI Research. The authors found that joint training, continual pre-training (CPT), and instruction tuning with a 50/50 data split achieve better results than sequential training. Their 8B parameter model outperformed larger 70B models on financial tasks.
Down-sampling CPT data to match IT data size improved performance on CFA Challenge exams from 34.44% to 55.56%, while maintaining strong general knowledge capabilities as shown by comparable or better performance on general knowledge benchmarks like AI2-ARC and MMLU.
Technical implementation involved two-stage training: Group 1 utilized 3.84B tokens from web and basic texts, followed by Group 2, which used 1.66B tokens from domain-specific books. Their preference alignment method used generative reward models to identify and correct reasoning errors rather than just rating full solutions.
Evaluation on 91,872 samples across 31 tasks showed their Llama-Fin model achieving 91.13% accuracy on sentiment analysis (FPB) and 95.32% on FiQA SA, exceeding GPT-4's performance of 82.16% and 68.51%, respectively, on these benchmarks.
It could be useful for many financial companies looking to build AI pipelines.
Interesting read, but neither the model nor GitHub repo is accessible yet. The key insight for AI builders is that with small models - it is fully possible to outperform much bigger models.
https://arxiv.org/abs/2501.04961
Down-sampling CPT data to match IT data size improved performance on CFA Challenge exams from 34.44% to 55.56%, while maintaining strong general knowledge capabilities as shown by comparable or better performance on general knowledge benchmarks like AI2-ARC and MMLU.
Technical implementation involved two-stage training: Group 1 utilized 3.84B tokens from web and basic texts, followed by Group 2, which used 1.66B tokens from domain-specific books. Their preference alignment method used generative reward models to identify and correct reasoning errors rather than just rating full solutions.
Evaluation on 91,872 samples across 31 tasks showed their Llama-Fin model achieving 91.13% accuracy on sentiment analysis (FPB) and 95.32% on FiQA SA, exceeding GPT-4's performance of 82.16% and 68.51%, respectively, on these benchmarks.
It could be useful for many financial companies looking to build AI pipelines.
Interesting read, but neither the model nor GitHub repo is accessible yet. The key insight for AI builders is that with small models - it is fully possible to outperform much bigger models.
https://arxiv.org/abs/2501.04961
- 1 reply
s-emanuilovย
posted
an
update
10 months ago
Post
2592
Hey HF community! ๐
Excited to share Monkt - a tool I built to solve the eternal headache of processing documents for ML/AI pipelines.
What it does: Converts PDFs, Word, PowerPoint, Excel, Web pages or raw HTML into clean Markdown or structured JSON.
Great for:
โ LLM training dataset preparation;
โ Knowledge base construction;
โ Research paper processing;
โ Technical documentation management.
It has API access for integration into ML pipelines.
Check it out at https://monkt.com/ if you want to save time on document processing infrastructure.
Looking forward to your feedback!
Excited to share Monkt - a tool I built to solve the eternal headache of processing documents for ML/AI pipelines.
What it does: Converts PDFs, Word, PowerPoint, Excel, Web pages or raw HTML into clean Markdown or structured JSON.
Great for:
โ LLM training dataset preparation;
โ Knowledge base construction;
โ Research paper processing;
โ Technical documentation management.
It has API access for integration into ML pipelines.
Check it out at https://monkt.com/ if you want to save time on document processing infrastructure.
Looking forward to your feedback!
-
- 3 replies