Update README.md
Browse files
README.md
CHANGED
|
@@ -27,8 +27,9 @@ Utilizing the re-ranking model (e.g., [bge-reranker](https://github.com/FlagOpen
|
|
| 27 |
|
| 28 |
|
| 29 |
## News:
|
| 30 |
-
-
|
| 31 |
-
- 2/
|
|
|
|
| 32 |
|
| 33 |
|
| 34 |
## Specs
|
|
@@ -46,9 +47,11 @@ Utilizing the re-ranking model (e.g., [bge-reranker](https://github.com/FlagOpen
|
|
| 46 |
|
| 47 |
- Data
|
| 48 |
|
| 49 |
-
|
|
| 50 |
-
|
| 51 |
-
|
|
|
|
|
|
|
|
| 52 |
|
| 53 |
|
| 54 |
## FAQ
|
|
@@ -88,7 +91,8 @@ In our experiments, we use [Pyserini](https://github.com/FlagOpen/FlagEmbedding/
|
|
| 88 |
You can follow the common in this [example](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune)
|
| 89 |
to fine-tune the dense embedding.
|
| 90 |
|
| 91 |
-
|
|
|
|
| 92 |
|
| 93 |
|
| 94 |
|
|
@@ -258,7 +262,6 @@ If you have no enough resource to fine-tuning model with long text, the method i
|
|
| 258 |
|
| 259 |
Refer to our [report](https://arxiv.org/pdf/2402.03216.pdf) for more details.
|
| 260 |
|
| 261 |
-
**The fine-tuning codes and datasets will be open-sourced in the near future.**
|
| 262 |
|
| 263 |
|
| 264 |
## Acknowledgement
|
|
|
|
| 27 |
|
| 28 |
|
| 29 |
## News:
|
| 30 |
+
- 2024/3/2: Release unified fine-tuning [example](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/unified_finetune) and [data](https://huggingface.co/datasets/Shitao/bge-m3-data)
|
| 31 |
+
- 2024/2/6: We release the [MLDR](https://huggingface.co/datasets/Shitao/MLDR) (a long document retrieval dataset covering 13 languages) and [evaluation pipeline](https://github.com/FlagOpen/FlagEmbedding/tree/master/C_MTEB/MLDR).
|
| 32 |
+
- 2024/2/1: **Thanks for the excellent tool from Vespa.** You can easily use multiple modes of BGE-M3 following this [notebook](https://github.com/vespa-engine/pyvespa/blob/master/docs/sphinx/source/examples/mother-of-all-embedding-models-cloud.ipynb)
|
| 33 |
|
| 34 |
|
| 35 |
## Specs
|
|
|
|
| 47 |
|
| 48 |
- Data
|
| 49 |
|
| 50 |
+
| Dataset | Introduction |
|
| 51 |
+
|:----------------------------------------------------------:|:-------------------------------------------------:|
|
| 52 |
+
| [MLDR](https://huggingface.co/datasets/Shitao/MLDR) | Docuemtn Retrieval Dataset, covering 13 languages |
|
| 53 |
+
| [bge-m3-data](https://huggingface.co/datasets/Shitao/bge-m3-data) | Fine-tuning data used by bge-m3 |
|
| 54 |
+
|
| 55 |
|
| 56 |
|
| 57 |
## FAQ
|
|
|
|
| 91 |
You can follow the common in this [example](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune)
|
| 92 |
to fine-tune the dense embedding.
|
| 93 |
|
| 94 |
+
If you want to fine-tune all embedding function of m3, you can refer to the [unified_fine-tuning example](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/unified_finetune)
|
| 95 |
+
|
| 96 |
|
| 97 |
|
| 98 |
|
|
|
|
| 262 |
|
| 263 |
Refer to our [report](https://arxiv.org/pdf/2402.03216.pdf) for more details.
|
| 264 |
|
|
|
|
| 265 |
|
| 266 |
|
| 267 |
## Acknowledgement
|