feat: first commit

Signed-off-by: thxCode <[email protected]>

.gitattributes

@@ -0,0 +1,37 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text
+*.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,170 @@
+---
+license: apache-2.0
+pipeline_tag: text-classification
+tags:
+- transformers
+- sentence-transformers
+- text-embeddings-inference
+language:
+- af
+- ar
+- az
+- be
+- bg
+- bn
+- ca
+- ceb
+- cs
+- cy
+- da
+- de
+- el
+- en
+- es
+- et
+- eu
+- fa
+- fi
+- fr
+- gl
+- gu
+- he
+- hi
+- hr
+- ht
+- hu
+- hy
+- id
+- is
+- it
+- ja
+- jv
+- ka
+- kk
+- km
+- kn
+- ko
+- ky
+- lo
+- lt
+- lv
+- mk
+- ml
+- mn
+- mr
+- ms
+- my
+- ne
+- nl
+- 'no'
+- pa
+- pl
+- pt
+- qu
+- ro
+- ru
+- si
+- sk
+- sl
+- so
+- sq
+- sr
+- sv
+- sw
+- ta
+- te
+- th
+- tl
+- tr
+- uk
+- ur
+- vi
+- yo
+- zh
+---
+
+# gte-multilingual-reranker-base-GGUF
+
+**!!! Experimental supported by [gpustack/llama-box v0.0.72+](https://github.com/gpustack/llama-box) only !!!**<br/>
+
+**Model creator**: [Alibaba-NLP](https://huggingface.co/Alibaba-NLP)<br/>
+**Original model**: [gte-multilingual-reranker-base](https://huggingface.co/Alibaba-NLP/gte-multilingual-reranker-base)<br/>
+**GGUF quantization**: based on llama.cpp [f4d2b](https://github.com/ggerganov/llama.cpp/commit/f4d2b8846a6b34419ff9e9491aee6cd95e444bfc) that patched by llama-box
+
+## gte-multilingual-reranker-base
+
+The **gte-multilingual-reranker-base** model is the first reranker model in the [GTE](https://huggingface.co/collections/Alibaba-NLP/gte-models-6680f0b13f885cb431e6d469) family of models, featuring several key attributes:
+- **High Performance**: Achieves state-of-the-art (SOTA) results in multilingual retrieval tasks and multi-task representation model evaluations when compared to reranker models of similar size.
+- **Training Architecture**: Trained using an encoder-only transformers architecture, resulting in a smaller model size. Unlike previous models based on decode-only LLM architecture (e.g., gte-qwen2-1.5b-instruct), this model has lower hardware requirements for inference, offering a 10x increase in inference speed.
+- **Long Context**: Supports text lengths up to **8192** tokens.
+- **Multilingual Capability**: Supports over **70** languages.
+
+
+## Model Information
+- Model Size: 306M
+- Max Input Tokens: 8192
+
+
+### Usage
+- **It is recommended to install xformers and enable unpadding for acceleration,
+refer to [enable-unpadding-and-xformers](https://huggingface.co/Alibaba-NLP/new-impl#recommendation-enable-unpadding-and-acceleration-with-xformers).**
+- **How to use it offline: [new-impl/discussions/2](https://huggingface.co/Alibaba-NLP/new-impl/discussions/2#662b08d04d8c3d0a09c88fa3)**
+
+
+Using Huggingface transformers (transformers>=4.36.0)
+```
+import torch
+from transformers import AutoModelForSequenceClassification, AutoTokenizer
+
+model_name_or_path = "Alibaba-NLP/gte-multilingual-reranker-base"
+
+tokenizer = AutoTokenizer.from_pretrained(model_name_or_path)
+model = AutoModelForSequenceClassification.from_pretrained(
+    model_name_or_path, trust_remote_code=True,
+    torch_dtype=torch.float16
+)
+model.eval()
+
+pairs = [["中国的首都在哪儿"，"北京"], ["what is the capital of China?", "北京"], ["how to implement quick sort in python?","Introduction of quick sort"]]
+with torch.no_grad():
+    inputs = tokenizer(pairs, padding=True, truncation=True, return_tensors='pt', max_length=512)
+    scores = model(**inputs, return_dict=True).logits.view(-1, ).float()
+    print(scores)
+
+# tensor([1.2315, 0.5923, 0.3041])
+```
+
+
+## Evaluation
+
+Results of reranking based on multiple text retreival datasets
+
+![image](./images/mgte-reranker.png)
+
+**More detailed experimental results can be found in the [paper](https://arxiv.org/pdf/2407.19669)**.
+
+## Cloud API Services
+
+In addition to the open-source [GTE](https://huggingface.co/collections/Alibaba-NLP/gte-models-6680f0b13f885cb431e6d469) series models, GTE series models are also available as commercial API services on Alibaba Cloud.
+
+- [Embedding Models](https://help.aliyun.com/zh/model-studio/developer-reference/general-text-embedding/): Rhree versions of the text embedding models are available: text-embedding-v1/v2/v3, with v3 being the latest API service.
+- [ReRank Models](https://help.aliyun.com/zh/model-studio/developer-reference/general-text-sorting-model/): The gte-rerank model service is available.
+
+Note that the models behind the commercial APIs are not entirely identical to the open-source models.
+
+
+## Citation
+
+If you find our paper or models helpful, please consider cite:
+
+```
+@misc{zhang2024mgtegeneralizedlongcontexttext,
+      title={mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval}, 
+      author={Xin Zhang and Yanzhao Zhang and Dingkun Long and Wen Xie and Ziqi Dai and Jialong Tang and Huan Lin and Baosong Yang and Pengjun Xie and Fei Huang and Meishan Zhang and Wenjie Li and Min Zhang},
+      year={2024},
+      eprint={2407.19669},
+      archivePrefix={arXiv},
+      primaryClass={cs.CL},
+      url={https://arxiv.org/abs/2407.19669}, 
+}
+```

gte-multilingual-reranker-base-FP16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8034ddccba0fb2690f5e7b4137482f5d183aa1055e16b45fdee10f016d8d4624
+size 621536672

gte-multilingual-reranker-base-Q2_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:76a1d2e07272d6920cc0fc1bf159713c6b74932bf3be1ba5aaff8d3296e0ef21
+size 208267808

gte-multilingual-reranker-base-Q3_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:333cb0f13c5cae67a36eaf0bcbead4fa2ef2432a7e4acc0408e1be8cdff522cb
+size 223755296

gte-multilingual-reranker-base-Q4_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6dfb19751a64ff7babd7e22d655ad4dedda61bc3a8aad000b41b88ee0ba6ac13
+size 231353888

gte-multilingual-reranker-base-Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3cb2d613d39dbefabc4af5cd03fadf1502719b8a75d5acda27821dc1455f215d
+size 237657632

gte-multilingual-reranker-base-Q5_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d54b27aca8f48429145c7f7829f22ae2705bd13d75cc55eb81c1c261d728e578
+size 245583392

gte-multilingual-reranker-base-Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dceaf0ab2eba92fd3f7569731ae3a961edafd0d0e924979bcd0fb149f8a9cf1e
+size 250283552

gte-multilingual-reranker-base-Q6_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6d838a92f36072514b8ff3695cadc601ece963655eb50dc1a9fa8b4c5cf15d7a
+size 260702240

gte-multilingual-reranker-base-Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:51c1df6a8830b219e6fffb5ddac85368198dbfaf6dca1a251caf0d1da14557c3
+size 334780832