Upload folder using huggingface_hub

README.md

@@ -1,3 +1,302 @@
----
-license: apache-2.0
----
+---
+tags:
+- mteb
+- qihoo360
+- 奇虎360
+- RAG-reranking
+model-index:
+- name: 360Zhinao-1_8B-reranking
+  results:
+  - task:
+      type: Reranking
+    dataset:
+      type: None
+      name: MTEB CMedQAv1
+      config: default
+      split: test
+      revision: None
+    metrics:
+    - type: map
+      value: 86.75017961853382
+    - type: mrr
+      value: 89.15436507936508
+  - task:
+      type: Reranking
+    dataset:
+      type: None
+      name: MTEB CMedQAv2
+      config: default
+      split: test
+      revision: None
+    metrics:
+    - type: map
+      value: 87.91572151930174
+    - type: mrr
+      value: 89.98869047619048
+  - task:
+      type: Reranking
+    dataset:
+      type: None
+      name: MTEB MMarcoReranking
+      config: default
+      split: dev
+      revision: None
+    metrics:
+    - type: map
+      value: 37.28779203409935
+    - type: mrr
+      value: 36.23730158730159
+  - task:
+      type: Reranking
+    dataset:
+      type: None
+      name: MTEB T2Reranking
+      config: default
+      split: dev
+      revision: None
+    metrics:
+    - type: map
+      value: 68.55153559405632
+    - type: mrr
+      value: 79.62773774596725
+license: apache-2.0
+library_name: transformers
+---
+
+<div align="center">
+<h1>
+  360智脑
+</h1>
+</div>
+<div align="center">
+     <a href="https://huggingface.co/qihoo360">Hugging Face</a>&nbsp&nbsp | &nbsp&nbspn
+     <a href="https://www.modelscope.cn/profile/qihoo360">ModelScope</a>&nbsp&nbsp ｜ &nbsp&nbspn
+</div>
+<br>
+<p align="center">
+ Welcome to 360 Zhinao<a href="https://ai.360.com"> https://ai.360.com </a>
+</p>
+
+<br>
+
+# MTEB Leaderboard Chinese Reranking Results
+We have validated the performance of our model on the [mteb-chinese-reranking leaderboard](https://huggingface.co/spaces/mteb/leaderboard). Currently, the open-source models on this leaderboard are primarily bidirectional discriminative models (BERT-like models). The only unidirectional generative model (GPT-like model) is gte-Qwen1.5-7B-instruct, which has an average score of 66.38, ranking 25th, with less than ideal results. Our self-developed unidirectional generative model, zhinao_1-8b_reranking, achieved an average score of 70.13, currently ranking first overall and first among open-source models, opening up new possibilities for generative models to undertake discriminative tasks.
+
+| Model | T2Reranking | MMarcoReranking | CMedQAv1 | CMedQAv2 | Avg |  
+|:-------------------------------|:--------:|:--------:|:--------:|:--------:|:--------:|  
+| **360Zhinao-1_8B-reranking** | 68.55 | 37.29 | 86.75 | 87.92 | 70.13 |
+| piccolo-large-zh-v2 | 67.15 | 33.39 | 90.14 | 89.31 | 70 |
+| Baichuan-text-embedding | 67.85 | 34.3 | 88.46 | 88.06 | 69.67 |
+| stella-mrl-large-zh-v3.5-1792d | 66.43 | 28.85 | 89.18 | 89.33 | 68.45 |
+| PEG | 69.43 | 33.55 | 86.56 | 84.09 | 68.41 |
+| bge-reranker-base | 67.28 | 35.46 | 81.27 | 84.1 | 67.03 |  
+| bge-reranker-large | 67.6 | 37.17 | 82.14 | 84.19 | 67.78 |
+
+
+# Requirements
+
+```bash
+pip install -r requirements.txt
+```
+
+If your GPU supports fp16 or bf16 precision, we also recommend installing [flash-attention](https://github.com/Dao-AILab/flash-attention) (**now with support for flash attention 2**) to improve your runtime efficiency and reduce memory usage. (**flash-attention is optional and not required for running this project**)
+
+```bash
+git clone https://github.com/Dao-AILab/flash-attention
+cd flash-attention && pip install .
+# The installation below is optional and might be slow.
+# pip install csrc/layer_norm
+# No need to install the following if the flash-attn version is above 2.1.1.
+# pip install csrc/rotary
+```
+
+# Model Introduction
+
+The zhinao_1-8b_reranking model utilizes the self-developed zhinao_1-8b_base model as its foundation. Through iterative discovery and resolution of the following technical issues, it continuously stimulates the world knowledge inherent in the large model during the pre-training phase, better bridging the gap between generative models and discriminative tasks.
+
+## Data Processing
+
+The model training did not utilize world knowledge, meaning it neither continued pre-training with domain-specific data nor fine-tuned datasets outside of the four datasets on the leaderboard. It only used the four datasets within the leaderboard, carefully iterating through data perception, and targeting different datasets for data cleaning and mining to ensure that the ranking in individual tasks could reach the top three level.
+
+## Resolving Task Conflicts
+
+When merging four tasks, due to different data domain distributions, answer patterns, training data volumes, convergence steps, and even sequence lengths, conflicts exist between different tasks. Deeply resolving these conflict issues is crucial to obtaining a universal model with the best comprehensive indicators across different tasks.
+
+## Resolving Training Instability
+
+Unlike generative tasks that produce multiple characters, using generative models for discriminative tasks requires the model to output a continuous value. Therefore, there is an oscillation problem during the training process. Deeply analyzing and resolving training instability can result in a model with better generalization and robustness.
+
+
+# Inference Script
+
+```python
+from typing import cast, List, Union, Tuple, Dict, Optional
+
+import numpy as np
+import torch
+from tqdm import tqdm
+from transformers import AutoModel, AutoTokenizer, AutoModelForSequenceClassification
+import transformers
+from transformers.trainer_pt_utils import LabelSmoother
+IGNORE_TOKEN_ID = LabelSmoother.ignore_index
+
+def preprocess(
+    sources,
+    tokenizer: transformers.PreTrainedTokenizer,
+    max_len: int = 1024,
+    system_message: str = "",
+    #system_message: str = "You are a helpful assistant.",
+    device = None,
+) -> Dict:
+    roles = {"user": "<|im_start|>user", "assistant": "<|im_start|>assistant"}
+    answer_len = 64
+
+    im_start = tokenizer.im_start_id
+    im_end = tokenizer.im_end_id
+    nl_tokens = tokenizer('\n').input_ids
+    _system = tokenizer('system').input_ids + nl_tokens
+    _user = tokenizer('user').input_ids + nl_tokens
+    _assistant = tokenizer('assistant').input_ids + nl_tokens
+
+    # Apply prompt templates
+    input_ids, targets = [], []
+    for i, source in enumerate(sources):
+        ## system_message
+        input_id, target = [], []
+        system = [im_start] + _system + tokenizer(system_message, max_length=max_len-answer_len, truncation=True).input_ids + [im_end] + nl_tokens
+        input_id += system
+        target += [im_start] + [IGNORE_TOKEN_ID] * (len(system)-3) + [im_end] + nl_tokens
+        assert len(input_id) == len(target)
+        
+        ## query ans
+        source = "\n\n".join(source)
+        role = "<|im_start|>user"
+        _input_id = tokenizer(role, max_length=max_len-answer_len, truncation=True).input_ids + nl_tokens + \
+            tokenizer(source, max_length=max_len-answer_len, truncation=True).input_ids + [im_end] + nl_tokens
+        input_id += _input_id
+        if role == '<|im_start|>user':
+            _target = [im_start] + [IGNORE_TOKEN_ID] * (len(_input_id)-3) + [im_end] + nl_tokens
+        elif role == '<|im_start|>assistant':
+            _target = [im_start] + [IGNORE_TOKEN_ID] * len(tokenizer(role, max_length=max_len-answer_len, truncation=True).input_ids) + \
+                _input_id[len(tokenizer(role, max_length=max_len-answer_len, truncation=True).input_ids)+1:-2] + [im_end] + nl_tokens
+        else:
+            raise NotImplementedError
+        target += _target
+
+        ## label use placeholder 0; It will be masked later in the modeling_zhinao.py
+        role = "<|im_start|>assistant"
+        _input_id = tokenizer(role, max_length=max_len-answer_len, truncation=True).input_ids + nl_tokens + \
+            tokenizer("0", max_length=max_len-answer_len, truncation=True).input_ids + [im_end] + nl_tokens
+        input_id += _input_id
+        if role == '<|im_start|>user':
+            _target = [im_start] + [IGNORE_TOKEN_ID] * (len(_input_id)-3) + [im_end] + nl_tokens
+        elif role == '<|im_start|>assistant':
+            _target = [im_start] + [IGNORE_TOKEN_ID] * len(tokenizer(role, max_length=max_len-answer_len, truncation=True).input_ids) + \
+                _input_id[len(tokenizer(role, max_length=max_len-answer_len, truncation=True).input_ids)+1:-2] + [im_end] + nl_tokens
+        else:
+            raise NotImplementedError
+        target += _target
+
+        assert len(input_id) == len(target)
+        input_id += [tokenizer.pad_token_id] * (max_len - len(input_id))
+        target += [IGNORE_TOKEN_ID] * (max_len - len(target))
+        if len(input_id) > max_len:
+            print("max_len_error")
+            print(tokenizer.decode(input_id))
+
+        input_ids.append(input_id[:max_len])
+        targets.append(target[:max_len])
+    input_ids = torch.tensor(input_ids, dtype=torch.int)
+    targets = torch.tensor(targets, dtype=torch.int)
+    #print(f"input_ids {input_ids.shape}")
+    #print(f"targets {targets.shape}")
+
+    return dict(
+        input_ids=input_ids.to(device),
+        labels=targets.to(device),
+        attention_mask=input_ids.ne(tokenizer.pad_token_id).to(device),
+    )
+
+class FlagRerankerCustom:
+    def __init__(
+            self,
+            model_name_or_path: str = None,
+            use_fp16: bool = False
+    ) -> None:
+        self.tokenizer = transformers.AutoTokenizer.from_pretrained(
+            model_name_or_path, 
+            model_max_length=1024, 
+            padding_side="right", 
+            use_fast=False, 
+            trust_remote_code=True
+            )
+        self.tokenizer.pad_token_id = self.tokenizer.eod_id
+        config = transformers.AutoConfig.from_pretrained(
+            model_name_or_path,
+            trust_remote_code=True,
+            bf16=True,
+            )
+        config.use_cache = False
+        self.model = transformers.AutoModelForCausalLM.from_pretrained(
+            model_name_or_path,
+            config=config,
+            trust_remote_code=True,
+            )
+        self.model.linear.bfloat16()
+
+        if torch.cuda.is_available():
+            self.device = torch.device('cuda')
+        elif torch.backends.mps.is_available():
+            self.device = torch.device('mps')
+        else:
+            self.device = torch.device('cpu')
+            use_fp16 = False
+        if use_fp16:
+            self.model.half()
+
+        self.model = self.model.to(self.device)
+
+        self.model.eval()
+
+        self.num_gpus = torch.cuda.device_count()
+        if self.num_gpus > 1:
+            print(f"----------using {self.num_gpus}*GPUs----------")
+            self.model = torch.nn.DataParallel(self.model)
+
+    @torch.no_grad()
+    def compute_score(self, sentence_pairs: Union[List[Tuple[str, str]], Tuple[str, str]], batch_size: int =128,
+                      max_length: int = 1024) -> List[float]:
+        if self.num_gpus > 0:
+            batch_size = batch_size * self.num_gpus
+
+        assert isinstance(sentence_pairs, list)
+        if isinstance(sentence_pairs[0], str):
+            sentence_pairs = [sentence_pairs]
+
+        all_scores = []
+        for start_index in tqdm(range(0, len(sentence_pairs), batch_size), desc="Compute Scores",
+                                disable=len(sentence_pairs) < 128):
+            sentences_batch = sentence_pairs[start_index:start_index + batch_size] # [[q,ans],[q, ans]...]
+            inputs = preprocess(sources=sentences_batch, tokenizer=self.tokenizer,max_len=1024,device=self.device)
+            scores = self.model(**inputs, return_dict=True).logits.view(-1, ).float()
+            all_scores.extend(scores.cpu().numpy().tolist())
+
+        if len(all_scores) == 1:
+            return all_scores[0]
+        return all_scores
+
+
+if __name__ == "__main__":
+    model_name_or_path = "360Zhinao-1_8B-reranking"
+    model = FlagRerankerCustom(model_name_or_path, use_fp16=False)
+    inputs=[["What Color Is the Sky","Blue"], ["What Color Is the Sky","Pink"],]
+    ret = model.compute_score(inputs)
+    print(ret)
+
+```
+
+## License
+The source code of this repository follows the open-source license Apache 2.0.
+360​Zhinao open-source models support commercial use. If you wish to use these models or continue training them for commercial purposes, please contact us via email ([email protected]) to apply. For the specific license agreement, please see <<360 Zhinao Open-Source Model License>>.
+
+

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:77baec385e5aab207797882f836e26018fb50f058f8c5dbc4430cedf9655be03
+size 2070999040

requirements.txt

@@ -0,0 +1,213 @@
+accelerate==0.26.1
+aiohttp==3.9.1
+aiosignal==1.3.1
+annotated-types==0.6.0
+anyio==4.2.0
+appdirs==1.4.4
+argon2-cffi==23.1.0
+argon2-cffi-bindings==21.2.0
+arrow==1.3.0
+asttokens==2.4.1
+async-lru==2.0.4
+async-timeout==4.0.3
+attrs==23.2.0
+auto-gptq==0.5.1
+Babel==2.14.0
+beautifulsoup4==4.12.3
+beir==2.0.0
+bleach==6.1.0
+blessed==1.20.0
+certifi==2023.11.17
+cffi==1.16.0
+charset-normalizer==3.3.2
+click==8.1.7
+cmake==3.28.1
+colorama==0.4.6
+coloredlogs==15.0.1
+comm==0.2.1
+datasets==2.14.7
+debugpy==1.8.0
+decorator==5.1.1
+deepspeed==0.13.1
+defusedxml==0.7.1
+dill==0.3.7
+docker-pycreds==0.4.0
+einops==0.7.0
+elasticsearch==7.9.1
+exceptiongroup==1.2.0
+executing==2.0.1
+faiss-cpu==1.7.4
+faiss-gpu==1.7.2
+fastjsonschema==2.19.1
+filelock==3.13.1
+FlagEmbedding==1.1.9
+flash_attn==2.3.6
+fqdn==1.5.1
+frozenlist==1.4.1
+fsspec==2023.10.0
+gekko==1.0.6
+gitdb==4.0.11
+GitPython==3.1.41
+google==3.0.0
+gpustat==1.1.1
+hjson==3.1.0
+huggingface-hub==0.17.3
+humanfriendly==10.0
+icecream==2.1.3
+idna==3.6
+importlib-metadata==7.0.1
+ipykernel==6.29.0
+ipython==8.18.1
+ipywidgets==8.1.1
+isoduration==20.11.0
+jedi==0.19.1
+Jinja2==3.1.3
+joblib==1.3.2
+json5==0.9.14
+jsonlines==4.0.0
+jsonpointer==2.4
+jsonschema==4.21.1
+jsonschema-specifications==2023.12.1
+jupyter==1.0.0
+jupyter-console==6.6.3
+jupyter-events==0.9.0
+jupyter-lsp==2.2.2
+jupyter_client==8.6.0
+jupyter_core==5.7.1
+jupyter_server==2.12.5
+jupyter_server_terminals==0.5.2
+jupyterlab==4.0.12
+jupyterlab-widgets==3.0.9
+jupyterlab_pygments==0.3.0
+jupyterlab_server==2.25.2
+libretranslatepy==2.1.1
+lightning-utilities==0.10.1
+lit==18.1.2
+lxml==5.1.0
+markdown-it-py==3.0.0
+MarkupSafe==2.1.3
+matplotlib-inline==0.1.6
+mdurl==0.1.2
+mistune==3.0.2
+mpmath==1.3.0
+mteb==1.1.1
+multidict==6.0.4
+multiprocess==0.70.15
+nbclient==0.9.0
+nbconvert==7.14.2
+nbformat==5.9.2
+nest-asyncio==1.6.0
+networkx==3.2.1
+ninja==1.11.1.1
+nltk==3.8.1
+notebook==7.0.7
+notebook_shim==0.2.3
+numpy==1.26.3
+nvidia-cublas-cu11==11.10.3.66
+nvidia-cublas-cu12==12.1.3.1
+nvidia-cuda-cupti-cu11==11.7.101
+nvidia-cuda-cupti-cu12==12.1.105
+nvidia-cuda-nvrtc-cu11==11.7.99
+nvidia-cuda-nvrtc-cu12==12.1.105
+nvidia-cuda-runtime-cu11==11.7.99
+nvidia-cuda-runtime-cu12==12.1.105
+nvidia-cudnn-cu11==8.5.0.96
+nvidia-cudnn-cu12==8.9.2.26
+nvidia-cufft-cu11==10.9.0.58
+nvidia-cufft-cu12==11.0.2.54
+nvidia-curand-cu11==10.2.10.91
+nvidia-curand-cu12==10.3.2.106
+nvidia-cusolver-cu11==11.4.0.1
+nvidia-cusolver-cu12==11.4.5.107
+nvidia-cusparse-cu11==11.7.4.91
+nvidia-cusparse-cu12==12.1.0.106
+nvidia-ml-py==12.535.133
+nvidia-nccl-cu12==2.18.1
+nvidia-nvjitlink-cu12==12.3.101
+nvidia-nvtx-cu11==11.7.91
+nvidia-nvtx-cu12==12.1.105
+optimum==1.14.0
+overrides==7.7.0
+packaging==23.2
+pandas==2.1.4
+pandocfilters==1.5.1
+parso==0.8.3
+peft==0.6.1
+pexpect==4.9.0
+pillow==10.2.0
+platformdirs==4.2.0
+prometheus-client==0.19.0
+prompt-toolkit==3.0.43
+protobuf==4.25.2
+psutil==5.9.7
+ptyprocess==0.7.0
+pure-eval==0.2.2
+py-cpuinfo==9.0.0
+pyarrow==14.0.2
+pyarrow-hotfix==0.6
+pycparser==2.21
+pydantic==2.6.0
+pydantic_core==2.16.1
+Pygments==2.17.2
+pynvml==11.5.0
+python-dateutil==2.8.2
+python-json-logger==2.0.7
+pytrec-eval==0.5
+pytz==2023.3.post1
+PyYAML==6.0.1
+pyzmq==25.1.2
+qtconsole==5.5.1
+QtPy==2.4.1
+referencing==0.33.0
+regex==2023.12.25
+requests==2.31.0
+rfc3339-validator==0.1.4
+rfc3986-validator==0.1.1
+rich==13.7.0
+rotary-emb==0.1
+rouge==1.0.1
+rpds-py==0.17.1
+safetensors==0.4.1
+scikit-learn==1.3.2
+scipy==1.11.4
+Send2Trash==1.8.2
+sentence-transformers==2.2.2
+sentencepiece==0.1.99
+sentry-sdk==1.40.0
+setproctitle==1.3.3
+six==1.16.0
+smmap==5.0.1
+sniffio==1.3.0
+soupsieve==2.5
+stack-data==0.6.3
+sympy==1.12
+terminado==0.18.0
+threadpoolctl==3.2.0
+tiktoken==0.5.2
+tinycss2==1.2.1
+tokenizers==0.14.1
+tomli==2.0.1
+torch==2.1.2
+torchmetrics==1.3.0.post0
+torchvision==0.16.2
+tornado==6.4
+tqdm==4.66.1
+traitlets==5.14.1
+transformers==4.34.0
+transformers-stream-generator==0.0.4
+translate==3.6.1
+triton==2.1.0
+types-python-dateutil==2.8.19.20240106
+typing_extensions==4.9.0
+tzdata==2023.4
+uri-template==1.3.0
+urllib3==2.1.0
+wandb==0.16.2
+wcwidth==0.2.13
+webcolors==1.13
+webencodings==0.5.1
+websocket-client==1.7.0
+widgetsnbextension==4.0.9
+xxhash==3.4.1
+yarl==1.9.4
+zipp==3.17.0

rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aa1a5768088961a39dd71fd2b1b14b3993332dc7ba835dd610bb0c5395be2c93
+size 15920

rng_state_6.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8628151f541fc32f66e6527d07b52899e6b27a9d7db8a5da5b88ab8fd3b80730
+size 15920

trainer_state.json

@@ -0,0 +1,2153 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 1.8537243006403776,
+  "eval_steps": 500,
+  "global_step": 11000,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.01,
+      "learning_rate": 1.2605042016806723e-05,
+      "loss": 0.0184,
+      "step": 50
+    },
+    {
+      "epoch": 0.02,
+      "learning_rate": 2.5210084033613446e-05,
+      "loss": 0.0142,
+      "step": 100
+    },
+    {
+      "epoch": 0.03,
+      "learning_rate": 2.999948467631686e-05,
+      "loss": 0.0136,
+      "step": 150
+    },
+    {
+      "epoch": 0.03,
+      "learning_rate": 2.999648186693593e-05,
+      "loss": 0.0132,
+      "step": 200
+    },
+    {
+      "epoch": 0.04,
+      "learning_rate": 2.999079852637007e-05,
+      "loss": 0.0126,
+      "step": 250
+    },
+    {
+      "epoch": 0.05,
+      "learning_rate": 2.9982435670482325e-05,
+      "loss": 0.0123,
+      "step": 300
+    },
+    {
+      "epoch": 0.06,
+      "learning_rate": 2.9971394794083023e-05,
+      "loss": 0.0122,
+      "step": 350
+    },
+    {
+      "epoch": 0.07,
+      "learning_rate": 2.9957677870662595e-05,
+      "loss": 0.0115,
+      "step": 400
+    },
+    {
+      "epoch": 0.08,
+      "learning_rate": 2.994128735203883e-05,
+      "loss": 0.0113,
+      "step": 450
+    },
+    {
+      "epoch": 0.08,
+      "learning_rate": 2.9922226167918624e-05,
+      "loss": 0.0113,
+      "step": 500
+    },
+    {
+      "epoch": 0.08,
+      "eval_ap_CMedQAv1": 0.8589192417053253,
+      "eval_ap_CMedQAv2": 0.8619373174701326,
+      "eval_ap_Mmarco": 0.33024273953903277,
+      "eval_ap_T2Reranking": 0.6854621357989829,
+      "eval_avg_ap": 0.6841403586283684,
+      "eval_loss": 0.1156548261642456,
+      "eval_mrr_CMedQAv1": 0.8823821428571428,
+      "eval_mrr_CMedQAv2": 0.8846333333333333,
+      "eval_mrr_Mmarco": 0.32088492063492063,
+      "eval_mrr_T2Reranking": 0.7938369818701573,
+      "eval_ndcg@10_CMedQAv1": 0.9590048789978027,
+      "eval_ndcg@10_CMedQAv2": 0.982421875,
+      "eval_ndcg@10_Mmarco": 0.15075407922267914,
+      "eval_ndcg@10_T2Reranking": 0.6162768602371216,
+      "eval_ndcg@1_CMedQAv1": 0.949999988079071,
+      "eval_ndcg@1_CMedQAv2": 1.0,
+      "eval_ndcg@1_Mmarco": 0.10000000149011612,
+      "eval_ndcg@1_T2Reranking": 0.6266667246818542,
+      "eval_ndcg@3_CMedQAv1": 0.9617319107055664,
+      "eval_ndcg@3_CMedQAv2": 0.9882680177688599,
+      "eval_ndcg@3_Mmarco": 0.2184327393770218,
+      "eval_ndcg@3_T2Reranking": 0.5918937921524048,
+      "eval_ndcg@5_CMedQAv1": 0.9577357172966003,
+      "eval_ndcg@5_CMedQAv2": 0.9842175245285034,
+      "eval_ndcg@5_Mmarco": 0.1987150013446808,
+      "eval_ndcg@5_T2Reranking": 0.5972660779953003,
+      "eval_ndcg_CMedQAv1": 0.9578067064285278,
+      "eval_ndcg_CMedQAv2": 0.9620075225830078,
+      "eval_ndcg_Mmarco": 0.4313792586326599,
+      "eval_ndcg_T2Reranking": 0.8985671997070312,
+      "eval_runtime": 1325.9604,
+      "eval_samples_per_second": 301.688,
+      "eval_steps_per_second": 0.295,
+      "step": 500
+    },
+    {
+      "epoch": 0.09,
+      "learning_rate": 2.9900497725374308e-05,
+      "loss": 0.0112,
+      "step": 550
+    },
+    {
+      "epoch": 0.1,
+      "learning_rate": 2.9876105908234656e-05,
+      "loss": 0.0111,
+      "step": 600
+    },
+    {
+      "epoch": 0.11,
+      "learning_rate": 2.9849055076390685e-05,
+      "loss": 0.0112,
+      "step": 650
+    },
+    {
+      "epoch": 0.12,
+      "learning_rate": 2.981935006501634e-05,
+      "loss": 0.011,
+      "step": 700
+    },
+    {
+      "epoch": 0.13,
+      "learning_rate": 2.978699618370422e-05,
+      "loss": 0.0107,
+      "step": 750
+    },
+    {
+      "epoch": 0.13,
+      "learning_rate": 2.9751999215516562e-05,
+      "loss": 0.0109,
+      "step": 800
+    },
+    {
+      "epoch": 0.14,
+      "learning_rate": 2.971436541595152e-05,
+      "loss": 0.0109,
+      "step": 850
+    },
+    {
+      "epoch": 0.15,
+      "learning_rate": 2.967410151182503e-05,
+      "loss": 0.0102,
+      "step": 900
+    },
+    {
+      "epoch": 0.16,
+      "learning_rate": 2.963121470006846e-05,
+      "loss": 0.0103,
+      "step": 950
+    },
+    {
+      "epoch": 0.17,
+      "learning_rate": 2.9585712646442172e-05,
+      "loss": 0.01,
+      "step": 1000
+    },
+    {
+      "epoch": 0.17,
+      "eval_ap_CMedQAv1": 0.8514522230239366,
+      "eval_ap_CMedQAv2": 0.8667097304641765,
+      "eval_ap_Mmarco": 0.33328945740605653,
+      "eval_ap_T2Reranking": 0.681851884051082,
+      "eval_avg_ap": 0.6833258237363129,
+      "eval_loss": 0.11742711067199707,
+      "eval_mrr_CMedQAv1": 0.875027380952381,
+      "eval_mrr_CMedQAv2": 0.8912928571428572,
+      "eval_mrr_Mmarco": 0.3221150793650793,
+      "eval_mrr_T2Reranking": 0.7927472568806352,
+      "eval_ndcg@10_CMedQAv1": 0.9848238229751587,
+      "eval_ndcg@10_CMedQAv2": 1.0,
+      "eval_ndcg@10_Mmarco": 0.1760159730911255,
+      "eval_ndcg@10_T2Reranking": 0.5729199647903442,
+      "eval_ndcg@1_CMedQAv1": 1.0,
+      "eval_ndcg@1_CMedQAv2": 1.0,
+      "eval_ndcg@1_Mmarco": 0.20000000298023224,
+      "eval_ndcg@1_T2Reranking": 0.6088888645172119,
+      "eval_ndcg@3_CMedQAv1": 0.9999998807907104,
+      "eval_ndcg@3_CMedQAv2": 0.9999998807907104,
+      "eval_ndcg@3_Mmarco": 0.2740204632282257,
+      "eval_ndcg@3_T2Reranking": 0.5606718063354492,
+      "eval_ndcg@5_CMedQAv1": 0.9999998807907104,
+      "eval_ndcg@5_CMedQAv2": 0.9999998807907104,
+      "eval_ndcg@5_Mmarco": 0.2053453028202057,
+      "eval_ndcg@5_T2Reranking": 0.5687648057937622,
+      "eval_ndcg_CMedQAv1": 0.958914577960968,
+      "eval_ndcg_CMedQAv2": 0.9623709917068481,
+      "eval_ndcg_Mmarco": 0.45813989639282227,
+      "eval_ndcg_T2Reranking": 0.8942973017692566,
+      "eval_runtime": 1104.4572,
+      "eval_samples_per_second": 362.192,
+      "eval_steps_per_second": 0.354,
+      "step": 1000
+    },
+    {
+      "epoch": 0.18,
+      "learning_rate": 2.953760348416533e-05,
+      "loss": 0.0099,
+      "step": 1050
+    },
+    {
+      "epoch": 0.19,
+      "learning_rate": 2.9486895812462135e-05,
+      "loss": 0.0103,
+      "step": 1100
+    },
+    {
+      "epoch": 0.19,
+      "learning_rate": 2.943359869502476e-05,
+      "loss": 0.0099,
+      "step": 1150
+    },
+    {
+      "epoch": 0.2,
+      "learning_rate": 2.9377721658393268e-05,
+      "loss": 0.0097,
+      "step": 1200
+    },
+    {
+      "epoch": 0.21,
+      "learning_rate": 2.9319274690252808e-05,
+      "loss": 0.0099,
+      "step": 1250
+    },
+    {
+      "epoch": 0.22,
+      "learning_rate": 2.9258268237648375e-05,
+      "loss": 0.0098,
+      "step": 1300
+    },
+    {
+      "epoch": 0.23,
+      "learning_rate": 2.9194713205117454e-05,
+      "loss": 0.0099,
+      "step": 1350
+    },
+    {
+      "epoch": 0.24,
+      "learning_rate": 2.9128620952740903e-05,
+      "loss": 0.0098,
+      "step": 1400
+    },
+    {
+      "epoch": 0.24,
+      "learning_rate": 2.906000329411242e-05,
+      "loss": 0.0096,
+      "step": 1450
+    },
+    {
+      "epoch": 0.25,
+      "learning_rate": 2.898887249422691e-05,
+      "loss": 0.0097,
+      "step": 1500
+    },
+    {
+      "epoch": 0.25,
+      "eval_ap_CMedQAv1": 0.8415707281395,
+      "eval_ap_CMedQAv2": 0.8617282454306143,
+      "eval_ap_Mmarco": 0.369688968005382,
+      "eval_ap_T2Reranking": 0.6881681817864197,
+      "eval_avg_ap": 0.690289030840479,
+      "eval_loss": 0.12302990257740021,
+      "eval_mrr_CMedQAv1": 0.8641345238095237,
+      "eval_mrr_CMedQAv2": 0.8869757936507937,
+      "eval_mrr_Mmarco": 0.36682936507936503,
+      "eval_mrr_T2Reranking": 0.8009392026952963,
+      "eval_ndcg@10_CMedQAv1": 0.9867491722106934,
+      "eval_ndcg@10_CMedQAv2": 1.0,
+      "eval_ndcg@10_Mmarco": 0.2047528326511383,
+      "eval_ndcg@10_T2Reranking": 0.6122068166732788,
+      "eval_ndcg@1_CMedQAv1": 1.0,
+      "eval_ndcg@1_CMedQAv2": 1.0,
+      "eval_ndcg@1_Mmarco": 0.4333333373069763,
+      "eval_ndcg@1_T2Reranking": 0.5628505945205688,
+      "eval_ndcg@3_CMedQAv1": 0.9999998807907104,
+      "eval_ndcg@3_CMedQAv2": 0.9999998807907104,
+      "eval_ndcg@3_Mmarco": 0.30372515320777893,
+      "eval_ndcg@3_T2Reranking": 0.5843338966369629,
+      "eval_ndcg@5_CMedQAv1": 0.9999998807907104,
+      "eval_ndcg@5_CMedQAv2": 0.9999998807907104,
+      "eval_ndcg@5_Mmarco": 0.26333072781562805,
+      "eval_ndcg@5_T2Reranking": 0.5988883972167969,
+      "eval_ndcg_CMedQAv1": 0.9555119276046753,
+      "eval_ndcg_CMedQAv2": 0.9630300402641296,
+      "eval_ndcg_Mmarco": 0.4870051443576813,
+      "eval_ndcg_T2Reranking": 0.8979281187057495,
+      "eval_runtime": 1126.8763,
+      "eval_samples_per_second": 354.987,
+      "eval_steps_per_second": 0.347,
+      "step": 1500
+    },
+    {
+      "epoch": 0.26,
+      "learning_rate": 2.8915241267288212e-05,
+      "loss": 0.0095,
+      "step": 1550
+    },
+    {
+      "epoch": 0.27,
+      "learning_rate": 2.8839122774436504e-05,
+      "loss": 0.0097,
+      "step": 1600
+    },
+    {
+      "epoch": 0.28,
+      "learning_rate": 2.8760530621395827e-05,
+      "loss": 0.0096,
+      "step": 1650
+    },
+    {
+      "epoch": 0.29,
+      "learning_rate": 2.8679478856042137e-05,
+      "loss": 0.0098,
+      "step": 1700
+    },
+    {
+      "epoch": 0.29,
+      "learning_rate": 2.8595981965892344e-05,
+      "loss": 0.0093,
+      "step": 1750
+    },
+    {
+      "epoch": 0.3,
+      "learning_rate": 2.851005487551475e-05,
+      "loss": 0.0094,
+      "step": 1800
+    },
+    {
+      "epoch": 0.31,
+      "learning_rate": 2.8421712943861372e-05,
+      "loss": 0.0095,
+      "step": 1850
+    },
+    {
+      "epoch": 0.32,
+      "learning_rate": 2.8330971961522614e-05,
+      "loss": 0.0094,
+      "step": 1900
+    },
+    {
+      "epoch": 0.33,
+      "learning_rate": 2.823784814790481e-05,
+      "loss": 0.0096,
+      "step": 1950
+    },
+    {
+      "epoch": 0.34,
+      "learning_rate": 2.8142358148331083e-05,
+      "loss": 0.0095,
+      "step": 2000
+    },
+    {
+      "epoch": 0.34,
+      "eval_ap_CMedQAv1": 0.8458553211346398,
+      "eval_ap_CMedQAv2": 0.8602027011804556,
+      "eval_ap_Mmarco": 0.3578987317739686,
+      "eval_ap_T2Reranking": 0.6852344199870449,
+      "eval_avg_ap": 0.6872977935190273,
+      "eval_loss": 0.12346025556325912,
+      "eval_mrr_CMedQAv1": 0.8699591269841269,
+      "eval_mrr_CMedQAv2": 0.8808488095238096,
+      "eval_mrr_Mmarco": 0.3493373015873016,
+      "eval_mrr_T2Reranking": 0.7949749331012027,
+      "eval_ndcg@10_CMedQAv1": 0.9936379194259644,
+      "eval_ndcg@10_CMedQAv2": 1.0000001192092896,
+      "eval_ndcg@10_Mmarco": 0.18332302570343018,
+      "eval_ndcg@10_T2Reranking": 0.5819754600524902,
+      "eval_ndcg@1_CMedQAv1": 1.0,
+      "eval_ndcg@1_CMedQAv2": 1.0,
+      "eval_ndcg@1_Mmarco": 0.20000000298023224,
+      "eval_ndcg@1_T2Reranking": 0.5881534814834595,
+      "eval_ndcg@3_CMedQAv1": 0.9999998807907104,
+      "eval_ndcg@3_CMedQAv2": 0.9999998807907104,
+      "eval_ndcg@3_Mmarco": 0.2092163860797882,
+      "eval_ndcg@3_T2Reranking": 0.5824980139732361,
+      "eval_ndcg@5_CMedQAv1": 0.9999998807907104,
+      "eval_ndcg@5_CMedQAv2": 0.9999998807907104,
+      "eval_ndcg@5_Mmarco": 0.19205407798290253,
+      "eval_ndcg@5_T2Reranking": 0.5816112160682678,
+      "eval_ndcg_CMedQAv1": 0.9610760807991028,
+      "eval_ndcg_CMedQAv2": 0.9652814865112305,
+      "eval_ndcg_Mmarco": 0.45809394121170044,
+      "eval_ndcg_T2Reranking": 0.896248996257782,
+      "eval_runtime": 1147.2183,
+      "eval_samples_per_second": 348.692,
+      "eval_steps_per_second": 0.341,
+      "step": 2000
+    },
+    {
+      "epoch": 0.35,
+      "learning_rate": 2.8044519031066117e-05,
+      "loss": 0.0094,
+      "step": 2050
+    },
+    {
+      "epoch": 0.35,
+      "learning_rate": 2.794434828426527e-05,
+      "loss": 0.0095,
+      "step": 2100
+    },
+    {
+      "epoch": 0.36,
+      "learning_rate": 2.7841863812848724e-05,
+      "loss": 0.0093,
+      "step": 2150
+    },
+    {
+      "epoch": 0.37,
+      "learning_rate": 2.773708393530104e-05,
+      "loss": 0.0093,
+      "step": 2200
+    },
+    {
+      "epoch": 0.38,
+      "learning_rate": 2.7630027380396854e-05,
+      "loss": 0.0092,
+      "step": 2250
+    },
+    {
+      "epoch": 0.39,
+      "learning_rate": 2.7520713283853237e-05,
+      "loss": 0.0091,
+      "step": 2300
+    },
+    {
+      "epoch": 0.4,
+      "learning_rate": 2.740916118490928e-05,
+      "loss": 0.0091,
+      "step": 2350
+    },
+    {
+      "epoch": 0.4,
+      "learning_rate": 2.729539102283358e-05,
+      "loss": 0.009,
+      "step": 2400
+    },
+    {
+      "epoch": 0.41,
+      "learning_rate": 2.7179423133360214e-05,
+      "loss": 0.009,
+      "step": 2450
+    },
+    {
+      "epoch": 0.42,
+      "learning_rate": 2.7061278245053856e-05,
+      "loss": 0.0093,
+      "step": 2500
+    },
+    {
+      "epoch": 0.42,
+      "eval_ap_CMedQAv1": 0.8494758904704668,
+      "eval_ap_CMedQAv2": 0.8578434981555412,
+      "eval_ap_Mmarco": 0.38035966777131036,
+      "eval_ap_T2Reranking": 0.6841403863981271,
+      "eval_avg_ap": 0.6929548606988614,
+      "eval_loss": 0.11943219602108002,
+      "eval_mrr_CMedQAv1": 0.8727809523809524,
+      "eval_mrr_CMedQAv2": 0.8802436507936509,
+      "eval_mrr_Mmarco": 0.37201190476190477,
+      "eval_mrr_T2Reranking": 0.7946123631127017,
+      "eval_ndcg@10_CMedQAv1": 0.9861699342727661,
+      "eval_ndcg@10_CMedQAv2": 1.0000001192092896,
+      "eval_ndcg@10_Mmarco": 0.21542616188526154,
+      "eval_ndcg@10_T2Reranking": 0.5951088666915894,
+      "eval_ndcg@1_CMedQAv1": 1.0,
+      "eval_ndcg@1_CMedQAv2": 1.0,
+      "eval_ndcg@1_Mmarco": 0.25,
+      "eval_ndcg@1_T2Reranking": 0.5847460031509399,
+      "eval_ndcg@3_CMedQAv1": 0.9999998807907104,
+      "eval_ndcg@3_CMedQAv2": 0.9999998807907104,
+      "eval_ndcg@3_Mmarco": 0.2740204632282257,
+      "eval_ndcg@3_T2Reranking": 0.5968767404556274,
+      "eval_ndcg@5_CMedQAv1": 0.9999998807907104,
+      "eval_ndcg@5_CMedQAv2": 0.9999998807907104,
+      "eval_ndcg@5_Mmarco": 0.25857046246528625,
+      "eval_ndcg@5_T2Reranking": 0.6022564172744751,
+      "eval_ndcg_CMedQAv1": 0.9562269449234009,
+      "eval_ndcg_CMedQAv2": 0.9615292549133301,
+      "eval_ndcg_Mmarco": 0.48261213302612305,
+      "eval_ndcg_T2Reranking": 0.8961272239685059,
+      "eval_runtime": 1052.5471,
+      "eval_samples_per_second": 380.055,
+      "eval_steps_per_second": 0.371,
+      "step": 2500
+    },
+    {
+      "epoch": 0.43,
+      "learning_rate": 2.694097747560465e-05,
+      "loss": 0.0093,
+      "step": 2550
+    },
+    {
+      "epoch": 0.44,
+      "learning_rate": 2.6818542328053576e-05,
+      "loss": 0.009,
+      "step": 2600
+    },
+    {
+      "epoch": 0.45,
+      "learning_rate": 2.66939946869489e-05,
+      "loss": 0.009,
+      "step": 2650
+    },
+    {
+      "epoch": 0.46,
+      "learning_rate": 2.6567356814434426e-05,
+      "loss": 0.0089,
+      "step": 2700
+    },
+    {
+      "epoch": 0.46,
+      "learning_rate": 2.6438651346270292e-05,
+      "loss": 0.0089,
+      "step": 2750
+    },
+    {
+      "epoch": 0.47,
+      "learning_rate": 2.630790128778696e-05,
+      "loss": 0.0089,
+      "step": 2800
+    },
+    {
+      "epoch": 0.48,
+      "learning_rate": 2.617513000977315e-05,
+      "loss": 0.009,
+      "step": 2850
+    },
+    {
+      "epoch": 0.49,
+      "learning_rate": 2.604036124429844e-05,
+      "loss": 0.0088,
+      "step": 2900
+    },
+    {
+      "epoch": 0.5,
+      "learning_rate": 2.590361908047132e-05,
+      "loss": 0.0091,
+      "step": 2950
+    },
+    {
+      "epoch": 0.51,
+      "learning_rate": 2.5764927960133396e-05,
+      "loss": 0.009,
+      "step": 3000
+    },
+    {
+      "epoch": 0.51,
+      "eval_ap_CMedQAv1": 0.8477064604578514,
+      "eval_ap_CMedQAv2": 0.8582835044558619,
+      "eval_ap_Mmarco": 0.31718590811535546,
+      "eval_ap_T2Reranking": 0.6816479742452222,
+      "eval_avg_ap": 0.6762059618185727,
+      "eval_loss": 0.12098982185125351,
+      "eval_mrr_CMedQAv1": 0.8740146825396825,
+      "eval_mrr_CMedQAv2": 0.8814178571428573,
+      "eval_mrr_Mmarco": 0.3068174603174603,
+      "eval_mrr_T2Reranking": 0.791166000365391,
+      "eval_ndcg@10_CMedQAv1": 0.9580147862434387,
+      "eval_ndcg@10_CMedQAv2": 0.9866949319839478,
+      "eval_ndcg@10_Mmarco": 0.13374407589435577,
+      "eval_ndcg@10_T2Reranking": 0.5563193559646606,
+      "eval_ndcg@1_CMedQAv1": 1.0,
+      "eval_ndcg@1_CMedQAv2": 1.0,
+      "eval_ndcg@1_Mmarco": 0.0,
+      "eval_ndcg@1_T2Reranking": 0.6533333659172058,
+      "eval_ndcg@3_CMedQAv1": 0.976535975933075,
+      "eval_ndcg@3_CMedQAv2": 0.9999998807907104,
+      "eval_ndcg@3_Mmarco": 0.11173196882009506,
+      "eval_ndcg@3_T2Reranking": 0.6220604777336121,
+      "eval_ndcg@5_CMedQAv1": 0.9553145170211792,
+      "eval_ndcg@5_CMedQAv2": 0.9999998807907104,
+      "eval_ndcg@5_Mmarco": 0.12382900714874268,
+      "eval_ndcg@5_T2Reranking": 0.5894810557365417,
+      "eval_ndcg_CMedQAv1": 0.9517234563827515,
+      "eval_ndcg_CMedQAv2": 0.9606796503067017,
+      "eval_ndcg_Mmarco": 0.39375704526901245,
+      "eval_ndcg_T2Reranking": 0.896885871887207,
+      "eval_runtime": 1062.2445,
+      "eval_samples_per_second": 376.586,
+      "eval_steps_per_second": 0.368,
+      "step": 3000
+    },
+    {
+      "epoch": 0.51,
+      "learning_rate": 2.5624312673490554e-05,
+      "loss": 0.0089,
+      "step": 3050
+    },
+    {
+      "epoch": 0.52,
+      "learning_rate": 2.5481798354681882e-05,
+      "loss": 0.0087,
+      "step": 3100
+    },
+    {
+      "epoch": 0.53,
+      "learning_rate": 2.5337410477287057e-05,
+      "loss": 0.0091,
+      "step": 3150
+    },
+    {
+      "epoch": 0.54,
+      "learning_rate": 2.5191174849773132e-05,
+      "loss": 0.0088,
+      "step": 3200
+    },
+    {
+      "epoch": 0.55,
+      "learning_rate": 2.5043117610881402e-05,
+      "loss": 0.0091,
+      "step": 3250
+    },
+    {
+      "epoch": 0.56,
+      "learning_rate": 2.4893265224955276e-05,
+      "loss": 0.0089,
+      "step": 3300
+    },
+    {
+      "epoch": 0.56,
+      "learning_rate": 2.4741644477209923e-05,
+      "loss": 0.0088,
+      "step": 3350
+    },
+    {
+      "epoch": 0.57,
+      "learning_rate": 2.4588282468944582e-05,
+      "loss": 0.0088,
+      "step": 3400
+    },
+    {
+      "epoch": 0.58,
+      "learning_rate": 2.4433206612698367e-05,
+      "loss": 0.0089,
+      "step": 3450
+    },
+    {
+      "epoch": 0.59,
+      "learning_rate": 2.4276444627350437e-05,
+      "loss": 0.0089,
+      "step": 3500
+    },
+    {
+      "epoch": 0.59,
+      "eval_ap_CMedQAv1": 0.8523776953190774,
+      "eval_ap_CMedQAv2": 0.8611607089962926,
+      "eval_ap_Mmarco": 0.35086797584501545,
+      "eval_ap_T2Reranking": 0.6760057559100935,
+      "eval_avg_ap": 0.6851030340176197,
+      "eval_loss": 0.12173164635896683,
+      "eval_mrr_CMedQAv1": 0.8750813492063492,
+      "eval_mrr_CMedQAv2": 0.8850503968253968,
+      "eval_mrr_Mmarco": 0.34225793650793657,
+      "eval_mrr_T2Reranking": 0.788319644603497,
+      "eval_ndcg@10_CMedQAv1": 0.9706858396530151,
+      "eval_ndcg@10_CMedQAv2": 1.0000001192092896,
+      "eval_ndcg@10_Mmarco": 0.16307643055915833,
+      "eval_ndcg@10_T2Reranking": 0.5981327891349792,
+      "eval_ndcg@1_CMedQAv1": 1.0,
+      "eval_ndcg@1_CMedQAv2": 1.0,
+      "eval_ndcg@1_Mmarco": 0.20000000298023224,
+      "eval_ndcg@1_T2Reranking": 0.6888889670372009,
+      "eval_ndcg@3_CMedQAv1": 0.9703917503356934,
+      "eval_ndcg@3_CMedQAv2": 0.9999998807907104,
+      "eval_ndcg@3_Mmarco": 0.17653605341911316,
+      "eval_ndcg@3_T2Reranking": 0.6413534879684448,
+      "eval_ndcg@5_CMedQAv1": 0.978601336479187,
+      "eval_ndcg@5_CMedQAv2": 0.9999998807907104,
+      "eval_ndcg@5_Mmarco": 0.1764817237854004,
+      "eval_ndcg@5_T2Reranking": 0.6482794284820557,
+      "eval_ndcg_CMedQAv1": 0.9572161436080933,
+      "eval_ndcg_CMedQAv2": 0.9616801142692566,
+      "eval_ndcg_Mmarco": 0.4346458315849304,
+      "eval_ndcg_T2Reranking": 0.8978231549263,
+      "eval_runtime": 1064.5644,
+      "eval_samples_per_second": 375.765,
+      "eval_steps_per_second": 0.367,
+      "step": 3500
+    },
+    {
+      "epoch": 0.6,
+      "learning_rate": 2.4118024533165415e-05,
+      "loss": 0.0089,
+      "step": 3550
+    },
+    {
+      "epoch": 0.61,
+      "learning_rate": 2.3957974646784935e-05,
+      "loss": 0.0085,
+      "step": 3600
+    },
+    {
+      "epoch": 0.62,
+      "learning_rate": 2.379632357616621e-05,
+      "loss": 0.0087,
+      "step": 3650
+    },
+    {
+      "epoch": 0.62,
+      "learning_rate": 2.363310021546853e-05,
+      "loss": 0.0085,
+      "step": 3700
+    },
+    {
+      "epoch": 0.63,
+      "learning_rate": 2.3468333739888613e-05,
+      "loss": 0.0087,
+      "step": 3750
+    },
+    {
+      "epoch": 0.64,
+      "learning_rate": 2.3302053600445695e-05,
+      "loss": 0.0088,
+      "step": 3800
+    },
+    {
+      "epoch": 0.65,
+      "learning_rate": 2.313428951871735e-05,
+      "loss": 0.0089,
+      "step": 3850
+    },
+    {
+      "epoch": 0.66,
+      "learning_rate": 2.2965071481526943e-05,
+      "loss": 0.0084,
+      "step": 3900
+    },
+    {
+      "epoch": 0.67,
+      "learning_rate": 2.2794429735583658e-05,
+      "loss": 0.0085,
+      "step": 3950
+    },
+    {
+      "epoch": 0.67,
+      "learning_rate": 2.262239478207607e-05,
+      "loss": 0.0087,
+      "step": 4000
+    },
+    {
+      "epoch": 0.67,
+      "eval_ap_CMedQAv1": 0.8508535708843126,
+      "eval_ap_CMedQAv2": 0.8605212604516235,
+      "eval_ap_Mmarco": 0.3549766633801287,
+      "eval_ap_T2Reranking": 0.6834420037661381,
+      "eval_avg_ap": 0.6874483746205506,
+      "eval_loss": 0.12024065852165222,
+      "eval_mrr_CMedQAv1": 0.8741781746031745,
+      "eval_mrr_CMedQAv2": 0.8835730158730158,
+      "eval_mrr_Mmarco": 0.34038492063492065,
+      "eval_mrr_T2Reranking": 0.7928938834617574,
+      "eval_ndcg@10_CMedQAv1": 0.9930569529533386,
+      "eval_ndcg@10_CMedQAv2": 0.9854609370231628,
+      "eval_ndcg@10_Mmarco": 0.13490335643291473,
+      "eval_ndcg@10_T2Reranking": 0.6463578939437866,
+      "eval_ndcg@1_CMedQAv1": 1.0,
+      "eval_ndcg@1_CMedQAv2": 1.0,
+      "eval_ndcg@1_Mmarco": 0.10000000149011612,
+      "eval_ndcg@1_T2Reranking": 0.6644444465637207,
+      "eval_ndcg@3_CMedQAv1": 0.9999998807907104,
+      "eval_ndcg@3_CMedQAv2": 0.9999998807907104,
+      "eval_ndcg@3_Mmarco": 0.15921637415885925,
+      "eval_ndcg@3_T2Reranking": 0.688578188419342,
+      "eval_ndcg@5_CMedQAv1": 0.9999998807907104,
+      "eval_ndcg@5_CMedQAv2": 0.9934396743774414,
+      "eval_ndcg@5_Mmarco": 0.14279724657535553,
+      "eval_ndcg@5_T2Reranking": 0.6808897852897644,
+      "eval_ndcg_CMedQAv1": 0.9591971635818481,
+      "eval_ndcg_CMedQAv2": 0.9625831842422485,
+      "eval_ndcg_Mmarco": 0.41239920258522034,
+      "eval_ndcg_T2Reranking": 0.8961763381958008,
+      "eval_runtime": 1074.1598,
+      "eval_samples_per_second": 372.408,
+      "eval_steps_per_second": 0.364,
+      "step": 4000
+    },
+    {
+      "epoch": 0.68,
+      "learning_rate": 2.2448997371220256e-05,
+      "loss": 0.0088,
+      "step": 4050
+    },
+    {
+      "epoch": 0.69,
+      "learning_rate": 2.2274268496763367e-05,
+      "loss": 0.0085,
+      "step": 4100
+    },
+    {
+      "epoch": 0.7,
+      "learning_rate": 2.2098239390443697e-05,
+      "loss": 0.0085,
+      "step": 4150
+    },
+    {
+      "epoch": 0.71,
+      "learning_rate": 2.192094151640817e-05,
+      "loss": 0.0085,
+      "step": 4200
+    },
+    {
+      "epoch": 0.72,
+      "learning_rate": 2.174240656558834e-05,
+      "loss": 0.0084,
+      "step": 4250
+    },
+    {
+      "epoch": 0.72,
+      "learning_rate": 2.156266645003582e-05,
+      "loss": 0.0082,
+      "step": 4300
+    },
+    {
+      "epoch": 0.73,
+      "learning_rate": 2.1381753297218183e-05,
+      "loss": 0.0083,
+      "step": 4350
+    },
+    {
+      "epoch": 0.74,
+      "learning_rate": 2.1199699444276374e-05,
+      "loss": 0.0082,
+      "step": 4400
+    },
+    {
+      "epoch": 0.75,
+      "learning_rate": 2.1016537432244663e-05,
+      "loss": 0.0081,
+      "step": 4450
+    },
+    {
+      "epoch": 0.76,
+      "learning_rate": 2.0832300000234076e-05,
+      "loss": 0.0083,
+      "step": 4500
+    },
+    {
+      "epoch": 0.76,
+      "eval_ap_CMedQAv1": 0.8549180059460657,
+      "eval_ap_CMedQAv2": 0.8599012485902339,
+      "eval_ap_Mmarco": 0.31690584463403754,
+      "eval_ap_T2Reranking": 0.6831123214291062,
+      "eval_avg_ap": 0.6787093551498609,
+      "eval_loss": 0.12336914986371994,
+      "eval_mrr_CMedQAv1": 0.8789436507936508,
+      "eval_mrr_CMedQAv2": 0.8820293650793651,
+      "eval_mrr_Mmarco": 0.30601190476190476,
+      "eval_mrr_T2Reranking": 0.7921361672631138,
+      "eval_ndcg@10_CMedQAv1": 0.9862348437309265,
+      "eval_ndcg@10_CMedQAv2": 0.983578085899353,
+      "eval_ndcg@10_Mmarco": 0.1612614393234253,
+      "eval_ndcg@10_T2Reranking": 0.6136462092399597,
+      "eval_ndcg@1_CMedQAv1": 1.0,
+      "eval_ndcg@1_CMedQAv2": 1.0,
+      "eval_ndcg@1_Mmarco": 0.30000001192092896,
+      "eval_ndcg@1_T2Reranking": 0.6665303111076355,
+      "eval_ndcg@3_CMedQAv1": 0.9999998807907104,
+      "eval_ndcg@3_CMedQAv2": 0.9999998807907104,
+      "eval_ndcg@3_Mmarco": 0.22346392273902893,
+      "eval_ndcg@3_T2Reranking": 0.6385782957077026,
+      "eval_ndcg@5_CMedQAv1": 0.9999998807907104,
+      "eval_ndcg@5_CMedQAv2": 0.9853931665420532,
+      "eval_ndcg@5_Mmarco": 0.20383748412132263,
+      "eval_ndcg@5_T2Reranking": 0.6306164860725403,
+      "eval_ndcg_CMedQAv1": 0.9580343961715698,
+      "eval_ndcg_CMedQAv2": 0.9610058069229126,
+      "eval_ndcg_Mmarco": 0.4440736174583435,
+      "eval_ndcg_T2Reranking": 0.8990576863288879,
+      "eval_runtime": 1049.34,
+      "eval_samples_per_second": 381.217,
+      "eval_steps_per_second": 0.373,
+      "step": 4500
+    },
+    {
+      "epoch": 0.77,
+      "learning_rate": 2.0647020079580543e-05,
+      "loss": 0.0081,
+      "step": 4550
+    },
+    {
+      "epoch": 0.78,
+      "learning_rate": 2.0460730787958573e-05,
+      "loss": 0.0082,
+      "step": 4600
+    },
+    {
+      "epoch": 0.78,
+      "learning_rate": 2.0273465423461677e-05,
+      "loss": 0.0084,
+      "step": 4650
+    },
+    {
+      "epoch": 0.79,
+      "learning_rate": 2.008525745865055e-05,
+      "loss": 0.0081,
+      "step": 4700
+    },
+    {
+      "epoch": 0.8,
+      "learning_rate": 1.989614053457002e-05,
+      "loss": 0.0081,
+      "step": 4750
+    },
+    {
+      "epoch": 0.81,
+      "learning_rate": 1.970614845473596e-05,
+      "loss": 0.0082,
+      "step": 4800
+    },
+    {
+      "epoch": 0.82,
+      "learning_rate": 1.9515315179093052e-05,
+      "loss": 0.0081,
+      "step": 4850
+    },
+    {
+      "epoch": 0.83,
+      "learning_rate": 1.9323674817944713e-05,
+      "loss": 0.0081,
+      "step": 4900
+    },
+    {
+      "epoch": 0.83,
+      "learning_rate": 1.9131261625856034e-05,
+      "loss": 0.0082,
+      "step": 4950
+    },
+    {
+      "epoch": 0.84,
+      "learning_rate": 1.8938109995531015e-05,
+      "loss": 0.0081,
+      "step": 5000
+    },
+    {
+      "epoch": 0.84,
+      "eval_ap_CMedQAv1": 0.8580422117025617,
+      "eval_ap_CMedQAv2": 0.8637152440470349,
+      "eval_ap_Mmarco": 0.31369636069838835,
+      "eval_ap_T2Reranking": 0.690709122210063,
+      "eval_avg_ap": 0.6815407346645119,
+      "eval_loss": 0.12387344986200333,
+      "eval_mrr_CMedQAv1": 0.8822130952380952,
+      "eval_mrr_CMedQAv2": 0.887097619047619,
+      "eval_mrr_Mmarco": 0.30120238095238094,
+      "eval_mrr_T2Reranking": 0.8040100321329163,
+      "eval_ndcg@10_CMedQAv1": 0.9928603172302246,
+      "eval_ndcg@10_CMedQAv2": 0.9926636815071106,
+      "eval_ndcg@10_Mmarco": 0.1475120484828949,
+      "eval_ndcg@10_T2Reranking": 0.6204460859298706,
+      "eval_ndcg@1_CMedQAv1": 1.0,
+      "eval_ndcg@1_CMedQAv2": 1.0,
+      "eval_ndcg@1_Mmarco": 0.10000000149011612,
+      "eval_ndcg@1_T2Reranking": 0.7071110606193542,
+      "eval_ndcg@3_CMedQAv1": 0.9999998807907104,
+      "eval_ndcg@3_CMedQAv2": 0.9999998807907104,
+      "eval_ndcg@3_Mmarco": 0.212288498878479,
+      "eval_ndcg@3_T2Reranking": 0.6500719785690308,
+      "eval_ndcg@5_CMedQAv1": 0.9999998807907104,
+      "eval_ndcg@5_CMedQAv2": 0.9999998807907104,
+      "eval_ndcg@5_Mmarco": 0.19501754641532898,
+      "eval_ndcg@5_T2Reranking": 0.6329637169837952,
+      "eval_ndcg_CMedQAv1": 0.9593874216079712,
+      "eval_ndcg_CMedQAv2": 0.9627755284309387,
+      "eval_ndcg_Mmarco": 0.4310723841190338,
+      "eval_ndcg_T2Reranking": 0.9022238850593567,
+      "eval_runtime": 1077.7629,
+      "eval_samples_per_second": 371.163,
+      "eval_steps_per_second": 0.363,
+      "step": 5000
+    },
+    {
+      "epoch": 0.85,
+      "learning_rate": 1.8744254451665046e-05,
+      "loss": 0.0081,
+      "step": 5050
+    },
+    {
+      "epoch": 0.86,
+      "learning_rate": 1.854972964477386e-05,
+      "loss": 0.0079,
+      "step": 5100
+    },
+    {
+      "epoch": 0.87,
+      "learning_rate": 1.835457034499991e-05,
+      "loss": 0.0083,
+      "step": 5150
+    },
+    {
+      "epoch": 0.88,
+      "learning_rate": 1.8158811435897493e-05,
+      "loss": 0.0081,
+      "step": 5200
+    },
+    {
+      "epoch": 0.88,
+      "learning_rate": 1.7962487908197434e-05,
+      "loss": 0.008,
+      "step": 5250
+    },
+    {
+      "epoch": 0.89,
+      "learning_rate": 1.7765634853552764e-05,
+      "loss": 0.0079,
+      "step": 5300
+    },
+    {
+      "epoch": 0.9,
+      "learning_rate": 1.7568287458266282e-05,
+      "loss": 0.0079,
+      "step": 5350
+    },
+    {
+      "epoch": 0.91,
+      "learning_rate": 1.7370480997001206e-05,
+      "loss": 0.0078,
+      "step": 5400
+    },
+    {
+      "epoch": 0.92,
+      "learning_rate": 1.717225082647604e-05,
+      "loss": 0.008,
+      "step": 5450
+    },
+    {
+      "epoch": 0.93,
+      "learning_rate": 1.6973632379144785e-05,
+      "loss": 0.008,
+      "step": 5500
+    },
+    {
+      "epoch": 0.93,
+      "eval_ap_CMedQAv1": 0.8627846587717232,
+      "eval_ap_CMedQAv2": 0.8679932179664334,
+      "eval_ap_Mmarco": 0.34792586376372187,
+      "eval_ap_T2Reranking": 0.6833881748363431,
+      "eval_avg_ap": 0.6905229788345554,
+      "eval_loss": 0.1255130171775818,
+      "eval_mrr_CMedQAv1": 0.8866321428571429,
+      "eval_mrr_CMedQAv2": 0.8911746031746032,
+      "eval_mrr_Mmarco": 0.3351349206349207,
+      "eval_mrr_T2Reranking": 0.7994619214194367,
+      "eval_ndcg@10_CMedQAv1": 0.9803870916366577,
+      "eval_ndcg@10_CMedQAv2": 0.992663562297821,
+      "eval_ndcg@10_Mmarco": 0.18294155597686768,
+      "eval_ndcg@10_T2Reranking": 0.5986461043357849,
+      "eval_ndcg@1_CMedQAv1": 1.0,
+      "eval_ndcg@1_CMedQAv2": 1.0,
+      "eval_ndcg@1_Mmarco": 0.30000001192092896,
+      "eval_ndcg@1_T2Reranking": 0.6328888535499573,
+      "eval_ndcg@3_CMedQAv1": 0.9999998807907104,
+      "eval_ndcg@3_CMedQAv2": 0.9999998807907104,
+      "eval_ndcg@3_Mmarco": 0.24078361690044403,
+      "eval_ndcg@3_T2Reranking": 0.6359131336212158,
+      "eval_ndcg@5_CMedQAv1": 0.9999998807907104,
+      "eval_ndcg@5_CMedQAv2": 0.9999998807907104,
+      "eval_ndcg@5_Mmarco": 0.21561172604560852,
+      "eval_ndcg@5_T2Reranking": 0.6260303258895874,
+      "eval_ndcg_CMedQAv1": 0.9603596925735474,
+      "eval_ndcg_CMedQAv2": 0.9649822115898132,
+      "eval_ndcg_Mmarco": 0.46445542573928833,
+      "eval_ndcg_T2Reranking": 0.8971077799797058,
+      "eval_runtime": 1115.45,
+      "eval_samples_per_second": 358.623,
+      "eval_steps_per_second": 0.351,
+      "step": 5500
+    },
+    {
+      "epoch": 0.94,
+      "learning_rate": 1.677466115686359e-05,
+      "loss": 0.0077,
+      "step": 5550
+    },
+    {
+      "epoch": 0.94,
+      "learning_rate": 1.6575372724545014e-05,
+      "loss": 0.0079,
+      "step": 5600
+    },
+    {
+      "epoch": 0.95,
+      "learning_rate": 1.6375802703801003e-05,
+      "loss": 0.008,
+      "step": 5650
+    },
+    {
+      "epoch": 0.96,
+      "learning_rate": 1.6175986766575735e-05,
+      "loss": 0.0078,
+      "step": 5700
+    },
+    {
+      "epoch": 0.97,
+      "learning_rate": 1.5975960628769506e-05,
+      "loss": 0.0081,
+      "step": 5750
+    },
+    {
+      "epoch": 0.98,
+      "learning_rate": 1.5775760043854687e-05,
+      "loss": 0.0077,
+      "step": 5800
+    },
+    {
+      "epoch": 0.99,
+      "learning_rate": 1.5575420796485038e-05,
+      "loss": 0.008,
+      "step": 5850
+    },
+    {
+      "epoch": 0.99,
+      "learning_rate": 1.5374978696099378e-05,
+      "loss": 0.0078,
+      "step": 5900
+    },
+    {
+      "epoch": 1.0,
+      "learning_rate": 1.5174469570520917e-05,
+      "loss": 0.0074,
+      "step": 5950
+    },
+    {
+      "epoch": 1.01,
+      "learning_rate": 1.4973929259553187e-05,
+      "loss": 0.0063,
+      "step": 6000
+    },
+    {
+      "epoch": 1.01,
+      "eval_ap_CMedQAv1": 0.8625306376678427,
+      "eval_ap_CMedQAv2": 0.8664329707401012,
+      "eval_ap_Mmarco": 0.34889147093898054,
+      "eval_ap_T2Reranking": 0.6829168328093741,
+      "eval_avg_ap": 0.6901929780390745,
+      "eval_loss": 0.12305427342653275,
+      "eval_mrr_CMedQAv1": 0.8871357142857142,
+      "eval_mrr_CMedQAv2": 0.8890535714285714,
+      "eval_mrr_Mmarco": 0.3384365079365079,
+      "eval_mrr_T2Reranking": 0.7948461058989156,
+      "eval_ndcg@10_CMedQAv1": 0.9933746457099915,
+      "eval_ndcg@10_CMedQAv2": 0.978777289390564,
+      "eval_ndcg@10_Mmarco": 0.15859688818454742,
+      "eval_ndcg@10_T2Reranking": 0.5381678938865662,
+      "eval_ndcg@1_CMedQAv1": 1.0,
+      "eval_ndcg@1_CMedQAv2": 1.0,
+      "eval_ndcg@1_Mmarco": 0.20000000298023224,
+      "eval_ndcg@1_T2Reranking": 0.4707619249820709,
+      "eval_ndcg@3_CMedQAv1": 0.9999998807907104,
+      "eval_ndcg@3_CMedQAv2": 0.9703917503356934,
+      "eval_ndcg@3_Mmarco": 0.17039181292057037,
+      "eval_ndcg@3_T2Reranking": 0.4994679093360901,
+      "eval_ndcg@5_CMedQAv1": 0.9999998807907104,
+      "eval_ndcg@5_CMedQAv2": 0.978601336479187,
+      "eval_ndcg@5_Mmarco": 0.18008770048618317,
+      "eval_ndcg@5_T2Reranking": 0.5133092999458313,
+      "eval_ndcg_CMedQAv1": 0.9596377611160278,
+      "eval_ndcg_CMedQAv2": 0.9619806408882141,
+      "eval_ndcg_Mmarco": 0.4373508393764496,
+      "eval_ndcg_T2Reranking": 0.8974445462226868,
+      "eval_runtime": 1135.765,
+      "eval_samples_per_second": 352.208,
+      "eval_steps_per_second": 0.344,
+      "step": 6000
+    },
+    {
+      "epoch": 1.02,
+      "learning_rate": 1.4773393608573946e-05,
+      "loss": 0.0064,
+      "step": 6050
+    },
+    {
+      "epoch": 1.03,
+      "learning_rate": 1.4572898462127985e-05,
+      "loss": 0.0066,
+      "step": 6100
+    },
+    {
+      "epoch": 1.04,
+      "learning_rate": 1.437247965752017e-05,
+      "loss": 0.0066,
+      "step": 6150
+    },
+    {
+      "epoch": 1.04,
+      "learning_rate": 1.4172173018409708e-05,
+      "loss": 0.0066,
+      "step": 6200
+    },
+    {
+      "epoch": 1.05,
+      "learning_rate": 1.3972014348406904e-05,
+      "loss": 0.0067,
+      "step": 6250
+    },
+    {
+      "epoch": 1.06,
+      "learning_rate": 1.377203942467347e-05,
+      "loss": 0.0064,
+      "step": 6300
+    },
+    {
+      "epoch": 1.07,
+      "learning_rate": 1.3572283991527582e-05,
+      "loss": 0.0064,
+      "step": 6350
+    },
+    {
+      "epoch": 1.08,
+      "learning_rate": 1.3372783754054776e-05,
+      "loss": 0.0064,
+      "step": 6400
+    },
+    {
+      "epoch": 1.09,
+      "learning_rate": 1.3173574371725902e-05,
+      "loss": 0.0064,
+      "step": 6450
+    },
+    {
+      "epoch": 1.1,
+      "learning_rate": 1.2974691452023195e-05,
+      "loss": 0.0065,
+      "step": 6500
+    },
+    {
+      "epoch": 1.1,
+      "eval_ap_CMedQAv1": 0.8620322442191521,
+      "eval_ap_CMedQAv2": 0.8704041872289263,
+      "eval_ap_Mmarco": 0.355717135660246,
+      "eval_ap_T2Reranking": 0.6809892589587305,
+      "eval_avg_ap": 0.6922857065167638,
+      "eval_loss": 0.12283767759799957,
+      "eval_mrr_CMedQAv1": 0.8863039682539682,
+      "eval_mrr_CMedQAv2": 0.8930746031746032,
+      "eval_mrr_Mmarco": 0.34052380952380956,
+      "eval_mrr_T2Reranking": 0.7916278438705656,
+      "eval_ndcg@10_CMedQAv1": 0.989758312702179,
+      "eval_ndcg@10_CMedQAv2": 0.983578085899353,
+      "eval_ndcg@10_Mmarco": 0.16711477935314178,
+      "eval_ndcg@10_T2Reranking": 0.5755314230918884,
+      "eval_ndcg@1_CMedQAv1": 1.0,
+      "eval_ndcg@1_CMedQAv2": 1.0,
+      "eval_ndcg@1_Mmarco": 0.10000000149011612,
+      "eval_ndcg@1_T2Reranking": 0.5387619137763977,
+      "eval_ndcg@3_CMedQAv1": 0.9882680177688599,
+      "eval_ndcg@3_CMedQAv2": 0.9999998807907104,
+      "eval_ndcg@3_Mmarco": 0.1265360563993454,
+      "eval_ndcg@3_T2Reranking": 0.566280722618103,
+      "eval_ndcg@5_CMedQAv1": 0.9842175245285034,
+      "eval_ndcg@5_CMedQAv2": 0.9853931665420532,
+      "eval_ndcg@5_Mmarco": 0.15853983163833618,
+      "eval_ndcg@5_T2Reranking": 0.5730277299880981,
+      "eval_ndcg_CMedQAv1": 0.9608818292617798,
+      "eval_ndcg_CMedQAv2": 0.9639202952384949,
+      "eval_ndcg_Mmarco": 0.4360330104827881,
+      "eval_ndcg_T2Reranking": 0.8998947143554688,
+      "eval_runtime": 1071.1802,
+      "eval_samples_per_second": 373.444,
+      "eval_steps_per_second": 0.365,
+      "step": 6500
+    },
+    {
+      "epoch": 1.1,
+      "learning_rate": 1.277617054407565e-05,
+      "loss": 0.0066,
+      "step": 6550
+    },
+    {
+      "epoch": 1.11,
+      "learning_rate": 1.2578047132304843e-05,
+      "loss": 0.0065,
+      "step": 6600
+    },
+    {
+      "epoch": 1.12,
+      "learning_rate": 1.2380356630082277e-05,
+      "loss": 0.0064,
+      "step": 6650
+    },
+    {
+      "epoch": 1.13,
+      "learning_rate": 1.2183134373399479e-05,
+      "loss": 0.0066,
+      "step": 6700
+    },
+    {
+      "epoch": 1.14,
+      "learning_rate": 1.1986415614551897e-05,
+      "loss": 0.0068,
+      "step": 6750
+    },
+    {
+      "epoch": 1.15,
+      "learning_rate": 1.1790235515837761e-05,
+      "loss": 0.0065,
+      "step": 6800
+    },
+    {
+      "epoch": 1.15,
+      "learning_rate": 1.1594629143273021e-05,
+      "loss": 0.0067,
+      "step": 6850
+    },
+    {
+      "epoch": 1.16,
+      "learning_rate": 1.1399631460323536e-05,
+      "loss": 0.0066,
+      "step": 6900
+    },
+    {
+      "epoch": 1.17,
+      "learning_rate": 1.1205277321655528e-05,
+      "loss": 0.0065,
+      "step": 6950
+    },
+    {
+      "epoch": 1.18,
+      "learning_rate": 1.1011601466905561e-05,
+      "loss": 0.0065,
+      "step": 7000
+    },
+    {
+      "epoch": 1.18,
+      "eval_ap_CMedQAv1": 0.856229674863834,
+      "eval_ap_CMedQAv2": 0.8728030227219203,
+      "eval_ap_Mmarco": 0.35545981895349504,
+      "eval_ap_T2Reranking": 0.6844460182092421,
+      "eval_avg_ap": 0.6922346336871228,
+      "eval_loss": 0.12674953043460846,
+      "eval_mrr_CMedQAv1": 0.8826107142857142,
+      "eval_mrr_CMedQAv2": 0.8945480158730159,
+      "eval_mrr_Mmarco": 0.34125,
+      "eval_mrr_T2Reranking": 0.7965651900570655,
+      "eval_ndcg@10_CMedQAv1": 0.9865903854370117,
+      "eval_ndcg@10_CMedQAv2": 0.9811555743217468,
+      "eval_ndcg@10_Mmarco": 0.1592722088098526,
+      "eval_ndcg@10_T2Reranking": 0.5544707179069519,
+      "eval_ndcg@1_CMedQAv1": 1.0,
+      "eval_ndcg@1_CMedQAv2": 1.0,
+      "eval_ndcg@1_Mmarco": 0.25,
+      "eval_ndcg@1_T2Reranking": 0.5022221803665161,
+      "eval_ndcg@3_CMedQAv1": 0.9999998807907104,
+      "eval_ndcg@3_CMedQAv2": 0.976535975933075,
+      "eval_ndcg@3_Mmarco": 0.2086598426103592,
+      "eval_ndcg@3_T2Reranking": 0.5088227987289429,
+      "eval_ndcg@5_CMedQAv1": 0.9999998807907104,
+      "eval_ndcg@5_CMedQAv2": 0.9830419421195984,
+      "eval_ndcg@5_Mmarco": 0.17853133380413055,
+      "eval_ndcg@5_T2Reranking": 0.5329388380050659,
+      "eval_ndcg_CMedQAv1": 0.9582996368408203,
+      "eval_ndcg_CMedQAv2": 0.962376594543457,
+      "eval_ndcg_Mmarco": 0.4456784129142761,
+      "eval_ndcg_T2Reranking": 0.9012719988822937,
+      "eval_runtime": 1104.857,
+      "eval_samples_per_second": 362.061,
+      "eval_steps_per_second": 0.354,
+      "step": 7000
+    },
+    {
+      "epoch": 1.19,
+      "learning_rate": 1.0818638514470987e-05,
+      "loss": 0.0066,
+      "step": 7050
+    },
+    {
+      "epoch": 1.2,
+      "learning_rate": 1.0626422955322185e-05,
+      "loss": 0.0066,
+      "step": 7100
+    },
+    {
+      "epoch": 1.2,
+      "learning_rate": 1.0434989146837435e-05,
+      "loss": 0.0065,
+      "step": 7150
+    },
+    {
+      "epoch": 1.21,
+      "learning_rate": 1.0244371306661786e-05,
+      "loss": 0.0066,
+      "step": 7200
+    },
+    {
+      "epoch": 1.22,
+      "learning_rate": 1.0054603506590841e-05,
+      "loss": 0.0065,
+      "step": 7250
+    },
+    {
+      "epoch": 1.23,
+      "learning_rate": 9.865719666480642e-06,
+      "loss": 0.0064,
+      "step": 7300
+    },
+    {
+      "epoch": 1.24,
+      "learning_rate": 9.677753548184684e-06,
+      "loss": 0.0067,
+      "step": 7350
+    },
+    {
+      "epoch": 1.25,
+      "learning_rate": 9.490738749519188e-06,
+      "loss": 0.0065,
+      "step": 7400
+    },
+    {
+      "epoch": 1.26,
+      "learning_rate": 9.30470869825771e-06,
+      "loss": 0.0063,
+      "step": 7450
+    },
+    {
+      "epoch": 1.26,
+      "learning_rate": 9.119696646156103e-06,
+      "loss": 0.0066,
+      "step": 7500
+    },
+    {
+      "epoch": 1.26,
+      "eval_ap_CMedQAv1": 0.8604172146745557,
+      "eval_ap_CMedQAv2": 0.8725561316868456,
+      "eval_ap_Mmarco": 0.3400242765180377,
+      "eval_ap_T2Reranking": 0.6826462319596516,
+      "eval_avg_ap": 0.6889109637097726,
+      "eval_loss": 0.1242843046784401,
+      "eval_mrr_CMedQAv1": 0.8834670634920635,
+      "eval_mrr_CMedQAv2": 0.8959575396825398,
+      "eval_mrr_Mmarco": 0.3298134920634921,
+      "eval_mrr_T2Reranking": 0.7943680750341211,
+      "eval_ndcg@10_CMedQAv1": 0.9862348437309265,
+      "eval_ndcg@10_CMedQAv2": 0.9811555743217468,
+      "eval_ndcg@10_Mmarco": 0.1542401760816574,
+      "eval_ndcg@10_T2Reranking": 0.6371672749519348,
+      "eval_ndcg@1_CMedQAv1": 1.0,
+      "eval_ndcg@1_CMedQAv2": 1.0,
+      "eval_ndcg@1_Mmarco": 0.10000000149011612,
+      "eval_ndcg@1_T2Reranking": 0.7219048738479614,
+      "eval_ndcg@3_CMedQAv1": 0.9999998807907104,
+      "eval_ndcg@3_CMedQAv2": 0.976535975933075,
+      "eval_ndcg@3_Mmarco": 0.15921637415885925,
+      "eval_ndcg@3_T2Reranking": 0.6977660655975342,
+      "eval_ndcg@5_CMedQAv1": 0.9999998807907104,
+      "eval_ndcg@5_CMedQAv2": 0.9830419421195984,
+      "eval_ndcg@5_Mmarco": 0.1712677776813507,
+      "eval_ndcg@5_T2Reranking": 0.6631223559379578,
+      "eval_ndcg_CMedQAv1": 0.9619690179824829,
+      "eval_ndcg_CMedQAv2": 0.965739369392395,
+      "eval_ndcg_Mmarco": 0.43817299604415894,
+      "eval_ndcg_T2Reranking": 0.9018659591674805,
+      "eval_runtime": 1054.3338,
+      "eval_samples_per_second": 379.411,
+      "eval_steps_per_second": 0.371,
+      "step": 7500
+    },
+    {
+      "epoch": 1.27,
+      "learning_rate": 8.935735663008975e-06,
+      "loss": 0.0065,
+      "step": 7550
+    },
+    {
+      "epoch": 1.28,
+      "learning_rate": 8.752858630738673e-06,
+      "loss": 0.0067,
+      "step": 7600
+    },
+    {
+      "epoch": 1.29,
+      "learning_rate": 8.57109823751782e-06,
+      "loss": 0.0063,
+      "step": 7650
+    },
+    {
+      "epoch": 1.3,
+      "learning_rate": 8.390486971926502e-06,
+      "loss": 0.0065,
+      "step": 7700
+    },
+    {
+      "epoch": 1.31,
+      "learning_rate": 8.211057117145137e-06,
+      "loss": 0.0063,
+      "step": 7750
+    },
+    {
+      "epoch": 1.31,
+      "learning_rate": 8.03284074518405e-06,
+      "loss": 0.0065,
+      "step": 7800
+    },
+    {
+      "epoch": 1.32,
+      "learning_rate": 7.855869711150798e-06,
+      "loss": 0.0066,
+      "step": 7850
+    },
+    {
+      "epoch": 1.33,
+      "learning_rate": 7.680175647556236e-06,
+      "loss": 0.0065,
+      "step": 7900
+    },
+    {
+      "epoch": 1.34,
+      "learning_rate": 7.505789958660412e-06,
+      "loss": 0.0065,
+      "step": 7950
+    },
+    {
+      "epoch": 1.35,
+      "learning_rate": 7.332743814859266e-06,
+      "loss": 0.0066,
+      "step": 8000
+    },
+    {
+      "epoch": 1.35,
+      "eval_ap_CMedQAv1": 0.8599415671346007,
+      "eval_ap_CMedQAv2": 0.8704569674862218,
+      "eval_ap_Mmarco": 0.3520724310208005,
+      "eval_ap_T2Reranking": 0.6867254999404075,
+      "eval_avg_ap": 0.6922991163955077,
+      "eval_loss": 0.1281006932258606,
+      "eval_mrr_CMedQAv1": 0.8837103174603175,
+      "eval_mrr_CMedQAv2": 0.8930769841269841,
+      "eval_mrr_Mmarco": 0.3405515873015873,
+      "eval_mrr_T2Reranking": 0.7984008097709858,
+      "eval_ndcg@10_CMedQAv1": 0.9783665537834167,
+      "eval_ndcg@10_CMedQAv2": 0.9797147512435913,
+      "eval_ndcg@10_Mmarco": 0.1712549477815628,
+      "eval_ndcg@10_T2Reranking": 0.5799515843391418,
+      "eval_ndcg@1_CMedQAv1": 1.0,
+      "eval_ndcg@1_CMedQAv2": 1.0,
+      "eval_ndcg@1_Mmarco": 0.10000000149011612,
+      "eval_ndcg@1_T2Reranking": 0.5411046147346497,
+      "eval_ndcg@3_CMedQAv1": 0.9999998807907104,
+      "eval_ndcg@3_CMedQAv2": 0.9734638929367065,
+      "eval_ndcg@3_Mmarco": 0.22653606534004211,
+      "eval_ndcg@3_T2Reranking": 0.568535327911377,
+      "eval_ndcg@5_CMedQAv1": 0.9868794679641724,
+      "eval_ndcg@5_CMedQAv2": 0.9808216094970703,
+      "eval_ndcg@5_Mmarco": 0.18608656525611877,
+      "eval_ndcg@5_T2Reranking": 0.5744965672492981,
+      "eval_ndcg_CMedQAv1": 0.9611911773681641,
+      "eval_ndcg_CMedQAv2": 0.9657419323921204,
+      "eval_ndcg_Mmarco": 0.4481244683265686,
+      "eval_ndcg_T2Reranking": 0.9010647535324097,
+      "eval_runtime": 1049.8477,
+      "eval_samples_per_second": 381.032,
+      "eval_steps_per_second": 0.372,
+      "step": 8000
+    },
+    {
+      "epoch": 1.36,
+      "learning_rate": 7.161068147113065e-06,
+      "loss": 0.0066,
+      "step": 8050
+    },
+    {
+      "epoch": 1.37,
+      "learning_rate": 6.990793641417708e-06,
+      "loss": 0.0065,
+      "step": 8100
+    },
+    {
+      "epoch": 1.37,
+      "learning_rate": 6.821950733319783e-06,
+      "loss": 0.0064,
+      "step": 8150
+    },
+    {
+      "epoch": 1.38,
+      "learning_rate": 6.654569602476402e-06,
+      "loss": 0.0064,
+      "step": 8200
+    },
+    {
+      "epoch": 1.39,
+      "learning_rate": 6.488680167260749e-06,
+      "loss": 0.0067,
+      "step": 8250
+    },
+    {
+      "epoch": 1.4,
+      "learning_rate": 6.324312079414362e-06,
+      "loss": 0.0066,
+      "step": 8300
+    },
+    {
+      "epoch": 1.41,
+      "learning_rate": 6.161494718747061e-06,
+      "loss": 0.0067,
+      "step": 8350
+    },
+    {
+      "epoch": 1.42,
+      "learning_rate": 6.000257187885497e-06,
+      "loss": 0.0066,
+      "step": 8400
+    },
+    {
+      "epoch": 1.42,
+      "learning_rate": 5.8406283070712074e-06,
+      "loss": 0.0065,
+      "step": 8450
+    },
+    {
+      "epoch": 1.43,
+      "learning_rate": 5.682636609009177e-06,
+      "loss": 0.0067,
+      "step": 8500
+    },
+    {
+      "epoch": 1.43,
+      "eval_ap_CMedQAv1": 0.8650759729076953,
+      "eval_ap_CMedQAv2": 0.8749745804892705,
+      "eval_ap_Mmarco": 0.3538804931837119,
+      "eval_ap_T2Reranking": 0.6878922264348706,
+      "eval_avg_ap": 0.695455818253887,
+      "eval_loss": 0.12287386506795883,
+      "eval_mrr_CMedQAv1": 0.8906345238095239,
+      "eval_mrr_CMedQAv2": 0.8960039682539682,
+      "eval_mrr_Mmarco": 0.34160714285714294,
+      "eval_mrr_T2Reranking": 0.7982257713511944,
+      "eval_ndcg@10_CMedQAv1": 1.0000001192092896,
+      "eval_ndcg@10_CMedQAv2": 0.9829331636428833,
+      "eval_ndcg@10_Mmarco": 0.19680270552635193,
+      "eval_ndcg@10_T2Reranking": 0.5907678008079529,
+      "eval_ndcg@1_CMedQAv1": 1.0,
+      "eval_ndcg@1_CMedQAv2": 1.0,
+      "eval_ndcg@1_Mmarco": 0.20000000298023224,
+      "eval_ndcg@1_T2Reranking": 0.5812433958053589,
+      "eval_ndcg@3_CMedQAv1": 0.9999998807907104,
+      "eval_ndcg@3_CMedQAv2": 0.9999998807907104,
+      "eval_ndcg@3_Mmarco": 0.22346392273902893,
+      "eval_ndcg@3_T2Reranking": 0.5841401815414429,
+      "eval_ndcg@5_CMedQAv1": 0.9999998807907104,
+      "eval_ndcg@5_CMedQAv2": 0.9853931665420532,
+      "eval_ndcg@5_Mmarco": 0.21695800125598907,
+      "eval_ndcg@5_T2Reranking": 0.5929743051528931,
+      "eval_ndcg_CMedQAv1": 0.9629640579223633,
+      "eval_ndcg_CMedQAv2": 0.9660819172859192,
+      "eval_ndcg_Mmarco": 0.463792085647583,
+      "eval_ndcg_T2Reranking": 0.9027825593948364,
+      "eval_runtime": 1128.8704,
+      "eval_samples_per_second": 354.36,
+      "eval_steps_per_second": 0.346,
+      "step": 8500
+    },
+    {
+      "epoch": 1.44,
+      "learning_rate": 5.5263103337678074e-06,
+      "loss": 0.0065,
+      "step": 8550
+    },
+    {
+      "epoch": 1.45,
+      "learning_rate": 5.371677423731162e-06,
+      "loss": 0.0064,
+      "step": 8600
+    },
+    {
+      "epoch": 1.46,
+      "learning_rate": 5.2187655186044135e-06,
+      "loss": 0.0063,
+      "step": 8650
+    },
+    {
+      "epoch": 1.47,
+      "learning_rate": 5.067601950473435e-06,
+      "loss": 0.0067,
+      "step": 8700
+    },
+    {
+      "epoch": 1.47,
+      "learning_rate": 4.918213738919363e-06,
+      "loss": 0.0064,
+      "step": 8750
+    },
+    {
+      "epoch": 1.48,
+      "learning_rate": 4.770627586188978e-06,
+      "loss": 0.0063,
+      "step": 8800
+    },
+    {
+      "epoch": 1.49,
+      "learning_rate": 4.624869872421859e-06,
+      "loss": 0.0064,
+      "step": 8850
+    },
+    {
+      "epoch": 1.5,
+      "learning_rate": 4.4809666509350785e-06,
+      "loss": 0.0063,
+      "step": 8900
+    },
+    {
+      "epoch": 1.51,
+      "learning_rate": 4.338943643566367e-06,
+      "loss": 0.0065,
+      "step": 8950
+    },
+    {
+      "epoch": 1.52,
+      "learning_rate": 4.1988262360764306e-06,
+      "loss": 0.0065,
+      "step": 9000
+    },
+    {
+      "epoch": 1.52,
+      "eval_ap_CMedQAv1": 0.8641924375078601,
+      "eval_ap_CMedQAv2": 0.8717899695221966,
+      "eval_ap_Mmarco": 0.37013444296743075,
+      "eval_ap_T2Reranking": 0.6848922218837102,
+      "eval_avg_ap": 0.6977522679702994,
+      "eval_loss": 0.12518064677715302,
+      "eval_mrr_CMedQAv1": 0.8881654761904761,
+      "eval_mrr_CMedQAv2": 0.8928563492063493,
+      "eval_mrr_Mmarco": 0.3619960317460318,
+      "eval_mrr_T2Reranking": 0.7966605006931683,
+      "eval_ndcg@10_CMedQAv1": 0.9898759126663208,
+      "eval_ndcg@10_CMedQAv2": 0.9804811477661133,
+      "eval_ndcg@10_Mmarco": 0.21500129997730255,
+      "eval_ndcg@10_T2Reranking": 0.5945191979408264,
+      "eval_ndcg@1_CMedQAv1": 1.0,
+      "eval_ndcg@1_CMedQAv2": 1.0,
+      "eval_ndcg@1_Mmarco": 0.20000000298023224,
+      "eval_ndcg@1_T2Reranking": 0.5946031808853149,
+      "eval_ndcg@3_CMedQAv1": 0.9999998807907104,
+      "eval_ndcg@3_CMedQAv2": 0.976535975933075,
+      "eval_ndcg@3_Mmarco": 0.2357524186372757,
+      "eval_ndcg@3_T2Reranking": 0.6052175760269165,
+      "eval_ndcg@5_CMedQAv1": 0.9999998807907104,
+      "eval_ndcg@5_CMedQAv2": 0.9699214696884155,
+      "eval_ndcg@5_Mmarco": 0.22286656498908997,
+      "eval_ndcg@5_T2Reranking": 0.6057425737380981,
+      "eval_ndcg_CMedQAv1": 0.9635534286499023,
+      "eval_ndcg_CMedQAv2": 0.9656587839126587,
+      "eval_ndcg_Mmarco": 0.46775779128074646,
+      "eval_ndcg_T2Reranking": 0.9013978838920593,
+      "eval_runtime": 1073.8791,
+      "eval_samples_per_second": 372.506,
+      "eval_steps_per_second": 0.364,
+      "step": 9000
+    },
+    {
+      "epoch": 1.53,
+      "learning_rate": 4.060639473611431e-06,
+      "loss": 0.0064,
+      "step": 9050
+    },
+    {
+      "epoch": 1.53,
+      "learning_rate": 3.924408056226315e-06,
+      "loss": 0.0063,
+      "step": 9100
+    },
+    {
+      "epoch": 1.54,
+      "learning_rate": 3.7901563344698305e-06,
+      "loss": 0.0064,
+      "step": 9150
+    },
+    {
+      "epoch": 1.55,
+      "learning_rate": 3.6579083050319985e-06,
+      "loss": 0.0063,
+      "step": 9200
+    },
+    {
+      "epoch": 1.56,
+      "learning_rate": 3.5276876064548523e-06,
+      "loss": 0.0064,
+      "step": 9250
+    },
+    {
+      "epoch": 1.57,
+      "learning_rate": 3.3995175149072066e-06,
+      "loss": 0.0064,
+      "step": 9300
+    },
+    {
+      "epoch": 1.58,
+      "learning_rate": 3.273420940024165e-06,
+      "loss": 0.0064,
+      "step": 9350
+    },
+    {
+      "epoch": 1.58,
+      "learning_rate": 3.149420420812157e-06,
+      "loss": 0.0064,
+      "step": 9400
+    },
+    {
+      "epoch": 1.59,
+      "learning_rate": 3.0275381216202334e-06,
+      "loss": 0.0064,
+      "step": 9450
+    },
+    {
+      "epoch": 1.6,
+      "learning_rate": 2.907795828178335e-06,
+      "loss": 0.0064,
+      "step": 9500
+    },
+    {
+      "epoch": 1.6,
+      "eval_ap_CMedQAv1": 0.8649280235435451,
+      "eval_ap_CMedQAv2": 0.8765992249434462,
+      "eval_ap_Mmarco": 0.37276029660147,
+      "eval_ap_T2Reranking": 0.6854808029670674,
+      "eval_avg_ap": 0.6999420870138822,
+      "eval_loss": 0.12521055340766907,
+      "eval_mrr_CMedQAv1": 0.8882769841269842,
+      "eval_mrr_CMedQAv2": 0.8982996031746031,
+      "eval_mrr_Mmarco": 0.36348015873015876,
+      "eval_mrr_T2Reranking": 0.7960031998581424,
+      "eval_ndcg@10_CMedQAv1": 0.9872758984565735,
+      "eval_ndcg@10_CMedQAv2": 0.9811555743217468,
+      "eval_ndcg@10_Mmarco": 0.2056836634874344,
+      "eval_ndcg@10_T2Reranking": 0.5942399501800537,
+      "eval_ndcg@1_CMedQAv1": 1.0,
+      "eval_ndcg@1_CMedQAv2": 1.0,
+      "eval_ndcg@1_Mmarco": 0.20000000298023224,
+      "eval_ndcg@1_T2Reranking": 0.5888352394104004,
+      "eval_ndcg@3_CMedQAv1": 0.9999998807907104,
+      "eval_ndcg@3_CMedQAv2": 0.976535975933075,
+      "eval_ndcg@3_Mmarco": 0.2765360474586487,
+      "eval_ndcg@3_T2Reranking": 0.5885987281799316,
+      "eval_ndcg@5_CMedQAv1": 0.9999998807907104,
+      "eval_ndcg@5_CMedQAv2": 0.9830419421195984,
+      "eval_ndcg@5_Mmarco": 0.22907359898090363,
+      "eval_ndcg@5_T2Reranking": 0.5933831930160522,
+      "eval_ndcg_CMedQAv1": 0.962390124797821,
+      "eval_ndcg_CMedQAv2": 0.9656065106391907,
+      "eval_ndcg_Mmarco": 0.470781147480011,
+      "eval_ndcg_T2Reranking": 0.9014945030212402,
+      "eval_runtime": 1080.5882,
+      "eval_samples_per_second": 370.193,
+      "eval_steps_per_second": 0.362,
+      "step": 9500
+    },
+    {
+      "epoch": 1.61,
+      "learning_rate": 2.7902149437031954e-06,
+      "loss": 0.0067,
+      "step": 9550
+    },
+    {
+      "epoch": 1.62,
+      "learning_rate": 2.6748164850726625e-06,
+      "loss": 0.0065,
+      "step": 9600
+    },
+    {
+      "epoch": 1.63,
+      "learning_rate": 2.5616210790690604e-06,
+      "loss": 0.0063,
+      "step": 9650
+    },
+    {
+      "epoch": 1.63,
+      "learning_rate": 2.4506489586922726e-06,
+      "loss": 0.0064,
+      "step": 9700
+    },
+    {
+      "epoch": 1.64,
+      "learning_rate": 2.3419199595431993e-06,
+      "loss": 0.0064,
+      "step": 9750
+    },
+    {
+      "epoch": 1.65,
+      "learning_rate": 2.2354535162782867e-06,
+      "loss": 0.0064,
+      "step": 9800
+    },
+    {
+      "epoch": 1.66,
+      "learning_rate": 2.1312686591356766e-06,
+      "loss": 0.0064,
+      "step": 9850
+    },
+    {
+      "epoch": 1.67,
+      "learning_rate": 2.0293840105336916e-06,
+      "loss": 0.0063,
+      "step": 9900
+    },
+    {
+      "epoch": 1.68,
+      "learning_rate": 1.92981778174216e-06,
+      "loss": 0.0064,
+      "step": 9950
+    },
+    {
+      "epoch": 1.69,
+      "learning_rate": 1.8325877696272857e-06,
+      "loss": 0.0063,
+      "step": 10000
+    },
+    {
+      "epoch": 1.69,
+      "eval_ap_CMedQAv1": 0.8635020358013351,
+      "eval_ap_CMedQAv2": 0.8796776803107693,
+      "eval_ap_Mmarco": 0.36752299795574767,
+      "eval_ap_T2Reranking": 0.6861142187897954,
+      "eval_avg_ap": 0.6992042332144118,
+      "eval_loss": 0.12555988132953644,
+      "eval_mrr_CMedQAv1": 0.8863333333333334,
+      "eval_mrr_CMedQAv2": 0.900195238095238,
+      "eval_mrr_Mmarco": 0.3552460317460317,
+      "eval_mrr_T2Reranking": 0.7958933138816348,
+      "eval_ndcg@10_CMedQAv1": 0.9933746457099915,
+      "eval_ndcg@10_CMedQAv2": 0.9811555743217468,
+      "eval_ndcg@10_Mmarco": 0.2201658934354782,
+      "eval_ndcg@10_T2Reranking": 0.5817710757255554,
+      "eval_ndcg@1_CMedQAv1": 1.0,
+      "eval_ndcg@1_CMedQAv2": 1.0,
+      "eval_ndcg@1_Mmarco": 0.30000001192092896,
+      "eval_ndcg@1_T2Reranking": 0.5484000444412231,
+      "eval_ndcg@3_CMedQAv1": 0.9999998807907104,
+      "eval_ndcg@3_CMedQAv2": 0.976535975933075,
+      "eval_ndcg@3_Mmarco": 0.29999998211860657,
+      "eval_ndcg@3_T2Reranking": 0.5612119436264038,
+      "eval_ndcg@5_CMedQAv1": 0.9999998807907104,
+      "eval_ndcg@5_CMedQAv2": 0.9830419421195984,
+      "eval_ndcg@5_Mmarco": 0.22993843257427216,
+      "eval_ndcg@5_T2Reranking": 0.5722955465316772,
+      "eval_ndcg_CMedQAv1": 0.9632137417793274,
+      "eval_ndcg_CMedQAv2": 0.9660226106643677,
+      "eval_ndcg_Mmarco": 0.4788861870765686,
+      "eval_ndcg_T2Reranking": 0.9014593958854675,
+      "eval_runtime": 1096.8598,
+      "eval_samples_per_second": 364.701,
+      "eval_steps_per_second": 0.356,
+      "step": 10000
+    },
+    {
+      "epoch": 1.69,
+      "learning_rate": 1.7377113534705436e-06,
+      "loss": 0.0063,
+      "step": 10050
+    },
+    {
+      "epoch": 1.7,
+      "learning_rate": 1.64520549186226e-06,
+      "loss": 0.0063,
+      "step": 10100
+    },
+    {
+      "epoch": 1.71,
+      "learning_rate": 1.555086719670345e-06,
+      "loss": 0.0065,
+      "step": 10150
+    },
+    {
+      "epoch": 1.72,
+      "learning_rate": 1.467371145084792e-06,
+      "loss": 0.0064,
+      "step": 10200
+    },
+    {
+      "epoch": 1.73,
+      "learning_rate": 1.3820744467384483e-06,
+      "loss": 0.0061,
+      "step": 10250
+    },
+    {
+      "epoch": 1.74,
+      "learning_rate": 1.2992118709045309e-06,
+      "loss": 0.0062,
+      "step": 10300
+    },
+    {
+      "epoch": 1.74,
+      "learning_rate": 1.2187982287714573e-06,
+      "loss": 0.0064,
+      "step": 10350
+    },
+    {
+      "epoch": 1.75,
+      "learning_rate": 1.1408478937954458e-06,
+      "loss": 0.0062,
+      "step": 10400
+    },
+    {
+      "epoch": 1.76,
+      "learning_rate": 1.0653747991313201e-06,
+      "loss": 0.0062,
+      "step": 10450
+    },
+    {
+      "epoch": 1.77,
+      "learning_rate": 9.923924351420716e-07,
+      "loss": 0.006,
+      "step": 10500
+    },
+    {
+      "epoch": 1.77,
+      "eval_ap_CMedQAv1": 0.8637393453944692,
+      "eval_ap_CMedQAv2": 0.8789491384323506,
+      "eval_ap_Mmarco": 0.368903977630621,
+      "eval_ap_T2Reranking": 0.6868251658388673,
+      "eval_avg_ap": 0.699604406824077,
+      "eval_loss": 0.12616057693958282,
+      "eval_mrr_CMedQAv1": 0.8865428571428572,
+      "eval_mrr_CMedQAv2": 0.9001646825396825,
+      "eval_mrr_Mmarco": 0.3571944444444445,
+      "eval_mrr_T2Reranking": 0.7977927426894929,
+      "eval_ndcg@10_CMedQAv1": 0.9866948127746582,
+      "eval_ndcg@10_CMedQAv2": 0.9709935188293457,
+      "eval_ndcg@10_Mmarco": 0.19204413890838623,
+      "eval_ndcg@10_T2Reranking": 0.5846768617630005,
+      "eval_ndcg@1_CMedQAv1": 1.0,
+      "eval_ndcg@1_CMedQAv2": 1.0,
+      "eval_ndcg@1_Mmarco": 0.10000000149011612,
+      "eval_ndcg@1_T2Reranking": 0.5960000157356262,
+      "eval_ndcg@3_CMedQAv1": 0.9999998807907104,
+      "eval_ndcg@3_CMedQAv2": 0.9703917503356934,
+      "eval_ndcg@3_Mmarco": 0.26536059379577637,
+      "eval_ndcg@3_T2Reranking": 0.6046911478042603,
+      "eval_ndcg@5_CMedQAv1": 0.9999998807907104,
+      "eval_ndcg@5_CMedQAv2": 0.972041130065918,
+      "eval_ndcg@5_Mmarco": 0.20490367710590363,
+      "eval_ndcg@5_T2Reranking": 0.597877025604248,
+      "eval_ndcg_CMedQAv1": 0.9632848501205444,
+      "eval_ndcg_CMedQAv2": 0.9656885266304016,
+      "eval_ndcg_Mmarco": 0.463381290435791,
+      "eval_ndcg_T2Reranking": 0.902158260345459,
+      "eval_runtime": 1089.917,
+      "eval_samples_per_second": 367.024,
+      "eval_steps_per_second": 0.359,
+      "step": 10500
+    },
+    {
+      "epoch": 1.78,
+      "learning_rate": 9.21913846987511e-07,
+      "loss": 0.0064,
+      "step": 10550
+    },
+    {
+      "epoch": 1.79,
+      "learning_rate": 8.539516322925401e-07,
+      "loss": 0.0063,
+      "step": 10600
+    },
+    {
+      "epoch": 1.79,
+      "learning_rate": 7.885179388954022e-07,
+      "loss": 0.0062,
+      "step": 10650
+    },
+    {
+      "epoch": 1.8,
+      "learning_rate": 7.256244626763186e-07,
+      "loss": 0.0063,
+      "step": 10700
+    },
+    {
+      "epoch": 1.81,
+      "learning_rate": 6.652824454669315e-07,
+      "loss": 0.0065,
+      "step": 10750
+    },
+    {
+      "epoch": 1.82,
+      "learning_rate": 6.075026730408817e-07,
+      "loss": 0.0061,
+      "step": 10800
+    },
+    {
+      "epoch": 1.83,
+      "learning_rate": 5.522954731859342e-07,
+      "loss": 0.0063,
+      "step": 10850
+    },
+    {
+      "epoch": 1.84,
+      "learning_rate": 4.996707138579266e-07,
+      "loss": 0.0063,
+      "step": 10900
+    },
+    {
+      "epoch": 1.85,
+      "learning_rate": 4.4963780141694446e-07,
+      "loss": 0.0062,
+      "step": 10950
+    },
+    {
+      "epoch": 1.85,
+      "learning_rate": 4.022056789459921e-07,
+      "loss": 0.0061,
+      "step": 11000
+    },
+    {
+      "epoch": 1.85,
+      "eval_ap_CMedQAv1": 0.8634801543277222,
+      "eval_ap_CMedQAv2": 0.8789994898446902,
+      "eval_ap_Mmarco": 0.37314697568316435,
+      "eval_ap_T2Reranking": 0.6854003707502277,
+      "eval_avg_ap": 0.7002567476514512,
+      "eval_loss": 0.12499513477087021,
+      "eval_mrr_CMedQAv1": 0.8869642857142858,
+      "eval_mrr_CMedQAv2": 0.8993952380952381,
+      "eval_mrr_Mmarco": 0.3613452380952381,
+      "eval_mrr_T2Reranking": 0.7958653722152367,
+      "eval_ndcg@10_CMedQAv1": 0.9802283048629761,
+      "eval_ndcg@10_CMedQAv2": 0.9713308215141296,
+      "eval_ndcg@10_Mmarco": 0.19949547946453094,
+      "eval_ndcg@10_T2Reranking": 0.5712553262710571,
+      "eval_ndcg@1_CMedQAv1": 1.0,
+      "eval_ndcg@1_CMedQAv2": 1.0,
+      "eval_ndcg@1_Mmarco": 0.20000000298023224,
+      "eval_ndcg@1_T2Reranking": 0.6044243574142456,
+      "eval_ndcg@3_CMedQAv1": 0.9999998807907104,
+      "eval_ndcg@3_CMedQAv2": 0.9703917503356934,
+      "eval_ndcg@3_Mmarco": 0.28268030285835266,
+      "eval_ndcg@3_T2Reranking": 0.5774673223495483,
+      "eval_ndcg@5_CMedQAv1": 0.9999998807907104,
+      "eval_ndcg@5_CMedQAv2": 0.978601336479187,
+      "eval_ndcg@5_Mmarco": 0.23054155707359314,
+      "eval_ndcg@5_T2Reranking": 0.5735260844230652,
+      "eval_ndcg_CMedQAv1": 0.9631568789482117,
+      "eval_ndcg_CMedQAv2": 0.9658399820327759,
+      "eval_ndcg_Mmarco": 0.4707724452018738,
+      "eval_ndcg_T2Reranking": 0.902129054069519,
+      "eval_runtime": 1050.4339,
+      "eval_samples_per_second": 380.82,
+      "eval_steps_per_second": 0.372,
+      "step": 11000
+    }
+  ],
+  "logging_steps": 50,
+  "max_steps": 11868,
+  "num_train_epochs": 2,
+  "save_steps": 500,
+  "total_flos": 3.458164495011021e+16,
+  "trial_name": null,
+  "trial_params": null
+}