---
license: apache-2.0
base_model: TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T
tags:
- xcomet_xl_xxl
- generated_from_trainer
model-index:
- name: sft-xcomet_xl_xxl-chosen-10lp-shuff-full-tiny
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# sft-xcomet_xl_xxl-chosen-10lp-shuff-full-tiny

This model is a fine-tuned version of [TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T](https://huggingface.co/TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T) on the Unbabel/TowerAligned-v0.1 dataset.
It achieves the following results on the evaluation set:
- Loss: 0.7027
- Nll Loss: 0.7027
- Logps/best: -69.8057
- Rewards/chosen: 3.3548
- Rewards/rejected: 2.9021
- Rewards/accuracies: 0.6820
- Rewards/margins: 0.4527
- Logps/rejected: -68.4018
- Logps/chosen: -69.8057
- Logits/rejected: -1.7405
- Logits/chosen: -1.8685

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 1
- eval_batch_size: 4
- seed: 42
- gradient_accumulation_steps: 16
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.95) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 1

### Training results

| Training Loss | Epoch  | Step | Validation Loss | Nll Loss | Logps/best | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
|:-------------:|:------:|:----:|:---------------:|:--------:|:----------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
| 0.8021        | 0.1063 | 100  | 0.7701          | 0.7701   | -76.4054   | 2.6949         | 2.3664           | 0.6740             | 0.3284          | -73.7585       | -76.4054     | -1.7763         | -1.9055       |
| 0.7255        | 0.2127 | 200  | 0.7367          | 0.7367   | -73.1546   | 3.0200         | 2.6460           | 0.6820             | 0.3740          | -70.9634       | -73.1546     | -1.7637         | -1.8923       |
| 0.6979        | 0.3190 | 300  | 0.7232          | 0.7232   | -71.8372   | 3.1517         | 2.7499           | 0.6660             | 0.4018          | -69.9242       | -71.8372     | -1.7452         | -1.8727       |
| 0.7072        | 0.4254 | 400  | 0.7137          | 0.7137   | -70.8879   | 3.2466         | 2.8103           | 0.6960             | 0.4363          | -69.3198       | -70.8879     | -1.7467         | -1.8743       |
| 0.6958        | 0.5317 | 500  | 0.7085          | 0.7085   | -70.3945   | 3.2960         | 2.8412           | 0.6920             | 0.4548          | -69.0110       | -70.3945     | -1.7476         | -1.8756       |
| 0.7216        | 0.6381 | 600  | 0.7055          | 0.7055   | -70.0888   | 3.3265         | 2.8702           | 0.6900             | 0.4564          | -68.7212       | -70.0888     | -1.7377         | -1.8651       |
| 0.7531        | 0.7444 | 700  | 0.7038          | 0.7038   | -69.9193   | 3.3435         | 2.8863           | 0.6860             | 0.4572          | -68.5603       | -69.9193     | -1.7392         | -1.8670       |
| 0.6531        | 0.8508 | 800  | 0.7028          | 0.7028   | -69.8163   | 3.3538         | 2.9020           | 0.6800             | 0.4518          | -68.4026       | -69.8163     | -1.7410         | -1.8690       |
| 0.6801        | 0.9571 | 900  | 0.7027          | 0.7027   | -69.8057   | 3.3548         | 2.9021           | 0.6820             | 0.4527          | -68.4018       | -69.8057     | -1.7405         | -1.8685       |


### Framework versions

- Transformers 4.41.2
- Pytorch 2.1.2
- Datasets 2.20.0
- Tokenizers 0.19.1