profoz/odsc-sawyer-sft

README.md

@@ -15,7 +15,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [bigscience/bloom-560m](https://huggingface.co/bigscience/bloom-560m) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 2.0050
+- Loss: 1.5514
 
 ## Model description
 
@@ -38,8 +38,8 @@ The following hyperparameters were used during training:
 - train_batch_size: 2
 - eval_batch_size: 4
 - seed: 42
-- gradient_accumulation_steps: 16
-- total_train_batch_size: 32
+- gradient_accumulation_steps: 32
+- total_train_batch_size: 64
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
 - lr_scheduler_warmup_ratio: 0.2
@@ -49,7 +49,7 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
-| 2.1116        | 1.0   | 54   | 2.0050          |
+| 1.5443        | 1.0   | 2770 | 1.5514          |
 
 
 ### Framework versions

adapter_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6cafb63baabc1c56cf1d1d4e9b02ed7307a899393488cc3e65aa1940f4e0f09c
+oid sha256:b2f4ae338aae2a56783f3dc1a4d5c8a457aaf1fa05f49d9f91a80a6e4547bc5f
 size 1049230477

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c99b5da63f3c72b923a283fe61f6c78031b32a87589875da9678a78868c53a4a
+oid sha256:73d6fd65e50a53be9db34ae3b92b179bbd67dd11abe12708d882540984f31b50
 size 4536