Finnish-NLP
/

Ahma-3B

Text Generation

text-generation-inference

Model card Files Files and versions

aapot commited on Mar 13, 2024

Commit

011e48d

·

1 Parent(s): 3657027

Fix

Files changed (1) hide show

EasyLM/data.py +2 -0

EasyLM/data.py CHANGED Viewed

@@ -217,6 +217,8 @@ class HuggingfaceDataset(object):
             if self._eval_dataset:
                 break
             else:
                 self._dataset_loc = 0
                 self._train_epochs += 1

             if self._eval_dataset:
                 break
             else:
+                if self._train_epochs == 0:
+                    self._dataset = self._dataset.shuffle(seed=42, buffer_size=10000)
                 self._dataset_loc = 0
                 self._train_epochs += 1