Some weights of MultiTaskDistilBert were not initialized from the model checkpoint at distilbert-base-uncased and are newly initialized: ['classifier_interv.bias', 'classifier_interv.weight', 'classifier_main.bias', 'classifier_main.weight', 'classifier_priority.bias', 'classifier_priority.weight', 'classifier_sub.bias', 'classifier_sub.weight', 'pre_classifier.bias', 'pre_classifier.weight'] You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference. Using device: cuda category main_category 0 Drug/Alcohol Abuse Advice and Counselling 1 Homelessness Advice and Counselling 2 HIV/AIDS Advice and Counselling 3 Relationships (Student/Teacher) Advice and Counselling 4 Foster Care Child Maintenance & Custody Map: 0%| | 0/8810 [00:00 trainer.train() File "/home/rogendo/chl_scratch/mcreativity/lib/python3.12/site-packages/transformers/trainer.py", line 2240, in train return inner_training_loop( ^^^^^^^^^^^^^^^^^^^^ File "/home/rogendo/chl_scratch/mcreativity/lib/python3.12/site-packages/transformers/trainer.py", line 2656, in _inner_training_loop self._maybe_log_save_evaluate( File "/home/rogendo/chl_scratch/mcreativity/lib/python3.12/site-packages/transformers/trainer.py", line 3095, in _maybe_log_save_evaluate metrics = self._evaluate(trial, ignore_keys_for_eval) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/rogendo/chl_scratch/mcreativity/lib/python3.12/site-packages/transformers/trainer.py", line 3044, in _evaluate metrics = self.evaluate(ignore_keys=ignore_keys_for_eval) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/rogendo/chl_scratch/mcreativity/lib/python3.12/site-packages/transformers/trainer.py", line 4173, in evaluate output = eval_loop( ^^^^^^^^^^ File "/home/rogendo/chl_scratch/mcreativity/lib/python3.12/site-packages/transformers/trainer.py", line 4463, in evaluation_loop metrics = self.compute_metrics( ^^^^^^^^^^^^^^^^^^^^^ File "/home/rogendo/Work/Training/multitask_classifier_pipeline.py", line 319, in compute_metrics add_task_metrics("main", labels_main, preds_main) File "/home/rogendo/Work/Training/multitask_classifier_pipeline.py", line 312, in add_task_metrics precision, recall, f1, _ = precision_recall_fscore_support(labels, preds, average='weighted', zero_division=0) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ NameError: name 'precision_recall_fscore_support' is not defined 🏃 View run ./results at: http://192.168.8.18:5000/#/experiments/13/runs/90f04209b8fb49cd9f7d77e8422f2cbe 🧪 View experiment at: http://192.168.8.18:5000/#/experiments/13 8%|▊ | 551/6612 [02:51<31:25, 3.22it/s] Some weights of MultiTaskDistilBert were not initialized from the model checkpoint at distilbert-base-uncased and are newly initialized: ['classifier_interv.bias', 'classifier_interv.weight', 'classifier_main.bias', 'classifier_main.weight', 'classifier_priority.bias', 'classifier_priority.weight', 'classifier_sub.bias', 'classifier_sub.weight', 'pre_classifier.bias', 'pre_classifier.weight'] You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference. Using device: cuda category main_category 0 Drug/Alcohol Abuse Advice and Counselling 1 Homelessness Advice and Counselling 2 HIV/AIDS Advice and Counselling 3 Relationships (Student/Teacher) Advice and Counselling 4 Foster Care Child Maintenance & Custody Map: 0%| | 0/8810 [00:00