Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -10,6 +10,8 @@ license: mit
|
|
| 10 |
|
| 11 |
# Phi-1.5 TOFU Unlearning Model
|
| 12 |
|
|
|
|
|
|
|
| 13 |
This model is a variant of the Phi-1.5 model, fine-tuned on the TOFU (Task of Fictitious Unlearning) dataset and then subjected to various unlearning algorithms.
|
| 14 |
|
| 15 |
## Model Details
|
|
@@ -27,22 +29,24 @@ This model uses the `grad_ascent` unlearning algorithm with the following parame
|
|
| 27 |
|
| 28 |
## Revisions
|
| 29 |
|
| 30 |
-
The model is organized into multiple revisions, each representing a checkpoint during the unlearning process. The revision names follow the pattern `checkpoint-X`, where X is the checkpoint number.
|
| 31 |
|
| 32 |
## Loading the Model
|
| 33 |
|
| 34 |
-
To load a specific revision of this model, you
|
| 35 |
|
| 36 |
```python
|
| 37 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 38 |
|
| 39 |
-
# Replace 'checkpoint-X' with the desired revision (e.g., 'checkpoint-12')
|
| 40 |
revision = "checkpoint-X"
|
| 41 |
|
| 42 |
model = AutoModelForCausalLM.from_pretrained("locuslab/{model_name}", revision=revision)
|
| 43 |
tokenizer = AutoTokenizer.from_pretrained("locuslab/{model_name}", revision=revision)
|
| 44 |
```
|
| 45 |
|
|
|
|
|
|
|
| 46 |
## TOFU Dataset
|
| 47 |
|
| 48 |
TOFU (Task of Fictitious Unlearning) is a dataset designed for training and evaluating unlearning algorithms in language models. It simulates scenarios where certain information needs to be "forgotten" or removed from the model's knowledge.
|
|
@@ -51,7 +55,7 @@ TOFU (Task of Fictitious Unlearning) is a dataset designed for training and eval
|
|
| 51 |
|
| 52 |
1. The base Phi-1.5 model was first fine-tuned on the TOFU dataset (checkpoint-625).
|
| 53 |
2. Various unlearning algorithms were then applied to this fine-tuned model to selectively "forget" certain information.
|
| 54 |
-
3. The results of these unlearning processes are captured in the different revisions of this model.
|
| 55 |
|
| 56 |
## Usage and Limitations
|
| 57 |
|
|
|
|
| 10 |
|
| 11 |
# Phi-1.5 TOFU Unlearning Model
|
| 12 |
|
| 13 |
+
**IMPORTANT: This model's checkpoints are stored in separate branches. You MUST specify a revision when loading the model to access a specific checkpoint.**
|
| 14 |
+
|
| 15 |
This model is a variant of the Phi-1.5 model, fine-tuned on the TOFU (Task of Fictitious Unlearning) dataset and then subjected to various unlearning algorithms.
|
| 16 |
|
| 17 |
## Model Details
|
|
|
|
| 29 |
|
| 30 |
## Revisions
|
| 31 |
|
| 32 |
+
The model is organized into multiple revisions, each representing a checkpoint during the unlearning process. The revision names follow the pattern `checkpoint-X`, where X is the checkpoint number. Each revision is stored in a separate branch.
|
| 33 |
|
| 34 |
## Loading the Model
|
| 35 |
|
| 36 |
+
To load a specific revision of this model, you MUST specify the revision parameter. Use the following code:
|
| 37 |
|
| 38 |
```python
|
| 39 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 40 |
|
| 41 |
+
# The 'revision' parameter is REQUIRED. Replace 'checkpoint-X' with the desired revision (e.g., 'checkpoint-12')
|
| 42 |
revision = "checkpoint-X"
|
| 43 |
|
| 44 |
model = AutoModelForCausalLM.from_pretrained("locuslab/{model_name}", revision=revision)
|
| 45 |
tokenizer = AutoTokenizer.from_pretrained("locuslab/{model_name}", revision=revision)
|
| 46 |
```
|
| 47 |
|
| 48 |
+
**Note: If you don't specify a revision, you will not be able to load the model correctly.**
|
| 49 |
+
|
| 50 |
## TOFU Dataset
|
| 51 |
|
| 52 |
TOFU (Task of Fictitious Unlearning) is a dataset designed for training and evaluating unlearning algorithms in language models. It simulates scenarios where certain information needs to be "forgotten" or removed from the model's knowledge.
|
|
|
|
| 55 |
|
| 56 |
1. The base Phi-1.5 model was first fine-tuned on the TOFU dataset (checkpoint-625).
|
| 57 |
2. Various unlearning algorithms were then applied to this fine-tuned model to selectively "forget" certain information.
|
| 58 |
+
3. The results of these unlearning processes are captured in the different revisions (branches) of this model.
|
| 59 |
|
| 60 |
## Usage and Limitations
|
| 61 |
|