DavidAU commited on
Commit
9914ef6
·
verified ·
1 Parent(s): bd78bc4

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +7 -1
README.md CHANGED
@@ -133,10 +133,16 @@ dtype: bfloat16
133
 
134
  NOTE:
135
 
136
- You can replace the BASE model with Llama 3.1 to create a 128k version(s) of this model.
 
 
137
 
138
  Likewise, you can replace the BASE model with different Llama 3s to create different flavors of this model.
139
 
 
 
 
 
140
  You can of course, replace all the models too...
141
 
142
  !! => Each of the "weights" apply to 4 layers of the model (32/8), you can expand this to 32 to fine tune each layer of each model.
 
133
 
134
  NOTE:
135
 
136
+ You can replace the BASE model with Llama 3.1 model(s) to create a 128k version(s) of this model.
137
+
138
+ You can also replace the BASE model with Nivida 1,2 or 4 million "Llama 3.1" base models too.
139
 
140
  Likewise, you can replace the BASE model with different Llama 3s to create different flavors of this model.
141
 
142
+ In this merge, the "base" model is consumed (weights replaced) COMPLETELY, yet characteristics of the base model will remain.
143
+
144
+ If you alter the "weights" (each total 1 of the 3 models) to less than 1 -> the remaining will be "BASE" weights.
145
+
146
  You can of course, replace all the models too...
147
 
148
  !! => Each of the "weights" apply to 4 layers of the model (32/8), you can expand this to 32 to fine tune each layer of each model.