Gemma 3 version of this
This model is great !
Do you have any plans to make a Gemma 3 version of it ?
What are all the different Gemma 3 models you have made ?
Thanks!
I did actually try to make a gemma 3 version using the same training recipe, but it didn't have the same magic. I'll hopefully try again when I have some spare time.
All those other models are experiments for a paper I'm in the middle of writing. It's a method for training slop out of models. They aren't trained specifically for writing, they just have the most common slop suppressed.
One thing about Gemma 3 is that as great as it is, it is too much of a people pleaser, maybe that can be considered slop as well :)
I did some merges with your Delirium v1 and it has yielded some very interesting results.
I have found that models that are not trained specifically for writing can produce very good writing results, for example some coding or math models can be good writers.