Awesome
Hello, awesome work. Now my videos take around 20% time less to generate with this encoder while maintaining really high quality!
But I found one small issue that is worth mentioning. I'm using version of Transformers 5.22.1 and in this version the Transformers already had registered the architecture of Molmo2. But the file modeling_molmo2.py was also trying to re-register it. So the Transformer was throwing error: ValueError:
Molmo2Config is already used by a Transformers model.
The fix was to change the ending of the code :
AutoModelForImageTextToText.register(
Molmo2Config,
Molmo2ForConditionalGeneration
)
to:
AutoModelForImageTextToText.register(
Molmo2Config,
Molmo2ForConditionalGeneration,
exist_ok=True
)
This made so the Transformers allow you to add your own model implementation, even though it already recognises the Molmo2 architecture. And now it works perfectly.
Hello, awesome work. Now my videos take around 20% time less to generate with this encoder while maintaining really high quality!
But I found one small issue that is worth mentioning. I'm using version of Transformers 5.22.1 and in this version the Transformers already had registered the architecture of Molmo2. But the file modeling_molmo2.py was also trying to re-register it. So the Transformer was throwing error: ValueError:
Molmo2Config is already used by a Transformers model.The fix was to change the ending of the code :
AutoModelForImageTextToText.register(
Molmo2Config,
Molmo2ForConditionalGeneration
)to:
AutoModelForImageTextToText.register(
Molmo2Config,
Molmo2ForConditionalGeneration,
exist_ok=True
)
This made so the Transformers allow you to add your own model implementation, even though it already recognises the Molmo2 architecture. And now it works perfectly.
Thanks for the report and for the fix! You're right: Transformers 5.22 ships Molmo2 itself, so the registration at the end of modeling_molmo2.py collided with it.
I pushed exactly your change (exist_ok=True) to the repository's modeling_molmo2.py. I checked that it loads with Transformers 4.57.1, 5.17 and 5.22.1. If you already have the model downloaded, keep your local edit or re-download that one file.
Glad it's saving you time!