gabrielbianchin/esm2_t12_long_int4

ESM2 Quantized

ESM2 Quantized is an adapted version of the ESM2 architectures. It uses local attention instead of global attention, allowing for models with longer input sizes. ESM2 Quantized models have a context size of 2,050, double that of the standard ESM2 model. This kind of model was trained with int4 quantization. Several ESM2 Quantized models are available:

Model	Num layers
gabrielbianchin/esm2_t36_long_int4	36
gabrielbianchin/esm2_t33_long_int4	33
gabrielbianchin/esm2_t30_long_int4	30
gabrielbianchin/esm2_t12_long_int4	12
gabrielbianchin/esm2_t6_long_int4	6

For detailed information, please refer to the paper.