---
base_model: TucanoBR/Tucano-630m
base_model_relation: quantized
library_name: mlx
license: apache-2.0
language:
- pt
datasets:
- TucanoBR/GigaVerbo
pipeline_tag: text-generation
tags:
- arxiv:2411.07854
- mlx-lm
- cpu
- local-inference
- pt-br
- small-language-model
- causal-lm
- float16
- mlx
- quantized
- 4-bit
- apple-silicon
- portuguese
- brazilian-portuguese
- llama
thumbnail: https://huggingface.co/guicybercode/Tucano-630m-mlx-4bit/resolve/main/cover.png
---

![Capa](cover.png)

# Tucano-630m MLX 4-bit

This is TucanoBR/Tucano-630m, a Portuguese language model, converted to 4-bit MLX so it can run with mlx-lm (size in the Files table below). I only tested it on the MLX CPU backend on Linux, not on Apple Silicon.

```
pip install mlx-lm
mlx_lm.generate --model guicybercode/Tucano-630m-mlx-4bit --prompt "A capital do Brasil é" --max-tokens 32 --temp 0
```

All credit for the model goes to the original authors, TucanoBR (Nicholas Kluge Corrêa, Aniket Sen, Sophia Falk, Shiza Fatimah; paper: https://arxiv.org/abs/2411.07854). I only converted the format. License: Apache-2.0, same as the original.

Versão quantizada do modelo TucanoBR/Tucano-630m para uso local; todo o crédito é dos autores originais.

## Files

| File | Quantization | Size |
|---|---|---|
| model.safetensors | 4-bit (group size 64), non-quantized tensors in float16 | 355 MB (354639240 bytes) |

## How this was made

mlx-lm 0.32.2 with mlx 0.32.4 (CPU backend, Linux x86_64):

```
mlx_lm.convert --hf-path TucanoBR/Tucano-630m --mlx-path Tucano-630m-mlx-4bit -q --q-bits 4 --dtype float16
```

`--dtype float16` was used because, on the MLX CPU backend on Linux, a bfloat16 conversion produced only `<unk>` tokens in testing, while float16 produced normal text. Not tested on Apple Silicon.

## Usage

```
pip install mlx-lm
mlx_lm.generate --model guicybercode/Tucano-630m-mlx-4bit --prompt "A capital do Brasil é" --max-tokens 32 --temp 0
```

## Smoke test

Prompt: `A capital do Brasil é`, 32 tokens, temperature 0, run on the MLX CPU backend on Linux. Only a load-and-generate check, not an evaluation.

```
também a cidade mais populosa da América Latina. A cidade do Rio de Janeiro é um importante centro financeiro, cultural e sede de diversos órgãos governamentais. A cidade
```

No benchmarks were run on this quantized model. See the original model card for the authors' evaluations and intended use.