Instructions to use guicybercode/tucano-160m-oasst2-ptbr-lora-cpu-piloto with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use guicybercode/tucano-160m-oasst2-ptbr-lora-cpu-piloto with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("TucanoBR/Tucano-160m")
model = PeftModel.from_pretrained(base_model, "guicybercode/tucano-160m-oasst2-ptbr-lora-cpu-piloto")

Transformers

How to use guicybercode/tucano-160m-oasst2-ptbr-lora-cpu-piloto with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="guicybercode/tucano-160m-oasst2-ptbr-lora-cpu-piloto")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("guicybercode/tucano-160m-oasst2-ptbr-lora-cpu-piloto", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use guicybercode/tucano-160m-oasst2-ptbr-lora-cpu-piloto with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "guicybercode/tucano-160m-oasst2-ptbr-lora-cpu-piloto"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "guicybercode/tucano-160m-oasst2-ptbr-lora-cpu-piloto",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/guicybercode/tucano-160m-oasst2-ptbr-lora-cpu-piloto

SGLang

How to use guicybercode/tucano-160m-oasst2-ptbr-lora-cpu-piloto with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "guicybercode/tucano-160m-oasst2-ptbr-lora-cpu-piloto" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "guicybercode/tucano-160m-oasst2-ptbr-lora-cpu-piloto",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "guicybercode/tucano-160m-oasst2-ptbr-lora-cpu-piloto" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "guicybercode/tucano-160m-oasst2-ptbr-lora-cpu-piloto",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use guicybercode/tucano-160m-oasst2-ptbr-lora-cpu-piloto with Docker Model Runner:
```
docker model run hf.co/guicybercode/tucano-160m-oasst2-ptbr-lora-cpu-piloto
```

tucano-160m-oasst2-ptbr-lora-cpu-piloto / training-log.jsonl

guicybercode

Publish CPU LoRA pilot with held-out float32 evaluation and full reproduction scripts

8ce4eff verified 2 days ago

Raw History Blame Contribute Delete

3.04 kB

{"event": "step", "epoch": 1, "optimizer_step": 5, "running_train_loss": 2.238007207674975, "lr": 0.00029728930458940595, "grad_norm": 0.8063221573829651, "epoch_elapsed_seconds": 22.473043352998502}
{"event": "epoch", "epoch": 1, "optimizer_steps": 6, "train_loss": 2.2584660765231943, "train_seconds": 25.907004280001274, "validation": {"loss": 2.2768132312225764, "perplexity": 9.745573980476506, "response_tokens": 1522, "examples": 12, "seconds": 0.5419957140038605}}
{"event": "step", "epoch": 2, "optimizer_step": 10, "running_train_loss": 2.1012608638544528, "lr": 0.00026790796421141813, "grad_norm": 0.5991487503051758, "epoch_elapsed_seconds": 18.22901766400173}
{"event": "epoch", "epoch": 2, "optimizer_steps": 12, "train_loss": 2.098645087989428, "train_seconds": 25.95503581200319, "validation": {"loss": 2.242185463416592, "perplexity": 9.413882518133018, "response_tokens": 1522, "examples": 12, "seconds": 0.5481375860035769}}
{"event": "step", "epoch": 3, "optimizer_step": 15, "running_train_loss": 1.9789021475735185, "lr": 0.00021231225195028297, "grad_norm": 0.5209558606147766, "epoch_elapsed_seconds": 12.166495024997857}
{"event": "epoch", "epoch": 3, "optimizer_steps": 18, "train_loss": 1.981365356522283, "train_seconds": 24.54493858999922, "validation": {"loss": 2.243609419633462, "perplexity": 9.427297023236907, "response_tokens": 1522, "examples": 12, "seconds": 0.6372171650000382}}
{"event": "step", "epoch": 4, "optimizer_step": 20, "running_train_loss": 1.9427905353859114, "lr": 0.00014286271262643866, "grad_norm": 0.5414717197418213, "epoch_elapsed_seconds": 8.372175031996449}
{"event": "epoch", "epoch": 4, "optimizer_steps": 24, "train_loss": 1.8920829225523852, "train_seconds": 25.12302869799896, "validation": {"loss": 2.24681126335134, "perplexity": 9.457530130000158, "response_tokens": 1522, "examples": 12, "seconds": 0.5965341569972225}}
{"event": "step", "epoch": 5, "optimizer_step": 25, "running_train_loss": 1.8080690098776657, "lr": 7.500000000000002e-05, "grad_norm": 0.4843766689300537, "epoch_elapsed_seconds": 5.079509700997733}
{"event": "step", "epoch": 5, "optimizer_step": 30, "running_train_loss": 1.8361153760542404, "lr": 2.38119700753228e-05, "grad_norm": 0.5224012732505798, "epoch_elapsed_seconds": 26.849627632000193}
{"event": "epoch", "epoch": 5, "optimizer_steps": 30, "train_loss": 1.8361153760542404, "train_seconds": 26.84980382199865, "validation": {"loss": 2.248372650961055, "perplexity": 9.472308534770303, "response_tokens": 1522, "examples": 12, "seconds": 0.5754360380014987}}
{"event": "step", "epoch": 6, "optimizer_step": 35, "running_train_loss": 1.8029368741417207, "lr": 6.792116140373116e-07, "grad_norm": 0.6384185552597046, "epoch_elapsed_seconds": 22.325316743001167}
{"event": "epoch", "epoch": 6, "optimizer_steps": 36, "train_loss": 1.8106174533771444, "train_seconds": 26.368592519997037, "validation": {"loss": 2.2492608048129488, "perplexity": 9.48072513914832, "response_tokens": 1522, "examples": 12, "seconds": 0.6144218890040065}}

	{"event": "step", "epoch": 1, "optimizer_step": 5, "running_train_loss": 2.238007207674975, "lr": 0.00029728930458940595, "grad_norm": 0.8063221573829651, "epoch_elapsed_seconds": 22.473043352998502}
	{"event": "epoch", "epoch": 1, "optimizer_steps": 6, "train_loss": 2.2584660765231943, "train_seconds": 25.907004280001274, "validation": {"loss": 2.2768132312225764, "perplexity": 9.745573980476506, "response_tokens": 1522, "examples": 12, "seconds": 0.5419957140038605}}
	{"event": "step", "epoch": 2, "optimizer_step": 10, "running_train_loss": 2.1012608638544528, "lr": 0.00026790796421141813, "grad_norm": 0.5991487503051758, "epoch_elapsed_seconds": 18.22901766400173}
	{"event": "epoch", "epoch": 2, "optimizer_steps": 12, "train_loss": 2.098645087989428, "train_seconds": 25.95503581200319, "validation": {"loss": 2.242185463416592, "perplexity": 9.413882518133018, "response_tokens": 1522, "examples": 12, "seconds": 0.5481375860035769}}
	{"event": "step", "epoch": 3, "optimizer_step": 15, "running_train_loss": 1.9789021475735185, "lr": 0.00021231225195028297, "grad_norm": 0.5209558606147766, "epoch_elapsed_seconds": 12.166495024997857}
	{"event": "epoch", "epoch": 3, "optimizer_steps": 18, "train_loss": 1.981365356522283, "train_seconds": 24.54493858999922, "validation": {"loss": 2.243609419633462, "perplexity": 9.427297023236907, "response_tokens": 1522, "examples": 12, "seconds": 0.6372171650000382}}
	{"event": "step", "epoch": 4, "optimizer_step": 20, "running_train_loss": 1.9427905353859114, "lr": 0.00014286271262643866, "grad_norm": 0.5414717197418213, "epoch_elapsed_seconds": 8.372175031996449}
	{"event": "epoch", "epoch": 4, "optimizer_steps": 24, "train_loss": 1.8920829225523852, "train_seconds": 25.12302869799896, "validation": {"loss": 2.24681126335134, "perplexity": 9.457530130000158, "response_tokens": 1522, "examples": 12, "seconds": 0.5965341569972225}}
	{"event": "step", "epoch": 5, "optimizer_step": 25, "running_train_loss": 1.8080690098776657, "lr": 7.500000000000002e-05, "grad_norm": 0.4843766689300537, "epoch_elapsed_seconds": 5.079509700997733}
	{"event": "step", "epoch": 5, "optimizer_step": 30, "running_train_loss": 1.8361153760542404, "lr": 2.38119700753228e-05, "grad_norm": 0.5224012732505798, "epoch_elapsed_seconds": 26.849627632000193}
	{"event": "epoch", "epoch": 5, "optimizer_steps": 30, "train_loss": 1.8361153760542404, "train_seconds": 26.84980382199865, "validation": {"loss": 2.248372650961055, "perplexity": 9.472308534770303, "response_tokens": 1522, "examples": 12, "seconds": 0.5754360380014987}}
	{"event": "step", "epoch": 6, "optimizer_step": 35, "running_train_loss": 1.8029368741417207, "lr": 6.792116140373116e-07, "grad_norm": 0.6384185552597046, "epoch_elapsed_seconds": 22.325316743001167}
	{"event": "epoch", "epoch": 6, "optimizer_steps": 36, "train_loss": 1.8106174533771444, "train_seconds": 26.368592519997037, "validation": {"loss": 2.2492608048129488, "perplexity": 9.48072513914832, "response_tokens": 1522, "examples": 12, "seconds": 0.6144218890040065}}