---
license: llama2
base_model:
- lmsys/vicuna-13b-v1.5
---

This is a half precision (float16) gguf file for vicuna-13b-v1.5.

Conversion code:


import os
import subprocess
from huggingface_hub import snapshot_download


model_id="lmsys/vicuna-13b-v1.5"
snapshot_download(repo_id=model_id, local_dir="vicuna-hf",
                  local_dir_use_symlinks=False, revision="main")

subprocess.run(["git", "clone", "https://github.com/ggerganov/llama.cpp.git"])

subprocess.run(["pip", "install", "-r", "llama.cpp/requirements.txt"])

subprocess.run(["python", "llama.cpp/convert_hf_to_gguf.py", "vicuna-hf", \
                "--outfile", "vicuna-13b-v1.5.gguf", "--outtype", "f16"])

(modified from https://www.substratus.ai/blog/converting-hf-model-gguf-model)