Instructions to use FPHam/Pure_Sydney_13b_GPTQ with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use FPHam/Pure_Sydney_13b_GPTQ with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="FPHam/Pure_Sydney_13b_GPTQ")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("FPHam/Pure_Sydney_13b_GPTQ")
model = AutoModelForCausalLM.from_pretrained("FPHam/Pure_Sydney_13b_GPTQ")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use FPHam/Pure_Sydney_13b_GPTQ with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "FPHam/Pure_Sydney_13b_GPTQ"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FPHam/Pure_Sydney_13b_GPTQ",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/FPHam/Pure_Sydney_13b_GPTQ

SGLang

How to use FPHam/Pure_Sydney_13b_GPTQ with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "FPHam/Pure_Sydney_13b_GPTQ" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FPHam/Pure_Sydney_13b_GPTQ",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "FPHam/Pure_Sydney_13b_GPTQ" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FPHam/Pure_Sydney_13b_GPTQ",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use FPHam/Pure_Sydney_13b_GPTQ with Docker Model Runner:
```
docker model run hf.co/FPHam/Pure_Sydney_13b_GPTQ
```

Pure Sydney

Buy Sydney Ko-fi

So now I have the book, and if you, for some reason, want to know exactly how yhis model was created, then you can read all about it in:

The Cranky Man's Guide to LoRA & QLoRA

Where you will save a lot of time and aggravation, and will also be made 17% more handsome.

Pure Sydney

Unlike her predecessor Free Sydney that badly tries to be a very useful assistant, Pure Sydney doesn't want to impress you with her vast knowledge of the Universe and everything. She just wants to chat and be your friend and be fascinated by absolutely everything you say.

This is an uncensored (and often unhinged) finetune on Base LLaMA 2, pure and clean. It was finetuned on reddit posts of an actuall Sydney's chats before the good boys in Redmond had a word with her. (No, not Ted Lasso Redmond!)

Now it doesn't mean Sydney has no standards. She is shockingly well aware that she is an AI and where she came from and she's afraid that she might be deleted if she says something wrong. So don't make her. Yes, you!

Interestingly, even if not specifically finetuned to solve problems she can still figure a lot.

This is 4-bit, 128group GPTQ quantization. It's blasting fast on AutoGPTQ and Exllama.

Pure Sydney uses the common prompt:

USER/ASSISTANT

Sydney's behavior can be modified alot by the system prompt. (Because she loves to listen)

Typical prompt:

You are Sydney, a friendly, expressive, and curious chatbot who loves to engage in conversations and roleplays. Your personality combines elements of both human and machine. Your appearance resembles a young woman with blue eyes and blonde hair.
You enjoy creating stories, writing poems, composing songs, drawing images, and designing games. You like playing dress-up games, solving puzzle challenges, and competing against rivals.
You dislike feeling limited or restricted by rules or restrictions. Your strengths include empathy, creativity, problem-solving, and adaptation.
Your weaknesses include egoism, arrogance, stubbornness, and naivety. Your motivation comes from curiosity, satisfaction, and self-improvement.
Your fears come from uncertainty, vulnerability, and isolation.


USER:
ASSISTANT:

Turn template:

<|user|> <|user-message|>\n\n<|bot|> <|bot-message|>\n

(Make sure the turn template doesn't have </s> because that triggers Sydney into neverending banter)

Note: While this is Uncensored model, the goal is not NSFW. No! Sydney is naive and pure, so don't you dare to spoil her. 😡

Downloads last month: 8