Related models: all models

Swift-1.5-Qwen3.8-27B-Uncensored-BF16 · Swift-1.5-Qwen3.8-27B-Uncensored-FP8 · Swift-1.5-Qwen3.8-27B-Uncensored-FP8-NInfer · Swift-1.5-Qwen3.8-Flash-Next-NVFP4-FP8PLE · Swift-1.5-Qwen3.8-Flash-Next-Rank2-Abliteration-Patch · Swift-Qwen3.8-27B-FP8 · Swift-Qwen3.8-27B-Uncensored-BF16 · Swift-Qwen3.8-27B-Uncensored-FP8 · Swift-Qwen3.8-27B-Uncensored-NVFP4-LocalHessian-ActivationHeadroom-NInfer

Swift-1.5 Qwen3.8 Flash-Next NVFP4 — FP8 PLE

Production-oriented derivative of:

ukisai/Swift-1.5-Qwen3.8-Flash-Next-NVFP4

The model weights remain in their original NVFP4 layout while the large PLE embedding table has been converted from BF16 to FP8 E4M3.

FP8 PLE conversion

This checkpoint keeps the original Flash-Next NVFP4 model weights and converts the large PLE embedding table from BF16 to FP8 E4M3.

Component	Representation	Size
Original PLE	BF16	95.37 GiB
This release	FP8 E4M3 + shared BF16 scale	47.68 GiB

This cuts the PLE storage footprint by approximately 50% while retaining the original NVFP4 weight layout.

The repository contains the portable checkpoint. The Pennyroyal NVMe PLE overlay is deliberately not published because it is a regenerable, machine-local runtime artifact.

Quick start — RTX PRO 6000 96 GB

The production-qualified path uses Pennyroyal / SGLang v2.5.3 on a single NVIDIA RTX PRO 6000 Blackwell 96 GB GPU.

Reference image:

ghcr.io/jpezzulli/sglang-rtxpro6000:v2.5.3

1. Download the checkpoint

hf download \
  d0xin/Swift-1.5-Qwen3.8-Flash-Next-NVFP4-FP8PLE \
  --local-dir /srv/models/Swift-1.5-Qwen3.8-Flash-Next-NVFP4-FP8PLE

2. Get Pennyroyal v2.5.3 setup files

git clone \
  --depth 1 \
  --branch pennyroyal-v2.5.3-setup1 \
  https://github.com/jpezzulli/sglang-rtxpro6000.git \
  pennyroyal

cd pennyroyal/docker/pennyroyal
cp .env.example .env

3. Configure the validated profile

A practical starting .env matching the validated production profile:

PENNYROYAL_PROFILE=next-plain
PENNYROYAL_IMAGE=ghcr.io/jpezzulli/sglang-rtxpro6000:v2.5.3
PENNYROYAL_PORT=8001

HOST_MODELS_ROOT=/srv/models
HOST_CACHE_BASE=/var/cache/pennyroyal
HOST_NIXL_STORAGE_BASE=/srv/pennyroyal-nixl

USER_ID=1000
GROUP_ID=1000
NVIDIA_GPU=0

TARGET_MODEL=/models/Swift-1.5-Qwen3.8-Flash-Next-NVFP4-FP8PLE

PENNY_PLE_BACKEND=nvme
PENNY_PLE_NVME_MODEL=/models/Swift-1.5-Qwen3.8-Flash-Next-NVFP4-FP8PLE-nvme

SGLANG_SM120_ONLINE_MXFP8=true
SGLANG_MM_PREPROCESS_DEVICE=cpu

MAX_RUNNING_REQUESTS=4
MAX_MAMBA_CACHE_SIZE=24

Create the writable cache directories with the same UID/GID used by the container:

sudo install -d \
  -o 1000 \
  -g 1000 \
  /var/cache/pennyroyal \
  /srv/pennyroyal-nixl

4. Prepare the NVMe PLE overlay once

Pennyroyal's normal /models mount is read-only, so the one-time preparation uses a writable override:

docker compose run --rm --no-deps \
  -v /srv/models:/models \
  pennyroyal exec .venv/bin/python \
  scripts/pennyroyal/prepare_ple_nvme.py \
  --source /models/Swift-1.5-Qwen3.8-Flash-Next-NVFP4-FP8PLE \
  --output /models/Swift-1.5-Qwen3.8-Flash-Next-NVFP4-FP8PLE-nvme

Keep both the source checkpoint and prepared overlay immutable while serving.

The prepared FP8 PLE table occupies approximately 47.68 GiB on local SSD/NVMe. NVMe mode replaces its fixed pinned-RAM residency with bounded streaming buffers and reclaimable filesystem page cache.

5. Start

docker compose config --quiet
docker compose pull
docker compose up -d
docker compose logs -f pennyroyal

Startup can take several minutes while the checkpoint is validated and loaded, kernels are prepared, CUDA graphs are captured, and the prepared PLE table checksum is verified.

Check readiness:

curl -fsS http://127.0.0.1:8001/health

OpenAI-compatible API:

http://127.0.0.1:8001/v1

Use pennyroyal as the served model name.

Smoke test:

curl -s \
  http://127.0.0.1:8001/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "pennyroyal",
    "messages": [
      {"role": "user", "content": "Reply with exactly: READY"}
    ],
    "temperature": 0,
    "max_tokens": 64
  }'

Validated production configuration

Production qualification was performed on a single NVIDIA RTX PRO 6000 Blackwell 96 GB GPU with:

Qwen3.8 Flash-Next
Swift 1.5 derivative
NVFP4 target weights
FP8 E4M3 PLE
FP8 KV cache
native NEXTN / MTP speculative decoding
next-plain / FR-Spec disabled
262,144-token configured context
NVMe-backed PLE
online MXFP8 enabled
TP=1
maximum running requests: 4
Pennyroyal / SGLang v2.5.3 in the current production deployment

Current Pennyroyal releases also support other profiles, including FR-Spec and larger context/pool configurations. The measurements below refer to this specific validated profile and should not be compared as if they came from the same benchmark regime.

Validation results

Quality

Test	Result
MMLU-Pro	226 / 280 = 80.71%

This was the model-quality check used during qualification of the FP8-PLE derivative.

Long context

Test	Result
Configured native context	262,144 tokens — PASS
Retrieval near context limit	~261.7K input tokens — PASS

The near-limit retrieval check verifies that the model can prefill and recover the expected information close to the configured 262K context boundary rather than merely accepting a large max_model_len value.

Agentic / tool behavior

Test	Result
Agentic tool/workflow smoke	7 / 7 PASS
Direct tool-call regression on v2.5.3	PASS
Hybrid Auto end-to-end routing	PASS

The v2.5.3 production regression test returned the expected write_file tool call with correctly formed arguments and finish_reason=tool_calls.

The same production target also passed an end-to-end request through the hybrid_auto routing layer.

Throughput

Measurement	Result
Historical agentic workload	174.68 effective tok/s
Fixed 4096-token generation	133.07 tok/s median
Current v2.5.3 production smoke	~161.52 E2E tok/s

The current production smoke generated 512 tokens in 3.169812 seconds, corresponding to approximately 161.52 end-to-end tokens/s.

These numbers come from different workloads and harnesses. They are reported separately and should not be interpreted as directly comparable benchmark samples.

PLE/runtime validation

The FP8 PLE derivative was validated before production deployment, and the deployed NVMe path additionally exercises Pennyroyal's source/config/index/header identity checks and prepared-table checksum verification at startup.

The current production container reports healthr and serves the model through the normal OpenAI-compatible endpoint.

What the tests establish

The validation suite was designed to cover different failure modes rather than a single synthetic throughput score:

MMLU-Pro — quality retention after the FP8-PLE conversion.
262K / ~261.7K retrieval — actual near-limit long-context operation.
7/7 agentic smoke — tool/workflow behavior.
direct tool regression — structured function-call correctness on the deployed v2.5.3 runtime.
Hybrid Auto check — successful integration through the production routing layer.
throughput runs — practical single-GPU serving performance.
startup integrity checks — consistency between the portable checkpoint and prepared NVMe PLE derivative.

Runtime notes

Pennyroyal v2.5.3 is the recommended reference runtime for this checkpoint.

The v2.5.3 container includes the matching NVMe PLE reader. A separate reader installation is therefore not required for the Docker deployment, but the NVMe overlay must still be prepared once from this checkpoint.

Pennyroyal documentation:

https://github.com/jpezzulli/sglang-rtxpro6000

NVMe PLE documentation:

https://github.com/jpezzulli/sglang-rtxpro6000/blob/pennyroyal-main-sm120-final/NVME-PLE.md

Rank-2 abliteration patch

A reproducible Rank-2 refusal-direction ablation / uncensoring patch for this exact checkpoint is published separately:

https://huggingface.co/d0xin/Swift-1.5-Qwen3.8-Flash-Next-Rank2-Abliteration-Patch

The patch repository contains the projection basis, transformation plan, validated offline baker and reference shard hashes instead of duplicating this full checkpoint.