Harsha001/Qwen3.8-27B-Uncensored-ROCmFPX

Qwen3.8-27B-Uncensored — ROCmFPX GGUF

Update planned: this repo will be updated to Qwen 4 27B Uncensored as soon as it is available.

orcarouter/Qwen3.8-27B-Uncensored, an abliterated (refusal-removed) build of Qwen's dense 27B reasoning model Qwen3.8-27B, quantized to the ROCmFPX formats, a llama.cpp fork with low-bit weight types for AMD GPUs (Vulkan and HIP/ROCm). It is the same base model as orcarouter's OrcaSAQ2 GGUF.

Start with Qwen3.8-27B-Uncensored-Q4_0_ROCMFP4_STRIX_O8.gguf (15.6 GB) on a 32 GB GPU or a Strix Halo, or the 6-bit file if you have the memory. On this model a stock llama.cpp Q4_K_M is closer to BF16 than the ROCmFPX 4-bit files (see below); the ROCmFPX files are for AMD speed.

These files need HarshaVardhan3002/ROCmFPX. Stock llama.cpp, LM Studio, Ollama and other ROCmFPX builds may not load them. These files have not been tested on AMD hardware yet.

Not an official release of Qwen or OrcaRouter. All credit for the model goes to its authors.

Safety alignment was removed from this model by its authors. It will answer requests that the original model refuses. You are responsible for how you use it and for what it generates; please read the disclaimer in the original model card.

Files and measured quality

Reference: the BF16 model (perplexity 5.9336). Measured on WikiText-2 test, 40 chunks of 512 tokens. KLD is the mean KL divergence of the next-token distribution from BF16 (lower is better). Same top is how often the quant picks the same next token as BF16.

File	Size	PPL	KLD	Same top
Qwen3.8-27B-Uncensored-Q8_0_ROCMFPX.gguf	28.2 GB	5.9587	0.0033	97.49%	8-bit
Qwen3.8-27B-Uncensored-Q6_K_ROCMFPX.gguf	22.5 GB	5.9435	0.0043	96.90%	6-bit, almost as close as 8-bit
Qwen3.8-27B-Uncensored-Q4_0_ROCMFP4_STRIX_O8.gguf	15.6 GB	5.9958	0.0334	91.80%	Recommended 4-bit: Strix Halo recipe with the output layer at 8-bit
Qwen3.8-27B-Uncensored-Q4_0_ROCMFP4_FAST.gguf	14.6 GB	6.0810	0.0440	89.43%	smallest, speed-first 4-bit (one scale per block)

The 6-bit file is named Qwen3.8-27B-Uncensored-Q6_K_ROCMFPX.gguf so that Hugging Face lists it as a 6-bit quant. Its weights are the ROCmFPX Q6_0_ROCMFPX type, not a stock llama.cpp Q6_K, and it needs the same fork as the other files.

For comparison, a stock llama.cpp Q4_K_M of the same model (16.8 GB), scored the same way, measured PPL 5.9486, KLD 0.0138, same top 94.89%. That is clearly closer to BF16 than both ROCmFPX 4-bit files. If you do not need the ROCmFPX types, use a stock Q4_K_M.

Keeping the output layer at 8-bit (the _O8 file) lowers KLD from 0.0432 to 0.0334 for 0.6 GB. Also measured, not published: plain Q4_0_ROCMFP4_STRIX (15.0 GB, KLD 0.0432), the FAST layout with an 8-bit output layer (15.6 GB, 0.0341, the same as the _O8 file), STRIX with attention and output at 8-bit (19.2 GB, 0.0243, larger and further from BF16 than stock Q4_K_M), and Q3_0_ROCMFPX (15.2 GB, 0.0645, larger and further from BF16 than the 4-bit files).

Vision: mmproj-Qwen3.8-27B-Uncensored-F16.gguf is the vision projector (F16), for image input with --mmproj (not tested with these files). The MTP speculative-decoding head of the original is not included.

How these were made

1. BF16 GGUF and the F16 vision projector converted with the fork's convert_hf_to_gguf.py.
2. Importance matrix from the BF16 file over a general calibration text (calibration_datav3), 512-token chunks.
3. llama-quantize --imatrix from the BF16 file, one run per type, from HarshaVardhan3002/ROCmFPX; the _O8 file adds --output-tensor-type q8_0_rocmfpx. No tensor scales were needed (the model's weights are large enough for the ROCmFPX block scales).
4. Quality: the BF16 reference ran on an NVIDIA GPU with stock kernels. Each quant was decoded by the fork's CPU code to F16 and scored on the same GPU, so the numbers measure the weights only. AMD GPU kernels compute the same math in a different order, so results on an AMD GPU can differ slightly.

Run

git clone https://github.com/HarshaVardhan3002/ROCmFPX && cd ROCmFPX
cmake -B build -DGGML_VULKAN=ON          # or -DGGML_HIP=ON -DAMDGPU_TARGETS=gfx1151 (gfx1100 for RX 7900)
cmake --build build --target llama-server -j
build/bin/llama-server -m Qwen3.8-27B-Uncensored-Q4_0_ROCMFP4_STRIX_O8.gguf -ngl 999 -fa on --jinja -c 32768 --temp 1.0 --top-p 0.95 --top-k 20

Sampling settings are the ones in the original generation_config.json. The model thinks before it answers (<think> block) by default.

License

Apache-2.0, same as the base model (see LICENSE).