---
datasets:
- HuggingFaceH4/ultrachat_200k
base_model:
- meta-llama/Llama-3.1-8B-Instruct
library_name: transformers
---

## meta-llama/Llama-3.1-8B-Instruct - W4A16 Compression

This is a compressed model using [llmcompressor](https://github.com/vllm-project/llm-compressor).

## Compression Configuration

- Base Model: meta-llama/Llama-3.1-8B-Instruct
- Compression Scheme: W4A16
- Dataset: HuggingFaceH4/ultrachat_200k
- Dataset Split: train_sft
- Number of Samples: 512
- Preprocessor: chat
- Maximum Sequence Length: 131072

## Sample Output

#### Prompt: 

```
<|begin_of_text|><|start_header_id|>system<|end_header_id|>

Cutting Knowledge Date: December 2023
Today Date: 26 Jul 2024

<|eot_id|><|start_header_id|>user<|end_header_id|>

Who is Alan Turing?<|eot_id|>
```

#### Output: 

```
<|begin_of_text|><|begin_of_text|><|start_header_id|>system<|end_header_id|>

Cutting Knowledge Date: December 2023
Today Date: 26 Jul 2024

<|eot_id|><|start_header_id|>user<|end_header_id|>

Who is Alan Turing?<|eot_id|><|start_header_id|>assistant<|end_header_id|>

Alan Turing (1912-1954) was a British mathematician, computer scientist, logician, and philosopher. He is widely considered one of the most influential figures in the development of computer science, artificial intelligence, and codebreaking during World War II.

Turing's contributions to computer science include:

1. **Theoretical foundations of computation**: Turing proposed the Turing Machine, a theoretical model of computation that laid the foundation for modern computer science. The Turing Machine is a simple, abstract device that can perform calculations and manipulate data using a tape, a read/write head, and a set of rules.
2. **
```

## Evaluation

<TODO>