Shirley04/hw1-hc3-detector

HC3 Human vs. ChatGPT Text Classifier

This model is a fine-tuned version of sentence-transformers/all-MiniLM-L6-v2 for binary text classification on the HC3 dataset.

Labels

• 0: human-written answer
• 1: ChatGPT-generated answer

Training setup

• Base model: sentence-transformers/all-MiniLM-L6-v2
• Training epochs: 5
• Batch size: 32
• Learning rate: 2e-5
• Maximum sequence length: 256
• Optimizer: AdamW
• Random seed: 42

The dataset was split by question ID so that answers associated with the same question did not appear in multiple splits.

Results

Approach	Test accuracy
Frozen embeddings + logistic regression	0.8449
Fine-tuned MiniLM classifier	0.9904

The fine-tuned model made 45 errors on 4,668 test examples. It misclassified 45 human-written answers as ChatGPT-generated and misclassified 0 ChatGPT-generated answers as human-written.

Limitations

The model was trained and evaluated on HC3, so its performance may depend on dataset-specific writing patterns. The result should not be interpreted as evidence that the model can reliably detect text from newer language models or human writing from different sources.