VerMichel/german-reviews-distilbert

German doctor reviews — rating classifier

DistilBERT model fine-tuned to classify German doctor reviews by rating (1–6).

Model Details

• Developed by: Mikhail Shostak
• Model type: DistilBERT fine-tuned for sequence classification
• Language: German
• License: CC BY-SA 4.0 (inherited from the training dataset)
• Finetuned from: distilbert-base-german-cased
• Num labels: 6 (ratings 1–6)

Uses

Direct Use: predicting a rating (1–6) from the text of a doctor review.

Out-of-Scope Use: languages other than German, domains other than doctor reviews, or rating scales other than 1–6.

How to Get Started

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_name = "VerMichel/german-reviews-distilbert"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

text = "Der Arzt war sehr freundlich und kompetent."
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=128)

with torch.no_grad():
    logits = model(**inputs).logits
pred = logits.argmax(dim=-1).item() + 1  # labels 0-5 -> ratings 1-6
print(f"Rating: {pred}")

Training Details

Training Data:

• Source: Patient Reviews of Doctor's (Kaggle)
• License: CC BY-SA 4.0
• Volume: 439,280 reviews
• Missing rating: 9,544 (2%) — dropped
• Class imbalance: 81% — rating 1, remaining 19% — ratings 2–6
• Mean rating: 1.56 (Std = 1.34)
• Split: 80/20, random_state=42, no stratification (verified: class proportion difference < 0.001)

Preprocessing:

• Removed URLs and tags
• Collapsed character repetitions of 3+ (e.g., seeehr → sehr)
• Preserved negations (nicht, kein) — not treated as stopwords

Training Hyperparameters:

• Epochs: 2
• Batch size: 16
• Max sequence length: 128
• Learning rate: 5e-5 (default)
• FP16: enabled
• Hardware: NVIDIA GTX 1650 (4 GB)
• Training time: ~4 hours

Evaluation

Metrics: Accuracy, macro F1, per-class F1.

Overall results:

Metric	Value
Accuracy	0.8608
F1 macro	0.4405

Per-class F1:

Rating	Precision	Recall	F1	Support
1	0.9427	0.9836	0.9627	13014
2	0.3618	0.1491	0.2112	825
3	0.3098	0.2469	0.2748	320
4	0.2792	0.2081	0.2384	471
5	0.3971	0.5252	0.4523	735
6	0.5694	0.4520	0.5040	635

Baseline comparison:

Model	F1 macro
LogisticRegression (TF-IDF)	0.379
LSTM (bidirectional, 14M params)	0.387
DistilBERT (this model)	0.441

DistilBERT improves over classical baselines by +0.05 macro F1. The gain is concentrated on ratings with clear textual signal (1, 5, 6). Ratings 2–4 remain difficult to distinguish due to textual similarity between adjacent ratings.

Bias, Risks, and Limitations

• Class imbalance: the model was trained on data with 81% rating-1 reviews. Rare classes (2–4) have F1 = 0.21–0.27.
• Subjective labels: ratings 2, 3, and 4 are textually close, and even humans may disagree on the correct label.
• Domain: the model was trained on doctor reviews. Performance on other domains is not guaranteed.
• Language: German only.

Citation

Not applicable.

Model Card Contact

Mikhail Shostak — GitHub