Deeps03/qwen2-1.5b-log-classifier

Model Card

This model is a fine-tuned version of Qwen/Qwen2-1.5B designed for log classification. It takes system or application log entries as input and categorizes them into one of five labels: Normal, Suspicious, Malicious, Informational, or Error. This helps in automating the process of monitoring and analyzing log data for security and operational insights.

Model Details

Model Description

This is a text classification model specifically adapted for the domain of log analysis. By fine-tuning the powerful Qwen2-1.5B base model, it has learned the specific patterns, keywords, and structures present in log data to accurately classify entries.

• Developed by: Deeps03
• Model type: Fine-tuned Causal Language Model (CLM) for Text Classification
• Language(s) (NLP): English
• License: apache-2.0
• Finetuned from model: Qwen/Qwen2-1.5B

Uses

Direct Use

The model is intended to be used directly for inference on log entries using the text-classification pipeline from the Hugging Face transformers library. It can classify individual log strings or a batch of them.

Downstream Use

This model can be a core component in larger systems, such as:

• Security Information and Event Management (SIEM) tools: To automatically triage incoming logs and flag potential threats.
• Automated Monitoring & Alerting Systems: To create alerts for specific log categories like Error or Malicious.
• Incident Response Platforms: To provide initial analysis of logs related to a security incident.

Out-of-Scope Use

This model is not designed for:

• General-purpose text classification: It is specialized for log data and will not perform well on other text types like emails, articles, or social media posts.
• Natural language generation: As a classifier, it cannot be used to generate text. The text-generation tag in the metadata is a common default for models of this architecture but this fine-tuned version is intended for classification.
• Use as a sole decision-maker for critical actions: The model's predictions should be used to assist human analysis, especially in high-stakes security scenarios. It should not be used to automatically block IPs or take irreversible actions without human oversight.

Bias, Risks, and Limitations

• Training Data Bias: The model's performance is highly dependent on the characteristics of the training data it was fine-tuned on. It may perform poorly on log formats that are significantly different from its training set.
• False Positives/Negatives: Like any classification model, it can produce false positives (flagging a normal log as malicious) or false negatives (missing a genuine threat). The tolerance for these errors depends on the specific application.
• Adversarial Attacks: It's possible for a malicious actor who understands the model's patterns to craft log entries that evade detection.

Recommendations

Users should perform their own evaluation on a representative sample of their own log data to determine the model's suitability for their use case. For critical systems, consider using this model as part of an ensemble or as a preliminary filter for human analysts.

How to Get Started with the Model

Use the code below to get started with the model.

from transformers import pipeline

# Load the classification pipeline with the model
classifier = pipeline("text-classification", model="Deeps03/qwen2-1.5b-log-classifier")

# Example log entries
logs_to_classify = [
    "INFO: User 'admin' logged in successfully from IP 192.168.1.10.",
    "WARN: Failed login attempt for user 'root' from IP 10.0.0.5.",
    "CRITICAL: Attempted SQL injection detected: SELECT * FROM users WHERE id = '1' OR '1'='1'",
    "DEBUG: Processing request for /api/v1/status.",
    "ERROR: Failed to connect to database: Connection refused."
]

# Get predictions
results = classifier(logs_to_classify)

# Print results
for log, result in zip(logs_to_classify, results):
    print(f"Log: '{log}'\nPrediction: {result}\n")

Training Details

Training Data

The model was fine-tuned on a private dataset of labeled log entries covering various systems and applications. The dataset includes examples for each of the five classification labels.

Training Procedure

The model was trained using a standard fine-tuning procedure for sequence classification. This involved adding a classification head on top of the pre-trained Qwen2-1.5B base model and training it to minimize the cross-entropy loss on the labeled log dataset.

Preprocessing

The log entries were tokenized using the Qwen2 tokenizer. No other significant preprocessing steps were applied.

Training Hyperparameters

• Training regime: Information not available. Likely trained with mixed precision (bf16 or fp16) to optimize memory and speed.

Evaluation

Testing Data, Factors & Metrics

Testing Data

The model was evaluated on a held-out test set from the same private dataset used for training.

Factors

The model's performance was likely evaluated across the different log categories to ensure it performs well on both common and rare log types.

Metrics

The primary metric used for evaluation was likely F1-score, which provides a balanced measure of precision and recall, making it suitable for potentially imbalanced log data. Accuracy was also likely monitored.

Results

The specific evaluation results on the private test set are not publicly available.

Technical Specifications

Model Architecture and Objective

The model uses the standard Transformer architecture from the Qwen2-1.5B model. A sequence classification head, consisting of a linear layer, was added on top of the base model's pooled output to produce logits for the five target labels. The objective function was Cross-Entropy Loss.

Compute Infrastructure

Software

• Transformers
• PyTorch
• Accelerate

Citation

BibTeX:

If you use this model, please cite the original Qwen2 paper:

@misc{qwen2,
  title={Qwen2 Technical Report},
  author={Qwen Team},
  year={2024}
}