cannj/2026-24679-animal-description-distilbert

Animal Description DistilBERT Classifier

This model classifies short English animal descriptions as a mammal, bird, or reptile.

The model was created for Homework 2, Problem 3. It fine-tunes distilbert-base-uncased using AutoGluon MultiModal.

Model Details

• Base model: distilbert-base-uncased
• Training framework: AutoGluon MultiModal 1.6.1
• Task: Multiclass text classification
• Language: English
• Input column: description
• Target column: label
• Fine-tuning mode: norm_fit
• Maximum text length: 128 subword tokens
• Random seed: 24679

Dataset

The model was trained on:

pcwoods/2026-animaldescription-text-dataset

The published dataset splits were used without repartitioning:

Split	Rows	Purpose
Train	1070	Parameter updates
Validation	15	Checkpoint selection
Test	15	Final evaluation

The training split contains 70 original descriptions and 1000 augmented descriptions.

The validation and test splits contain only original descriptions. Related examples were kept within the same split using the dataset's parent_id and animal_row provenance fields.

Labels

Label	Animal type
0	Mammal
1	Bird
2	Reptile

Training Procedure

The model was trained for a maximum of five epochs. The checkpoint with the best validation accuracy was retained.

Hyperparameter	Value
Learning rate	2e-5
Maximum epochs	5
Weight decay	0.01
Batch size	8
Maximum text length	128
Fine-tuning mode	norm_fit
Trainable parameters	63,747
Total parameters	66,365,187
Trainable percentage	0.096%

Validation accuracy was 0.6000 and validation weighted F1 was 0.4500.

Test Results

The following metrics were computed on the untouched 15-example test split.

Metric	Value
Accuracy	0.4667
Macro precision	0.1556
Macro recall	0.3333
Macro F1	0.2121
Weighted F1	0.2970

Confusion Matrix

Error Analysis

The model made 14 misclassification(s) across the 30 validation and test examples. The observed error directions were: Reptile → Mammal: 10 error(s); Bird → Mammal: 4 error(s). These cases suggest that short descriptions may sometimes contain characteristics shared by multiple animal classes or may omit a uniquely identifying trait. The test set contains only 15 examples, so each test error changes test accuracy by approximately 6.67 percentage points.

The complete holdout prediction table and error table are included in the artifacts directory.

Because the final test split contains only 15 examples, each incorrect prediction changes test accuracy by approximately 6.67 percentage points.

These results should not be interpreted as a comprehensive estimate of performance on all possible animal descriptions.

Intended Use

This model is intended for educational demonstration of multiclass text classification and parameter-efficient DistilBERT fine-tuning.

The supported output categories are:

• Mammal
• Bird
• Reptile

Limitations

• The model supports only three animal categories.
• The training dataset is small and contains synthetic augmentations.
• The validation and test splits contain only 15 examples each.
• Descriptions outside the dataset's writing style may produce unreliable predictions.
• The model should not be used for scientific taxonomy, wildlife management, or safety-critical decisions.
• The saved artifact is an AutoGluon MultiModal predictor rather than a standard Transformers pipeline.

Loading the Model

Download the repository and load the predictor from its automm subdirectory.

from pathlib import Path
import pandas as pd
from huggingface_hub import snapshot_download
from autogluon.multimodal import MultiModalPredictor

snapshot = Path(
    snapshot_download(
        repo_id="cannj/2026-24679-animal-description-distilbert",
        allow_patterns=["automm/*"],
    )
)

model = MultiModalPredictor.load(
    str(snapshot / "automm")
)

model.set_num_gpus(0)

example = pd.DataFrame(
    {
        "description": [
            "This animal has fur and feeds milk to its young."
        ]
    }
)

prediction = model.predict(example)
print(prediction)

The returned numeric labels follow the label table above.

Repository Contents

• automm/requirements.txt
• automm/model_notes.json
• Test metrics
• Test classification report
• Test confusion matrix
• Validation and test predictions
• Holdout error analysis

AI Usage Disclosure

Generative AI tools were used to assist with drafting, organizing, and debugging portions of the notebook code and documentation.

The reported metrics were produced by executing the notebook and were not generated or estimated by the AI tool.