ShantanuT01/dactyl-modernbert-base-finetuned

DACTYL Text Detector

Training Configuration

{
    "training_split": "training",
    "evaluation_split": "testing",
    "results_path": "bce-finetuning-modernbert.csv",
    "num_epochs": 1,
    "model_path": "ShantanuT01/dactyl-modernbert-base-pretrained",
    "tokenizer": "answerdotai/ModernBERT-base",
    "optimizer": "SOTAs",
    "optimizer_type": "libauc",
    "optimizer_args": {
        "lr": 1e-05
    },
    "loss_fn": "tpAUC_KL_Loss",
    "reset_classification_head": true,
    "loss_type": "libauc",
    "loss_fn_args": {
        "data_len": 466005
    },
    "needs_loss_fn_as_parameter": false,
    "save_path": "ShantanuT01/dactyl-modernbert-base-finetuned",
    "training_args": {
        "batch_size": 64,
        "needs_sampler": true,
        "needs_index": true,
        "shuffle": false,
        "sampling_rate": 0.5,
        "apply_sigmoid": true
    },
    "best_model_path": "best-tpauc-model"
}

Results

model	AP Score	AUC Score	OPAUC Score	TPAUC Score
DeepSeek-V3	0.999089	0.999938	0.999459	0.994926
ShantanuT01/fine-tuned-Llama-3.2-1B-Instruct-apollo-mini-RedditWritingPrompts-testing	0.895249	0.997241	0.982234	0.83088
ShantanuT01/fine-tuned-Llama-3.2-1B-Instruct-apollo-mini-abstracts-testing	0.88244	0.988173	0.974526	0.782076
ShantanuT01/fine-tuned-Llama-3.2-1B-Instruct-apollo-mini-news-testing	0.587536	0.981587	0.91101	0.258648
ShantanuT01/fine-tuned-Llama-3.2-1B-Instruct-apollo-mini-reviews-testing	0.400519	0.943471	0.817978	0
ShantanuT01/fine-tuned-Llama-3.2-1B-Instruct-apollo-mini-student_essays-testing	0.238706	0.915752	0.774792	0
ShantanuT01/fine-tuned-Llama-3.2-1B-Instruct-apollo-mini-tweets-testing	0.743253	0.97897	0.918286	0.279746
claude-3-5-haiku-20241022	0.983634	0.997636	0.988756	0.890673
claude-3-5-sonnet-20241022	0.997952	0.999812	0.998828	0.989234
gemini-1.5-flash	0.978596	0.996352	0.985728	0.861063
gemini-1.5-pro	0.959431	0.992509	0.972711	0.73439
gpt-4o-2024-11-20	0.981379	0.996717	0.987477	0.878747
gpt-4o-mini	0.999396	0.999971	0.999703	0.99719
llama-3.2-90b	0.965884	0.990335	0.978577	0.79221
llama-3.3-70b	0.989961	0.99789	0.993615	0.938358
mistral-large-latest	0.997618	0.999558	0.998528	0.985946
mistral-small-latest	0.998361	0.999689	0.999022	0.990606
overall	0.994015	0.995186	0.984633	0.85068