10mabdulmoid/deepseek-medical-lora

Model Card for DeepSeek-R1-Distill-Qwen-1.5B Fine-tuned on PubMedQA

🧠 deepseek-medical-lora

A LoRA-finetuned variant of DeepSeek-Medical for medical QA and summarization tasks.

Model Details

Model Description

This model is a LoRA fine-tuned version of deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B on the PubMedQA - pqa_labeled dataset. It was trained for 1 epoch on 1,000 biomedical QA pairs to generate Yes/No/Maybe style answers from PubMed article abstracts.

Developed by: Abdul Moid
Shared by: Abdul Moid
Model type: Causal Language Model with LoRA adapters
Language(s): English (biomedical domain)
License: [More Information Needed]
Base model: deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B

Model Sources [optional]

• Repository: [More Information Needed]
• Paper [optional]: [More Information Needed]
• Demo [optional]: [More Information Needed]

Uses

• Biomedical question answering
• Literature-based evidence QA

Direct Use

• Clinical research assistance
• PubMed-style abstract summarization for QA [More Information Needed]

Intended Tasks

• Clinical research assistance
• PubMed-style abstract summarization for QA

Downstream Use [optional]

• Clinical decision-support systems (non-diagnostic)
• Biomedical search/chatbots

[More Information Needed]

Out-of-Scope Use

• Not for real-time diagnosis or patient care
• Not tested on full-text clinical records [More Information Needed]

Bias, Risks, and Limitations

• Reflects bias from PubMed abstracts and annotations
• Limited to biomedical QA domain
• Use with medical oversight

Recommendations

• Validate outputs with domain experts
• Avoid usage for diagnostic or treatment decisions

How to Get Started with the Model

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

tokenizer = AutoTokenizer.from_pretrained("10mabdulmoid/deepseek-medical-lora")
model = AutoModelForCausalLM.from_pretrained("10mabdulmoid/deepseek-medical-lora")
model.to("cuda" if torch.cuda.is_available() else "cpu")

question = "Is aspirin effective for migraine prevention?"
context = "...abstract from PubMed article..."
prompt = f"Question: {question}\nContext: {context}\nAnswer:"

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=50)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Procedure

• Fine-tuned for 1 epoch using LoRA adapters.

Preprocessing [optional]

• Concatenated question and context for input.
• Used long_answer and final_decision fields.

Training Hyperparameters

• Batch Size: 8
• Epochs: 1
• Optimizer: AdamW
• Learning Rate: 2e-5

Speeds, Sizes, Times [optional]

[More Information Needed]

Evaluation

Testing Data, Factors & Metrics

Testing Data

• Evaluation Set: 20% of PubMedQA labeled split
• Metric: Accuracy on "yes", "no", "maybe" classification
• Results: [More Information Needed] [More Information Needed]

Factors

• Biomedical domain only [More Information Needed]

Metrics

• Accuracy
• F1 Score (optional) [More Information Needed]

Results

[More Information Needed]

Summary

Biomedical QA model using DeepSeek + LoRA + PubMedQA. Quick to adapt and fine-tune further.

Model Examination [optional]

[More Information Needed]

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• Hardware Type: NVIDIA A100 40GB
• Hours used: ~1 hour
• Cloud Provider: Google Cloud
• Compute Region: us-central1
• Carbon Emitted: Use calculator

Technical Specifications [optional]

Model Architecture and Objective

• Transformer + LoRA adapters

Compute Infrastructure

[More Information Needed]

Hardware

[More Information Needed]

Software

[More Information Needed]

Citation [optional]

BibTeX:

@misc{deepseekmedical-lora,
  title={DeepSeek Medical - LoRA Fine-Tuned Model},
  author={Abdul Moid},
  howpublished={\url{https://huggingface.co/10mabdulmoid/deepseek-medical-lora}},
  year={2025}
}

[More Information Needed]

APA:

[More Information Needed]

📞 Contact

For issues, contact huggingface.co/10mabdulmoid

Glossary [optional]

[More Information Needed]

More Information [optional]

[More Information Needed]

Model Card Authors [optional]

Abdul Moid

Model Card Contact

[More Information Needed]