---
library_name: transformers
tags:
- biology
- Proteins
license: mit
datasets:
- MichelNivard/UniRef90-GPCR-Proteins
---

# Model Card for Model ID

This is a protein language model trained on the "G-protein coupled receptor" cluster of UniRef90. This is a very crude selection of GPCR proteins, selected to train a small model purely for training perpuses. 

By focusing on 80k GCPR sequences (which are relatively similar) we are able to train a small model on a MacBook Air, yet still run some follow up experiments within this particular protein domain.


## Model Details

### Model Description

<!-- Provide a longer summary of what this model is. -->

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

- **Developed by:** Michel Nivard
- **Model type:** 30m parameter protein language model with a ModernBERT architecture
- **Language(s) (NLP):** Protein sequences


## Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

### Direct Use

<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->

[More Information Needed]

### Downstream Use [optional]

<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->

[More Information Needed]

### Out-of-Scope Use

<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->

[More Information Needed]

## Bias, Risks, and Limitations

<!-- This section is meant to convey both technical and sociotechnical limitations. -->

[More Information Needed]

### Recommendations

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

## How to Get Started with the Model

Use the code below to get started with the model.

[More Information Needed]

## Training Details

### Training Data

<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

80k GPCR protein sequences

### Training Procedure


MLM with 15% of amino-acids masked