Add files using upload-large-folder tool

.env

@@ -0,0 +1,2 @@
+PHYSIONET_PASSWORD=my_password
+PHYSIONET_USER=my_username

.gitignore

@@ -0,0 +1,3 @@
+*.env
+*.jpg
+*.ipynb_checkpoints

README.md

@@ -0,0 +1,141 @@
+# A new generative model for radiology
+
+tl;dr you should include text inputs along with images
+
+[![YourActionName Actions Status](https://github.com/nathansutton/prerad/workflows/CI/badge.svg)](https://github.com/nathansutton/prerad/actions)
+
+Available @ https://nathansutton-prerad.hf.space
+
+Machine learning in radiology has come a long way. For a long time the goal was simply to make a probability estimate of different conditions available to the radiologist at the time of interpretation. As evidence, see any of the hundreds of AI vendors that have commercialized computer vision algorithms. On the academic frontier, recent advances have made it possible to generate realistic sounding radiology reports directly from an image. The first paper I found describing such a model was from 2017, but there have been many more recently with the onset of transformers.
+
+However, every example architecture I have found suffers from the same structural problem.
+
+__They aren't answering a clinical question.__
+
+When a provider orders an imaging study the ordering provider is asking the radiologist to combine their education and experience to answer a clinical question. Unlike most specialist consultations, the format is asynchronous. Occasional telephone or face-to-face consults do occur for time-sensitive findings, but this is not the norm. Instead, the radiology report is the primary product of this provider-radiologist specialist consultation. This document's purpose to answer the clinical question, or indication. It usually has a succinct 'impression' section at the end. For example…
+
+```
+INDICATION: 36yo M with hypoxia // ?pna, aspiration.
+
+FINDINGS: 
+
+PA and lateral views of the chest provided. The lungs are adequately aerated. 
+There is a focal consolidation at the left lung base adjacent to the lateral 
+hemidiaphragm. There is mild vascular engorgement. There is bilateral apical 
+pleural thickening. The cardiomediastinal silhouette is remarkable for 
+arch calcifications. The heart is top normal in size.
+
+IMPRESSION: 
+
+Focal consolidation at the left lung base, possibly representing aspiration 
+or pneumonia. Central vascular engorgement.
+```
+
+__Incorrect Architectures__  
+Medical images alone are not sufficient to answer why an imaging study was ordered by a provider. For example, a provider might order a chest x-ray because their patient is presenting with shortness of breath and they suspect pneumonia. In another case, they might suspect a fracture after a motor vehicle collision and order a chest x-ray to rule out a broken rib. The same image could answer either of these clinical questions.
+In my search I found over ten studies since 2017 describing model architectures that could conditionally generate entire radiology reports from an image (Jing et al. 2017, Li et al. 2018, Xue et al. 2018, Singh et al. 2019, Yuan et al. 2019, Chen et al. 2020, Miura et al. 2020, Fenglin et al. 2021, Nooralahzadeh et al. 2021, Sirshar et al. 2022, Chen et al. 2022, Yang et al. 2022). Unfortunately, none had text inputs. The one exception was a recent paper included the full-text radiology report as a textual input alongside the image, but its goal was to remove hallucinated references by cleaning up the data used for model training (Ramesh et al. 2022). These oversights are a problem because all the variation in the generated reports will come from the images. They cannot answer a clinical question posed in text by the ordering provider.
+
+![](./resources/baseline.png)
+
+__Better Architectures__  
+To realistically describe what a radiologist is doing when they write a report a deep learning model needs to accept the same inputs. This means the conditional generation of radiology reports should include both image and text inputs and have a text output. This year a new transformer architecture particularly suited for this type of multi-modal problem was just released by SalesForce (Li et al. 2022). BLIP has a dual text and vision encoder paired with a text decoder. This allows it to continue generating new text for a radiology report from a given prompt's starting point. Lucky for us, the first paragraph of most radiology reports is the clinical question!
+This makes conditionally generating a radiology report possible in couple of lines of code.
+
+```
+from PIL import Image
+from transformers import BlipForConditionalGeneration, BlipProcessor
+
+# read in the model
+processor = BlipProcessor.from_pretrained("nathansutton/generate-cxr")
+model = BlipForConditionalGeneration.from_pretrained("nathansutton/generate-cxr")
+
+# your data
+my_image = 'my-chest-x-ray.jpg'
+my_indication = 'RLL crackles, eval for pneumonia'
+
+# process the inputs
+inputs = processor(
+    images=Image.open(my_image), 
+    text='indication:' + my_indication,
+    return_tensors="pt"
+)
+
+# generate an entire radiology report
+output = model.generate(**inputs,max_length=512)
+report = processor.decode(output[0], skip_special_tokens=True)
+```
+
+__Simplified Application__  
+Starting from the base BLIP image captioning model, I fine-tuned a causal language model to generate radiology reports from a chest x-ray and a clinical prompt. The data used to fine-tune these assessments were derived from the MIMIC critical care database. Specifically, I cross referenced the original radiology reports in the MIMIC-CXR project with the JPG images available in the MIMIC-CXR-JPG project.
+More information on how to reproduce these labels can be found in the corresponding Github repository.
+
+Does it work? Let's go back to our original radiology report and perturb it with two different clinical indications. On the left we show the original question for this image ('question pneumonia') and on the right a fictitious concern ('question pneumothorax'). The original reference report is in quotes above. You can play around with your own de-identified images in this interactive web application hosted graciously by huggingface spaces.
+
+![](./resources/streamlit.png)  
+
+The same image with two different clinical prompts. This shows that by changing the prompt to 'chest pain, history of pneumothorax' the model successfully changed its answer in response to a different clinical question.
+
+This is a simplified example was meant to demonstrate one concept. Conditionally generated radiology reports should include text inputs alongside the medical images to answer a clinical question. Obviously this kind of automation is not intended to replace radiologists, but it could help them quickly template their reports so they aren't starting from scratch.  
+
+In the bigger scheme of things, I have seen this kind of disparity between the question technologists are answering and providers are asking repeated across healthcare. Bring a provider on board to collaborate, and you'll be rewarded with actually useful models.
+
+## Data
+All data were derived from MIMIC and require signing a data use agreement with Physionet.  None are provided here.
+
+## Services
+
+This repository exposes four components that are useful in a data science proof of concept.
+- A container running Jupyter notebooks with common machine learning libraries (available @ localhost:8888).  Any notebooks will persist in a mounted volume (./volumes/notebooks)
+- A container running Streamlit allows a user to access the predictions from the model based on user inputs (available at localhost:8501)
+
+## Usage
+
+turn on the application 
+```
+docker-compose up 
+```
+
+download the data from physionet, passing any argument downloads the data (no arguments does nothing)
+```
+docker-compose run physionet True 
+```
+
+run the etl migrations
+```
+docker-compose run etl 
+```
+
+train the model
+```
+docker-compose run train
+```
+
+## Structure
+
+```
+|-- containers - code
+|   |-- etl         # transforms raw data from physionet into jsonlines files
+|   |-- jupyter     # interactive notebooks
+|   |-- physionet   # download the MIMIC-CXR and MIMIC-CXR-JPG data from physionet
+|   |-- streamlit   # a small streamlit application to demo the model functionality 
+|-- volumes         # persistent data
+|   |-- notebooks   # jupyter notebooks persisted here
+|   |-- physionet   # physionet data is persisted here
+```
+
+## References
+
+- Chen, Zhihong, et al. "Generating radiology reports via memory-driven transformer." arXiv preprint arXiv:2010.16056 (2020).  
+- Chen, Zhihong, et al. "Cross-modal memory networks for radiology report generation." arXiv preprint arXiv:2204.13258 (2022).  
+- Jing, Baoyu, Pengtao Xie, and Eric Xing. "On the automatic generation of medical imaging reports." arXiv preprint arXiv:1711.08195 (2017).  
+- Li, Junnan, et al. "Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation." International Conference on Machine Learning. PMLR, 2022.  
+- Li, Yuan, et al. "Hybrid retrieval-generation reinforced agent for medical image report generation." Advances in neural information processing systems 31 (2018).  
+- Liu, Fenglin, et al. "Exploring and distilling posterior and prior knowledge for radiology report generation." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021.  
+- Miura, Yasuhide, et al. "Improving factual completeness and consistency of image-to-text radiology report generation." arXiv preprint arXiv:2010.10042 (2020).  
+- Nooralahzadeh, Farhad, et al. "Progressive transformer-based generation of radiology reports." arXiv preprint arXiv:2102.09777 (2021).  
+- Ramesh, Vignav, Nathan A. Chi, and Pranav Rajpurkar. "Improving Radiology Report Generation Systems by Removing Hallucinated References to Non-existent Priors." Machine Learning for Health. PMLR, 2022.   
+- Sirshar, Mehreen, et al. "Attention based automated radiology report generation using CNN and LSTM." Plos one 17.1 (2022): e0262209.  
+- Singh, Sonit, et al. "From chest x-rays to radiology reports: a multimodal machine learning approach." 2019 Digital Image Computing: Techniques and Applications (DICTA). IEEE, 2019.  
+- Yang, Shuxin, et al. "Knowledge matters: Chest radiology report generation with general and specific knowledge." Medical Image Analysis 80 (2022): 102510.  
+- Xue, Yuan, et al. "Multimodal recurrent model with attention for automated radiology report generation." Medical Image Computing and Computer Assisted Intervention–MICCAI 2018: 21st International Conference, Granada, Spain, September 16–20, 2018, Proceedings, Part I. Springer International Publishing, 2018.  
+- Yuan, Jianbo, et al. "Automatic radiology report generation based on multi-view image fusion and medical concept enrichment." Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part VI 22. Springer International Publishing, 2019.  

containers/etl/Dockerfile

@@ -0,0 +1,37 @@
+FROM python:3.9-buster
+
+RUN \
+    apt-get update && \
+    apt-get -y upgrade && \
+    apt-get clean && \
+    rm -rf /var/lib/apt/lists/*
+
+RUN useradd --create-home app
+WORKDIR /home/app
+
+COPY requirements.txt /home/app/
+COPY __init__.py /home/app/
+COPY common.py /home/app/
+COPY run.py /home/app/
+
+RUN \
+    chown app:app /home/app/requirements.txt && \
+    chmod 0755 /home/app/requirements.txt && \
+    chown app:app /home/app/__init__.py && \
+    chmod 0755 /home/app/__init__.py && \
+    chown app:app /home/app/run.py && \
+    chmod 0755 /home/app/run.py && \
+    chown app:app /home/app/common.py && \
+    chmod 0755 /home/app/common.py
+
+USER app
+
+ENV VIRTUAL_ENV=/home/app/venv
+RUN python3 -m venv $VIRTUAL_ENV
+ENV PATH="$VIRTUAL_ENV/bin:$PATH"
+
+RUN \
+    pip install --upgrade pip && \
+    pip install -r requirements.txt
+
+CMD ["python", "run.py", "worker", "-l", "info"]

containers/etl/common.py

@@ -0,0 +1,119 @@
+import re 
+import pandas as pd
+import os 
+import random
+import jsonlines
+
+def flatten_json(data: dict) -> dict:
+    """ recursive flatten json elements from https://www.geeksforgeeks.org/flattening-json-objects-in-python/"""
+    out = {}
+
+    def flatten(x, name=""):
+        # If the Nested key-value
+        # pair is of dict type
+        if type(x) is dict:
+            for a in x:
+                flatten(x[a], name + a + "_")
+
+        # If the Nested key-value
+        # pair is of list type
+        elif type(x) is list:
+            i = 0
+
+            for a in x:
+                flatten(a, name + str(i) + "_")
+                i += 1
+        else:
+            out[name[:-1]] = x
+
+    flatten(data)
+    return out
+
+
+def construct_report(string: str) -> tuple:
+
+    # normalize sections
+    keywords = [x.replace(":","").lower() for x in re.findall("[A-Z0-9][A-Z0-9. ]*:",string)]
+
+    # normalize sections
+    paragraphs = re.findall("(\w+)*: *(.*?)(?=\s*(?:\w+:|$))", string.lower())
+    sections = []
+    for header, paragraph in paragraphs:
+        if header in [x.replace(" ","_").replace("/","_") for x in keywords]:
+            sections.append(":".join([header, ". ".join([x.strip() for x in paragraph.split(". ") if x])]))
+        else:
+            sections.append(" - ".join([header, ". ".join([x.strip() for x in paragraph.split(". ") if x])]))
+    sections = list(map(lambda a: a + "." if a[-1] != "." else a, sections))
+    paragraphs = re.findall("(\w+) *: *(.*?)(?=\s*(?:\w+:|$))", "  ".join(sections))
+
+    report = {}
+    for header, paragraph in paragraphs:
+        sentence = paragraph.replace("  ", ".  ").replace("..", ".").replace(" - ."," - ")
+        sentence = re.split(r"(?<!\w\.\w.)(?<![A-Z][a-z]\.)(?<=\.|\?)\s", sentence)
+        sentence = [x.strip() for x in sentence if len(x) > 2]
+        report[header.replace("_", " ")] = [x.replace("_", " ") for x in sentence]
+    report = flatten_json(report)
+    topic = [x.split("_")[0] for x in report.keys()]
+    body = [x for x in report.values()]
+    report = pd.DataFrame(list(zip(topic, body)))
+    try:
+        report.columns = ["paragraph", "sentence"]
+        report["ranking"] = report.index
+        report["screen"] = report["sentence"].apply(lambda x: 1 if 'interval change' in x or 'compar' in x or 'prior' in x or 'improved from' in x else 0)
+        reason = re.sub(" +", " ", " ".join([": ".join([key, value]) for (key,value) in collapse_report(report).items() if key in ['indication','history']]))
+        text = re.sub(" +", " ", " ".join([": ".join([key, value]) for (key,value) in collapse_report(report[report.screen==0]).items() if key in ['findings','impression']]))
+        if 'findings' in text and 'impression' in text:
+            return reason, text 
+        else:
+            return None, None
+    except ValueError:
+        return None, None
+
+# take a report dataframe and return a dictionary of the paragraphs
+def collapse_report(report: pd.DataFrame) -> dict:
+    """take raw text and return paragraphs in sections as key:value pairs"""
+    out = pd.merge(
+        report['paragraph'].drop_duplicates(),    
+        report.groupby(['paragraph'])['sentence'].transform(lambda x: '  '.join(x)).drop_duplicates(),
+        left_index=True,
+        right_index=True
+    )
+    structure = dict()
+    for index, row in out.iterrows():
+        structure[row['paragraph']] = row['sentence']
+    return structure
+
+
+def extract_transform(row: dict) -> None:
+
+    report_root = "./physionet.org/files/mimic-cxr/2.0.0/files"
+    image_root = "./physionet.org/files/mimic-cxr-jpg/2.0.0/files"
+
+    try:
+        scans = os.listdir(os.path.join(image_root,row["part"],row["patient"]))
+        scans = [x for x in scans if 'txt' not in x]
+        for scan in scans:
+            report = os.path.join(report_root,row["part"],row["patient"],scan+".txt")
+            if os.path.exists(report):
+                with open(report,"r") as f:
+                    original = f.read()
+                transformed = re.sub(" +"," ",original.replace("FINAL REPORT","").strip().replace("\n \n",".").replace("\n"," ")).replace(" . "," ").replace("..",".").replace("CHEST RADIOGRAPHS."," ").strip()
+                if len(transformed) > 0:
+                    reason, text = construct_report(transformed)
+                    images = [os.path.join(image_root,row["part"],row["patient"],scan,x) for x in os.listdir(os.path.join(image_root,row["part"],row["patient"],scan))]
+                    images = [x for x in images if os.path.exists(x)]
+                    random.shuffle(images) # shuffle so we can reasonably sample 1 image per study
+                    with jsonlines.open("dataset.jsonl","a") as writer:
+                        for image in images:
+                            writer.write({
+                                "fold": row["patient"][0:3],
+                                "image": image,
+                                "study": image.split("/")[-2],
+                                "original": transformed,
+                                "report": report,
+                                "patient": row["patient"],
+                                "reason": reason, 
+                                "text": " ".join([reason,text]) if reason is not None and text is not None else None
+                            })
+    except FileNotFoundError:
+        pass

containers/etl/requirements.txt

@@ -0,0 +1,2 @@
+jsonlines>=3.1.0,<3.2
+pandas>=1.5.3,<1.6

containers/etl/run.py

@@ -0,0 +1,38 @@
+import os 
+import jsonlines
+from concurrent.futures import ThreadPoolExecutor
+from common import extract_transform
+
+def run():
+
+    # remove previous executions
+    if os.path.exists("/opt/physionet/dataset.jsonl"):
+        os.remove("/opt/physionet/dataset.jsonl")
+
+    if os.path.exists("/opt/physionet/control.jsonl"):
+        os.remove("/opt/physionet/control.jsonl")
+
+    # create a control dictionary
+    root = "/opt/physionet/physionet.org/files/mimic-cxr/2.0.0/files"
+    with jsonlines.open("/opt/physionet/control.jsonl","w") as writer:
+        parts = os.listdir(root)
+        for part in parts:
+            patients = os.listdir(os.path.join(root,part))
+            for patient in patients:
+                scan = [x for x in os.listdir(os.path.join(root,part,patient))  if x.endswith('.txt')]
+                writer.write({"part": part, "patient": patient,"scan": scan})         
+
+
+    # parse each record
+    with ThreadPoolExecutor(max_workers=4) as executor:
+        with jsonlines.open("/opt/physionet/control.jsonl","r") as reader:
+            executor.map(extract_transform, reader)
+
+
+# only run it if there are files downloaded
+if __name__ == "__main__":
+    try:
+        if len(os.listdir('/opt/physionet/physionet.org/files/mimic-cxr/2.0.0/files')) > 0: 
+            run()   
+    except OSError:
+        print("not downloaded yet")

containers/jupyter/Dockerfile

@@ -0,0 +1,28 @@
+FROM python:3.9-buster
+
+RUN \
+    apt-get update && \
+    apt-get -y upgrade && \
+    apt-get clean && \
+    rm -rf /var/lib/apt/lists/*
+
+RUN useradd --create-home app
+WORKDIR /home/app
+
+COPY requirements.txt /home/app/
+
+RUN \
+    chown app:app /home/app/requirements.txt && \
+    chmod 0755 /home/app/requirements.txt
+
+USER app
+
+ENV VIRTUAL_ENV=/home/app/venv
+RUN python3 -m venv $VIRTUAL_ENV
+ENV PATH="$VIRTUAL_ENV/bin:$PATH"
+
+RUN \
+    pip install --upgrade pip && \
+    pip install -r requirements.txt
+
+CMD ["jupyter", "notebook", "--notebook-dir=/opt/notebooks", "--ip='*'", "--port=8888", "--no-browser"]

containers/jupyter/requirements.txt

@@ -0,0 +1 @@
+jupyter>=1.0.0,<1.1

containers/physionet/Dockerfile

@@ -0,0 +1,11 @@
+FROM debian:buster
+
+RUN apt-get update -y && \
+    apt-get -y install parallel wget && \
+    apt-get -y autoclean && \
+    apt-get -y autoremove && \
+    rm -rf /var/lib/apt/lists/*
+
+COPY entrypoint.sh /opt/entrypoint.sh
+
+ENTRYPOINT ["/bin/bash", "/opt/entrypoint.sh"]

containers/physionet/entrypoint.sh

@@ -0,0 +1,11 @@
+#!/bin/bash
+set -e
+
+if [ $# -eq 0 ] 
+    then 
+      echo no download requested
+    else
+        cd /opt/physionet
+        wget -A .txt -r -nc -c -np --user $PHYSIONET_USER --password $PHYSIONET_PASSWORD https://physionet.org/files/mimic-cxr/2.0.0/files/
+        seq 10 19 | parallel -j4 wget -A .jpg -r -nc -c -np --user $PHYSIONET_USER --password $PHYSIONET_PASSWORD https://physionet.org/files/mimic-cxr-jpg/2.0.0/files/p{}/
+fi

containers/physionet/run.sh

@@ -0,0 +1,6 @@
+
+if ${1:-false}; then 
+    # get the JPG
+    # spread out over 4 cores
+    echo True
+fi

containers/streamlit/Dockerfile

@@ -0,0 +1,31 @@
+FROM python:3.9-buster
+
+RUN \
+    apt-get update && \
+    apt-get -y upgrade && \
+    apt-get clean && \
+    rm -rf /var/lib/apt/lists/*
+
+RUN useradd --create-home app
+
+COPY requirements.txt /home/app/
+COPY app.py /home/app/
+
+RUN \
+    chown app:app /home/app/requirements.txt && \
+    chmod 0755 /home/app/requirements.txt && \
+    chown app:app /home/app/app.py && \
+    chmod 0755 /home/app/app.py    
+
+USER app
+WORKDIR /home/app
+
+ENV VIRTUAL_ENV=/home/app/venv
+RUN python3 -m venv $VIRTUAL_ENV
+ENV PATH="$VIRTUAL_ENV/bin:$PATH"
+
+RUN \
+    pip install --upgrade pip && \
+    pip install -r requirements.txt
+
+CMD ["streamlit", "run", "app.py"]

containers/streamlit/README.md

@@ -0,0 +1,13 @@
+---
+title: Prerad
+emoji: ⚡
+colorFrom: red
+colorTo: gray
+sdk: streamlit
+sdk_version: 1.17.0
+app_file: app.py
+pinned: false
+license: apache-2.0
+---
+
+Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference

containers/streamlit/app.py

@@ -0,0 +1,30 @@
+import streamlit as st
+from PIL import Image
+from transformers import BlipForConditionalGeneration, BlipProcessor
+
+processor = BlipProcessor.from_pretrained("nathansutton/generate-cxr")
+model = BlipForConditionalGeneration.from_pretrained("nathansutton/generate-cxr")
+
+def humanize_report(report: str) -> str:
+    report = report.replace("impression :","IMPRESSION:\n").replace("findings :","FINDINGS:\n").replace("indication :","INDICATION:\n")
+    sentences = [x.split("\n") for x in report.split(".") if x]
+    sentences = [item for sublist in sentences for item in sublist]
+    sentences = [x.strip().capitalize() if ':' not in x else x for x in sentences]
+    return ".  ".join(sentences).replace(":.",":").replace("IMPRESSION:","\n\nIMPRESSION:\n\n").replace("FINDINGS:","\n\nFINDINGS:\n\n").replace("INDICATION:","INDICATION:\n\n")
+
+indication = st.text_input("What is the indication for this study")
+img_file_buffer = st.file_uploader("Upload a single view from a Chest X-Ray (JPG preferred)")
+if img_file_buffer is not None and indication is not None:
+
+    image = Image.open(img_file_buffer)
+    st.image(image, use_column_width=True)
+    inputs = processor(
+        images=Image.open(img_file_buffer), 
+        text='indication:' + indication,
+        return_tensors="pt"
+    )
+    output = model.generate(**inputs,max_length=512)
+    report = processor.decode(output[0], skip_special_tokens=True)
+    st.write(humanize_report(report))
+
+

containers/streamlit/requirements.txt

@@ -0,0 +1,3 @@
+streamlit>=1.17.0,<1.18
+transformers[torch]>=4.26.0,<4.27
+pillow>=9.4.0,<9.5

containers/train/Dockerfile

@@ -0,0 +1,31 @@
+FROM python:3.9-buster
+
+RUN \
+    apt-get update && \
+    apt-get -y upgrade && \
+    apt-get clean && \
+    rm -rf /var/lib/apt/lists/*
+
+RUN useradd --create-home app
+WORKDIR /home/app
+
+COPY requirements.txt /home/app/
+COPY run.py /home/app/
+
+RUN \
+    chown app:app /home/app/requirements.txt && \
+    chmod 0755 /home/app/requirements.txt && \
+    chown app:app /home/app/run.py && \
+    chmod 0755 /home/app/run.py
+
+USER app
+
+ENV VIRTUAL_ENV=/home/app/venv
+RUN python3 -m venv $VIRTUAL_ENV
+ENV PATH="$VIRTUAL_ENV/bin:$PATH"
+
+RUN \
+    pip install --upgrade pip && \
+    pip install -r requirements.txt
+
+CMD ["python", "run.py", "worker", "-l", "info"]

containers/train/requirements.txt

@@ -0,0 +1,3 @@
+transformers[torch]>=4.26.0,<4.27
+pillow>=9.4.0,<9.5
+datasets>=2.9.0,<2.10

containers/train/run.py

@@ -0,0 +1,83 @@
+import pandas as pd
+from datasets import Dataset, Image
+import torch
+from transformers import Trainer, TrainingArguments
+from transformers import DataCollatorForLanguageModeling
+from transformers import BlipProcessor, BlipForConditionalGeneration
+device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+
+# model initialize form pretrained
+repo = "Salesforce/blip-image-captioning-large"
+processor = BlipProcessor.from_pretrained(repo)
+tokenizer = processor.tokenizer
+model = BlipForConditionalGeneration.from_pretrained(repo)
+
+# load the data configuration and split into test/valid
+dt = pd.read_json("dataset.jsonl",lines=True).dropna()
+dt["train"] = dt["fold"].apply(lambda x: 0 if x in ['p19'] else 1) # 10% of data
+dt["patient"]= dt["patient"].apply(lambda x: x[0:5])
+train=dt[dt.train==1]
+valid=dt[dt.train==0]
+
+# create datasets
+train_dataset = Dataset.from_dict({
+    "image": train["image"].to_list(),
+    "fold": train["fold"].to_list(),
+    "text": train["text"].to_list(),
+    "reason": train["reason"].to_list(),
+    "id": [x.split("/")[-1].replace(".jpg","") for x in train["image"].to_list()]
+}).cast_column("image", Image())
+
+valid_dataset = Dataset.from_dict({
+    "image": valid["image"].to_list(),
+    "fold": valid["fold"].to_list(),
+    "text": valid["text"].to_list(),
+    "reason": valid["reason"].to_list(),
+    "id": [x.split("/")[-1].replace(".jpg","") for x in valid["image"].to_list()]
+}).cast_column("image", Image())
+
+def transform(example_batch):
+    return processor(
+        images=[image for image in example_batch["image"]],
+        text=[text for text in example_batch["text"]],
+        return_tensors="np",
+        padding='max_length',
+        max_length=512
+    )
+
+# apply 
+train_prepared = train_dataset.shuffle(seed=42).with_transform(transform)
+valid_prepared = valid_dataset.shuffle(seed=42).with_transform(transform)
+
+# " ".join(processor.batch_decode(train_prepared[0]["input_ids"])).replace(" ##","")
+training_args = TrainingArguments(
+    num_train_epochs=5,
+    evaluation_strategy="epoch",
+    save_steps=1000,
+    logging_steps=100,
+    per_device_eval_batch_size=2,
+    per_device_train_batch_size=2,
+    gradient_accumulation_steps=8,
+    lr_scheduler_type='cosine_with_restarts',
+    warmup_ratio=0.1,
+    learning_rate=5e-5,
+    save_total_limit=1,
+    output_dir="/opt/models/generate-cxr-checkpoints"
+)
+
+data_collator = DataCollatorForLanguageModeling(
+    tokenizer=tokenizer,
+    mlm = False
+)
+
+trainer = Trainer(
+    model=model,
+    tokenizer=processor,
+    args=training_args,
+    train_dataset=train_prepared,
+    eval_dataset=valid_prepared,
+    data_collator=data_collator,
+)
+
+trainer.train()
+trainer.save_model("/opt/models/generate-cxr")

docker-compose.yaml

@@ -0,0 +1,42 @@
+version: '3'
+services:
+
+  # download data
+  physionet:
+    build: "./containers/physionet"
+    container_name: physionet
+    env_file:
+      - .env
+    volumes:
+      - ./volumes/physionet:/opt/physionet
+
+  # etl
+  etl:
+    build: "./containers/etl"
+    container_name: etl
+    env_file:
+      - .env
+    volumes:
+      - ./volumes/physionet:/opt/physionet
+
+  # interactive notebooks
+  jupyter:
+    build: "./containers/jupyter"
+    container_name: jupyter
+    volumes:
+      - ./volumes/notebooks:/opt/notebooks
+      - ./volumes/physionet:/opt/physionet
+      - ./volumes/models:/opt/models      
+    ports:
+      - "8888:8888"
+
+  # interactive application
+  streamlit:
+    build: "./containers/streamlit"
+    container_name: streamlit
+    ports:
+      - "8501:8501"
+    volumes:
+      - ./volumes/models:/opt/models      
+
+

venv/.gitignore

@@ -0,0 +1,2 @@
+# Created by venv; see https://docs.python.org/3/library/venv.html
+*

venv/bin/Activate.ps1

@@ -0,0 +1,248 @@
+<#
+.Synopsis
+Activate a Python virtual environment for the current PowerShell session.
+
+.Description
+Pushes the python executable for a virtual environment to the front of the
+$Env:PATH environment variable and sets the prompt to signify that you are
+in a Python virtual environment. Makes use of the command line switches as
+well as the `pyvenv.cfg` file values present in the virtual environment.
+
+.Parameter VenvDir
+Path to the directory that contains the virtual environment to activate. The
+default value for this is the parent of the directory that the Activate.ps1
+script is located within.
+
+.Parameter Prompt
+The prompt prefix to display when this virtual environment is activated. By
+default, this prompt is the name of the virtual environment folder (VenvDir)
+surrounded by parentheses and followed by a single space (ie. '(.venv) ').
+
+.Example
+Activate.ps1
+Activates the Python virtual environment that contains the Activate.ps1 script.
+
+.Example
+Activate.ps1 -Verbose
+Activates the Python virtual environment that contains the Activate.ps1 script,
+and shows extra information about the activation as it executes.
+
+.Example
+Activate.ps1 -VenvDir C:\Users\MyUser\Common\.venv
+Activates the Python virtual environment located in the specified location.
+
+.Example
+Activate.ps1 -Prompt "MyPython"
+Activates the Python virtual environment that contains the Activate.ps1 script,
+and prefixes the current prompt with the specified string (surrounded in
+parentheses) while the virtual environment is active.
+
+.Notes
+On Windows, it may be required to enable this Activate.ps1 script by setting the
+execution policy for the user. You can do this by issuing the following PowerShell
+command:
+
+PS C:\> Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
+
+For more information on Execution Policies: 
+https://go.microsoft.com/fwlink/?LinkID=135170
+
+#>
+Param(
+    [Parameter(Mandatory = $false)]
+    [String]
+    $VenvDir,
+    [Parameter(Mandatory = $false)]
+    [String]
+    $Prompt
+)
+
+<# Function declarations --------------------------------------------------- #>
+
+<#
+.Synopsis
+Remove all shell session elements added by the Activate script, including the
+addition of the virtual environment's Python executable from the beginning of
+the PATH variable.
+
+.Parameter NonDestructive
+If present, do not remove this function from the global namespace for the
+session.
+
+#>
+function global:deactivate ([switch]$NonDestructive) {
+    # Revert to original values
+
+    # The prior prompt:
+    if (Test-Path -Path Function:_OLD_VIRTUAL_PROMPT) {
+        Copy-Item -Path Function:_OLD_VIRTUAL_PROMPT -Destination Function:prompt
+        Remove-Item -Path Function:_OLD_VIRTUAL_PROMPT
+    }
+
+    # The prior PYTHONHOME:
+    if (Test-Path -Path Env:_OLD_VIRTUAL_PYTHONHOME) {
+        Copy-Item -Path Env:_OLD_VIRTUAL_PYTHONHOME -Destination Env:PYTHONHOME
+        Remove-Item -Path Env:_OLD_VIRTUAL_PYTHONHOME
+    }
+
+    # The prior PATH:
+    if (Test-Path -Path Env:_OLD_VIRTUAL_PATH) {
+        Copy-Item -Path Env:_OLD_VIRTUAL_PATH -Destination Env:PATH
+        Remove-Item -Path Env:_OLD_VIRTUAL_PATH
+    }
+
+    # Just remove the VIRTUAL_ENV altogether:
+    if (Test-Path -Path Env:VIRTUAL_ENV) {
+        Remove-Item -Path env:VIRTUAL_ENV
+    }
+
+    # Just remove VIRTUAL_ENV_PROMPT altogether.
+    if (Test-Path -Path Env:VIRTUAL_ENV_PROMPT) {
+        Remove-Item -Path env:VIRTUAL_ENV_PROMPT
+    }
+
+    # Just remove the _PYTHON_VENV_PROMPT_PREFIX altogether:
+    if (Get-Variable -Name "_PYTHON_VENV_PROMPT_PREFIX" -ErrorAction SilentlyContinue) {
+        Remove-Variable -Name _PYTHON_VENV_PROMPT_PREFIX -Scope Global -Force
+    }
+
+    # Leave deactivate function in the global namespace if requested:
+    if (-not $NonDestructive) {
+        Remove-Item -Path function:deactivate
+    }
+}
+
+<#
+.Description
+Get-PyVenvConfig parses the values from the pyvenv.cfg file located in the
+given folder, and returns them in a map.
+
+For each line in the pyvenv.cfg file, if that line can be parsed into exactly
+two strings separated by `=` (with any amount of whitespace surrounding the =)
+then it is considered a `key = value` line. The left hand string is the key,
+the right hand is the value.
+
+If the value starts with a `'` or a `"` then the first and last character is
+stripped from the value before being captured.
+
+.Parameter ConfigDir
+Path to the directory that contains the `pyvenv.cfg` file.
+#>
+function Get-PyVenvConfig(
+    [String]
+    $ConfigDir
+) {
+    Write-Verbose "Given ConfigDir=$ConfigDir, obtain values in pyvenv.cfg"
+
+    # Ensure the file exists, and issue a warning if it doesn't (but still allow the function to continue).
+    $pyvenvConfigPath = Join-Path -Resolve -Path $ConfigDir -ChildPath 'pyvenv.cfg' -ErrorAction Continue
+
+    # An empty map will be returned if no config file is found.
+    $pyvenvConfig = @{ }
+
+    if ($pyvenvConfigPath) {
+
+        Write-Verbose "File exists, parse `key = value` lines"
+        $pyvenvConfigContent = Get-Content -Path $pyvenvConfigPath
+
+        $pyvenvConfigContent | ForEach-Object {
+            $keyval = $PSItem -split "\s*=\s*", 2
+            if ($keyval[0] -and $keyval[1]) {
+                $val = $keyval[1]
+
+                # Remove extraneous quotations around a string value.
+                if ("'""".Contains($val.Substring(0, 1))) {
+                    $val = $val.Substring(1, $val.Length - 2)
+                }
+
+                $pyvenvConfig[$keyval[0]] = $val
+                Write-Verbose "Adding Key: '$($keyval[0])'='$val'"
+            }
+        }
+    }
+    return $pyvenvConfig
+}
+
+
+<# Begin Activate script --------------------------------------------------- #>
+
+# Determine the containing directory of this script
+$VenvExecPath = Split-Path -Parent $MyInvocation.MyCommand.Definition
+$VenvExecDir = Get-Item -Path $VenvExecPath
+
+Write-Verbose "Activation script is located in path: '$VenvExecPath'"
+Write-Verbose "VenvExecDir Fullname: '$($VenvExecDir.FullName)"
+Write-Verbose "VenvExecDir Name: '$($VenvExecDir.Name)"
+
+# Set values required in priority: CmdLine, ConfigFile, Default
+# First, get the location of the virtual environment, it might not be
+# VenvExecDir if specified on the command line.
+if ($VenvDir) {
+    Write-Verbose "VenvDir given as parameter, using '$VenvDir' to determine values"
+}
+else {
+    Write-Verbose "VenvDir not given as a parameter, using parent directory name as VenvDir."
+    $VenvDir = $VenvExecDir.Parent.FullName.TrimEnd("\\/")
+    Write-Verbose "VenvDir=$VenvDir"
+}
+
+# Next, read the `pyvenv.cfg` file to determine any required value such
+# as `prompt`.
+$pyvenvCfg = Get-PyVenvConfig -ConfigDir $VenvDir
+
+# Next, set the prompt from the command line, or the config file, or
+# just use the name of the virtual environment folder.
+if ($Prompt) {
+    Write-Verbose "Prompt specified as argument, using '$Prompt'"
+}
+else {
+    Write-Verbose "Prompt not specified as argument to script, checking pyvenv.cfg value"
+    if ($pyvenvCfg -and $pyvenvCfg['prompt']) {
+        Write-Verbose "  Setting based on value in pyvenv.cfg='$($pyvenvCfg['prompt'])'"
+        $Prompt = $pyvenvCfg['prompt'];
+    }
+    else {
+        Write-Verbose "  Setting prompt based on parent's directory's name. (Is the directory name passed to venv module when creating the virtual environment)"
+        Write-Verbose "  Got leaf-name of $VenvDir='$(Split-Path -Path $venvDir -Leaf)'"
+        $Prompt = Split-Path -Path $venvDir -Leaf
+    }
+}
+
+Write-Verbose "Prompt = '$Prompt'"
+Write-Verbose "VenvDir='$VenvDir'"
+
+# Deactivate any currently active virtual environment, but leave the
+# deactivate function in place.
+deactivate -nondestructive
+
+# Now set the environment variable VIRTUAL_ENV, used by many tools to determine
+# that there is an activated venv.
+$env:VIRTUAL_ENV = $VenvDir
+
+$env:VIRTUAL_ENV_PROMPT = $Prompt
+
+if (-not $Env:VIRTUAL_ENV_DISABLE_PROMPT) {
+
+    Write-Verbose "Setting prompt to '$Prompt'"
+
+    # Set the prompt to include the env name
+    # Make sure _OLD_VIRTUAL_PROMPT is global
+    function global:_OLD_VIRTUAL_PROMPT { "" }
+    Copy-Item -Path function:prompt -Destination function:_OLD_VIRTUAL_PROMPT
+    New-Variable -Name _PYTHON_VENV_PROMPT_PREFIX -Description "Python virtual environment prompt prefix" -Scope Global -Option ReadOnly -Visibility Public -Value $Prompt
+
+    function global:prompt {
+        Write-Host -NoNewline -ForegroundColor Green "($_PYTHON_VENV_PROMPT_PREFIX) "
+        _OLD_VIRTUAL_PROMPT
+    }
+}
+
+# Clear PYTHONHOME
+if (Test-Path -Path Env:PYTHONHOME) {
+    Copy-Item -Path Env:PYTHONHOME -Destination Env:_OLD_VIRTUAL_PYTHONHOME
+    Remove-Item -Path Env:PYTHONHOME
+}
+
+# Add the venv to the PATH
+Copy-Item -Path Env:PATH -Destination Env:_OLD_VIRTUAL_PATH
+$Env:PATH = "$VenvExecDir$([System.IO.Path]::PathSeparator)$Env:PATH"

venv/bin/activate

@@ -0,0 +1,76 @@
+# This file must be used with "source bin/activate" *from bash*
+# You cannot run it directly
+
+deactivate () {
+    # reset old environment variables
+    if [ -n "${_OLD_VIRTUAL_PATH:-}" ] ; then
+        PATH="${_OLD_VIRTUAL_PATH:-}"
+        export PATH
+        unset _OLD_VIRTUAL_PATH
+    fi
+    if [ -n "${_OLD_VIRTUAL_PYTHONHOME:-}" ] ; then
+        PYTHONHOME="${_OLD_VIRTUAL_PYTHONHOME:-}"
+        export PYTHONHOME
+        unset _OLD_VIRTUAL_PYTHONHOME
+    fi
+
+    # Call hash to forget past locations. Without forgetting
+    # past locations the $PATH changes we made may not be respected.
+    # See "man bash" for more details. hash is usually a builtin of your shell
+    hash -r 2> /dev/null
+
+    if [ -n "${_OLD_VIRTUAL_PS1:-}" ] ; then
+        PS1="${_OLD_VIRTUAL_PS1:-}"
+        export PS1
+        unset _OLD_VIRTUAL_PS1
+    fi
+
+    unset VIRTUAL_ENV
+    unset VIRTUAL_ENV_PROMPT
+    if [ ! "${1:-}" = "nondestructive" ] ; then
+    # Self destruct!
+        unset -f deactivate
+    fi
+}
+
+# unset irrelevant variables
+deactivate nondestructive
+
+# on Windows, a path can contain colons and backslashes and has to be converted:
+case "$(uname)" in
+    CYGWIN*|MSYS*|MINGW*)
+        # transform D:\path\to\venv to /d/path/to/venv on MSYS and MINGW
+        # and to /cygdrive/d/path/to/venv on Cygwin
+        VIRTUAL_ENV=$(cygpath /home/dheena/prerad/venv)
+        export VIRTUAL_ENV
+        ;;
+    *)
+        # use the path as-is
+        export VIRTUAL_ENV=/home/dheena/prerad/venv
+        ;;
+esac
+
+_OLD_VIRTUAL_PATH="$PATH"
+PATH="$VIRTUAL_ENV/"bin":$PATH"
+export PATH
+
+VIRTUAL_ENV_PROMPT=venv
+export VIRTUAL_ENV_PROMPT
+
+# unset PYTHONHOME if set
+# this will fail if PYTHONHOME is set to the empty string (which is bad anyway)
+# could use `if (set -u; : $PYTHONHOME) ;` in bash
+if [ -n "${PYTHONHOME:-}" ] ; then
+    _OLD_VIRTUAL_PYTHONHOME="${PYTHONHOME:-}"
+    unset PYTHONHOME
+fi
+
+if [ -z "${VIRTUAL_ENV_DISABLE_PROMPT:-}" ] ; then
+    _OLD_VIRTUAL_PS1="${PS1:-}"
+    PS1="("venv") ${PS1:-}"
+    export PS1
+fi
+
+# Call hash to forget past commands. Without forgetting
+# past commands the $PATH changes we made may not be respected
+hash -r 2> /dev/null

venv/bin/activate.csh

@@ -0,0 +1,27 @@
+# This file must be used with "source bin/activate.csh" *from csh*.
+# You cannot run it directly.
+
+# Created by Davide Di Blasi <[email protected]>.
+# Ported to Python 3.3 venv by Andrew Svetlov <[email protected]>
+
+alias deactivate 'test $?_OLD_VIRTUAL_PATH != 0 && setenv PATH "$_OLD_VIRTUAL_PATH" && unset _OLD_VIRTUAL_PATH; rehash; test $?_OLD_VIRTUAL_PROMPT != 0 && set prompt="$_OLD_VIRTUAL_PROMPT" && unset _OLD_VIRTUAL_PROMPT; unsetenv VIRTUAL_ENV; unsetenv VIRTUAL_ENV_PROMPT; test "\!:*" != "nondestructive" && unalias deactivate'
+
+# Unset irrelevant variables.
+deactivate nondestructive
+
+setenv VIRTUAL_ENV /home/dheena/prerad/venv
+
+set _OLD_VIRTUAL_PATH="$PATH"
+setenv PATH "$VIRTUAL_ENV/"bin":$PATH"
+setenv VIRTUAL_ENV_PROMPT venv
+
+
+set _OLD_VIRTUAL_PROMPT="$prompt"
+
+if (! "$?VIRTUAL_ENV_DISABLE_PROMPT") then
+    set prompt = "("venv") $prompt:q"
+endif
+
+alias pydoc python -m pydoc
+
+rehash

venv/bin/activate.fish

@@ -0,0 +1,69 @@
+# This file must be used with "source <venv>/bin/activate.fish" *from fish*
+# (https://fishshell.com/). You cannot run it directly.
+
+function deactivate  -d "Exit virtual environment and return to normal shell environment"
+    # reset old environment variables
+    if test -n "$_OLD_VIRTUAL_PATH"
+        set -gx PATH $_OLD_VIRTUAL_PATH
+        set -e _OLD_VIRTUAL_PATH
+    end
+    if test -n "$_OLD_VIRTUAL_PYTHONHOME"
+        set -gx PYTHONHOME $_OLD_VIRTUAL_PYTHONHOME
+        set -e _OLD_VIRTUAL_PYTHONHOME
+    end
+
+    if test -n "$_OLD_FISH_PROMPT_OVERRIDE"
+        set -e _OLD_FISH_PROMPT_OVERRIDE
+        # prevents error when using nested fish instances (Issue #93858)
+        if functions -q _old_fish_prompt
+            functions -e fish_prompt
+            functions -c _old_fish_prompt fish_prompt
+            functions -e _old_fish_prompt
+        end
+    end
+
+    set -e VIRTUAL_ENV
+    set -e VIRTUAL_ENV_PROMPT
+    if test "$argv[1]" != "nondestructive"
+        # Self-destruct!
+        functions -e deactivate
+    end
+end
+
+# Unset irrelevant variables.
+deactivate nondestructive
+
+set -gx VIRTUAL_ENV /home/dheena/prerad/venv
+
+set -gx _OLD_VIRTUAL_PATH $PATH
+set -gx PATH "$VIRTUAL_ENV/"bin $PATH
+set -gx VIRTUAL_ENV_PROMPT venv
+
+# Unset PYTHONHOME if set.
+if set -q PYTHONHOME
+    set -gx _OLD_VIRTUAL_PYTHONHOME $PYTHONHOME
+    set -e PYTHONHOME
+end
+
+if test -z "$VIRTUAL_ENV_DISABLE_PROMPT"
+    # fish uses a function instead of an env var to generate the prompt.
+
+    # Save the current fish_prompt function as the function _old_fish_prompt.
+    functions -c fish_prompt _old_fish_prompt
+
+    # With the original prompt function renamed, we can override with our own.
+    function fish_prompt
+        # Save the return status of the last command.
+        set -l old_status $status
+
+        # Output the venv prompt; color taken from the blue of the Python logo.
+        printf "%s(%s)%s " (set_color 4B8BBE) venv (set_color normal)
+
+        # Restore the return status of the previous command.
+        echo "exit $old_status" | .
+        # Output the original/"old" prompt.
+        _old_fish_prompt
+    end
+
+    set -gx _OLD_FISH_PROMPT_OVERRIDE "$VIRTUAL_ENV"
+end

venv/bin/f2py

@@ -0,0 +1,7 @@
+#!/home/dheena/prerad/venv/bin/python
+import sys
+from numpy.f2py.f2py2e import main
+if __name__ == '__main__':
+    if sys.argv[0].endswith('.exe'):
+        sys.argv[0] = sys.argv[0][:-4]
+    sys.exit(main())

venv/bin/hf

@@ -0,0 +1,7 @@
+#!/home/dheena/prerad/venv/bin/python
+import sys
+from huggingface_hub.cli.hf import main
+if __name__ == '__main__':
+    if sys.argv[0].endswith('.exe'):
+        sys.argv[0] = sys.argv[0][:-4]
+    sys.exit(main())

venv/bin/huggingface-cli

@@ -0,0 +1,7 @@
+#!/home/dheena/prerad/venv/bin/python
+import sys
+from huggingface_hub.commands.huggingface_cli import main
+if __name__ == '__main__':
+    if sys.argv[0].endswith('.exe'):
+        sys.argv[0] = sys.argv[0][:-4]
+    sys.exit(main())

venv/bin/normalizer

@@ -0,0 +1,7 @@
+#!/home/dheena/prerad/venv/bin/python
+import sys
+from charset_normalizer.cli import cli_detect
+if __name__ == '__main__':
+    if sys.argv[0].endswith('.exe'):
+        sys.argv[0] = sys.argv[0][:-4]
+    sys.exit(cli_detect())

venv/bin/numpy-config

@@ -0,0 +1,7 @@
+#!/home/dheena/prerad/venv/bin/python
+import sys
+from numpy._configtool import main
+if __name__ == '__main__':
+    if sys.argv[0].endswith('.exe'):
+        sys.argv[0] = sys.argv[0][:-4]
+    sys.exit(main())

venv/bin/pip

@@ -0,0 +1,7 @@
+#!/home/dheena/prerad/venv/bin/python
+import sys
+from pip._internal.cli.main import main
+if __name__ == '__main__':
+    if sys.argv[0].endswith('.exe'):
+        sys.argv[0] = sys.argv[0][:-4]
+    sys.exit(main())

venv/bin/pip3

@@ -0,0 +1,7 @@
+#!/home/dheena/prerad/venv/bin/python
+import sys
+from pip._internal.cli.main import main
+if __name__ == '__main__':
+    if sys.argv[0].endswith('.exe'):
+        sys.argv[0] = sys.argv[0][:-4]
+    sys.exit(main())

venv/bin/pip3.13

@@ -0,0 +1,7 @@
+#!/home/dheena/prerad/venv/bin/python
+import sys
+from pip._internal.cli.main import main
+if __name__ == '__main__':
+    if sys.argv[0].endswith('.exe'):
+        sys.argv[0] = sys.argv[0][:-4]
+    sys.exit(main())

venv/bin/tiny-agents

@@ -0,0 +1,7 @@
+#!/home/dheena/prerad/venv/bin/python
+import sys
+from huggingface_hub.inference._mcp.cli import app
+if __name__ == '__main__':
+    if sys.argv[0].endswith('.exe'):
+        sys.argv[0] = sys.argv[0][:-4]
+    sys.exit(app())

venv/bin/tqdm

@@ -0,0 +1,7 @@
+#!/home/dheena/prerad/venv/bin/python
+import sys
+from tqdm.cli import main
+if __name__ == '__main__':
+    if sys.argv[0].endswith('.exe'):
+        sys.argv[0] = sys.argv[0][:-4]
+    sys.exit(main())

venv/bin/transformers

@@ -0,0 +1,7 @@
+#!/home/dheena/prerad/venv/bin/python
+import sys
+from transformers.commands.transformers_cli import main
+if __name__ == '__main__':
+    if sys.argv[0].endswith('.exe'):
+        sys.argv[0] = sys.argv[0][:-4]
+    sys.exit(main())

venv/bin/transformers-cli

@@ -0,0 +1,7 @@
+#!/home/dheena/prerad/venv/bin/python
+import sys
+from transformers.commands.transformers_cli import main_cli
+if __name__ == '__main__':
+    if sys.argv[0].endswith('.exe'):
+        sys.argv[0] = sys.argv[0][:-4]
+    sys.exit(main_cli())

venv/lib/python3.13/site-packages/huggingface_hub-0.36.0.dist-info/INSTALLER

@@ -0,0 +1 @@
+pip

venv/lib/python3.13/site-packages/huggingface_hub-0.36.0.dist-info/LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

venv/lib/python3.13/site-packages/huggingface_hub-0.36.0.dist-info/METADATA

@@ -0,0 +1,334 @@
+Metadata-Version: 2.1
+Name: huggingface-hub
+Version: 0.36.0
+Summary: Client library to download and publish models, datasets and other repos on the huggingface.co hub
+Home-page: https://github.com/huggingface/huggingface_hub
+Author: Hugging Face, Inc.
+Author-email: [email protected]
+License: Apache
+Keywords: model-hub machine-learning models natural-language-processing deep-learning pytorch pretrained-models
+Platform: UNKNOWN
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Education
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.8.0
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: filelock
+Requires-Dist: fsspec>=2023.5.0
+Requires-Dist: packaging>=20.9
+Requires-Dist: pyyaml>=5.1
+Requires-Dist: requests
+Requires-Dist: tqdm>=4.42.1
+Requires-Dist: typing-extensions>=3.7.4.3
+Requires-Dist: hf-xet<2.0.0,>=1.1.3; platform_machine == "x86_64" or platform_machine == "amd64" or platform_machine == "arm64" or platform_machine == "aarch64"
+Provides-Extra: all
+Requires-Dist: InquirerPy==0.3.4; extra == "all"
+Requires-Dist: aiohttp; extra == "all"
+Requires-Dist: authlib>=1.3.2; extra == "all"
+Requires-Dist: fastapi; extra == "all"
+Requires-Dist: httpx; extra == "all"
+Requires-Dist: itsdangerous; extra == "all"
+Requires-Dist: jedi; extra == "all"
+Requires-Dist: Jinja2; extra == "all"
+Requires-Dist: pytest<8.2.2,>=8.1.1; extra == "all"
+Requires-Dist: pytest-cov; extra == "all"
+Requires-Dist: pytest-env; extra == "all"
+Requires-Dist: pytest-xdist; extra == "all"
+Requires-Dist: pytest-vcr; extra == "all"
+Requires-Dist: pytest-asyncio; extra == "all"
+Requires-Dist: pytest-rerunfailures<16.0; extra == "all"
+Requires-Dist: pytest-mock; extra == "all"
+Requires-Dist: urllib3<2.0; extra == "all"
+Requires-Dist: soundfile; extra == "all"
+Requires-Dist: Pillow; extra == "all"
+Requires-Dist: gradio>=4.0.0; extra == "all"
+Requires-Dist: numpy; extra == "all"
+Requires-Dist: ruff>=0.9.0; extra == "all"
+Requires-Dist: libcst>=1.4.0; extra == "all"
+Requires-Dist: ty; extra == "all"
+Requires-Dist: typing-extensions>=4.8.0; extra == "all"
+Requires-Dist: types-PyYAML; extra == "all"
+Requires-Dist: types-requests; extra == "all"
+Requires-Dist: types-simplejson; extra == "all"
+Requires-Dist: types-toml; extra == "all"
+Requires-Dist: types-tqdm; extra == "all"
+Requires-Dist: types-urllib3; extra == "all"
+Requires-Dist: mypy<1.15.0,>=1.14.1; python_version == "3.8" and extra == "all"
+Requires-Dist: mypy==1.15.0; python_version >= "3.9" and extra == "all"
+Provides-Extra: cli
+Requires-Dist: InquirerPy==0.3.4; extra == "cli"
+Provides-Extra: dev
+Requires-Dist: InquirerPy==0.3.4; extra == "dev"
+Requires-Dist: aiohttp; extra == "dev"
+Requires-Dist: authlib>=1.3.2; extra == "dev"
+Requires-Dist: fastapi; extra == "dev"
+Requires-Dist: httpx; extra == "dev"
+Requires-Dist: itsdangerous; extra == "dev"
+Requires-Dist: jedi; extra == "dev"
+Requires-Dist: Jinja2; extra == "dev"
+Requires-Dist: pytest<8.2.2,>=8.1.1; extra == "dev"
+Requires-Dist: pytest-cov; extra == "dev"
+Requires-Dist: pytest-env; extra == "dev"
+Requires-Dist: pytest-xdist; extra == "dev"
+Requires-Dist: pytest-vcr; extra == "dev"
+Requires-Dist: pytest-asyncio; extra == "dev"
+Requires-Dist: pytest-rerunfailures<16.0; extra == "dev"
+Requires-Dist: pytest-mock; extra == "dev"
+Requires-Dist: urllib3<2.0; extra == "dev"
+Requires-Dist: soundfile; extra == "dev"
+Requires-Dist: Pillow; extra == "dev"
+Requires-Dist: gradio>=4.0.0; extra == "dev"
+Requires-Dist: numpy; extra == "dev"
+Requires-Dist: ruff>=0.9.0; extra == "dev"
+Requires-Dist: libcst>=1.4.0; extra == "dev"
+Requires-Dist: ty; extra == "dev"
+Requires-Dist: typing-extensions>=4.8.0; extra == "dev"
+Requires-Dist: types-PyYAML; extra == "dev"
+Requires-Dist: types-requests; extra == "dev"
+Requires-Dist: types-simplejson; extra == "dev"
+Requires-Dist: types-toml; extra == "dev"
+Requires-Dist: types-tqdm; extra == "dev"
+Requires-Dist: types-urllib3; extra == "dev"
+Requires-Dist: mypy<1.15.0,>=1.14.1; python_version == "3.8" and extra == "dev"
+Requires-Dist: mypy==1.15.0; python_version >= "3.9" and extra == "dev"
+Provides-Extra: fastai
+Requires-Dist: toml; extra == "fastai"
+Requires-Dist: fastai>=2.4; extra == "fastai"
+Requires-Dist: fastcore>=1.3.27; extra == "fastai"
+Provides-Extra: hf_transfer
+Requires-Dist: hf-transfer>=0.1.4; extra == "hf-transfer"
+Provides-Extra: hf_xet
+Requires-Dist: hf-xet<2.0.0,>=1.1.2; extra == "hf-xet"
+Provides-Extra: inference
+Requires-Dist: aiohttp; extra == "inference"
+Provides-Extra: mcp
+Requires-Dist: mcp>=1.8.0; extra == "mcp"
+Requires-Dist: typer; extra == "mcp"
+Requires-Dist: aiohttp; extra == "mcp"
+Provides-Extra: oauth
+Requires-Dist: authlib>=1.3.2; extra == "oauth"
+Requires-Dist: fastapi; extra == "oauth"
+Requires-Dist: httpx; extra == "oauth"
+Requires-Dist: itsdangerous; extra == "oauth"
+Provides-Extra: quality
+Requires-Dist: ruff>=0.9.0; extra == "quality"
+Requires-Dist: libcst>=1.4.0; extra == "quality"
+Requires-Dist: ty; extra == "quality"
+Requires-Dist: mypy<1.15.0,>=1.14.1; python_version == "3.8" and extra == "quality"
+Requires-Dist: mypy==1.15.0; python_version >= "3.9" and extra == "quality"
+Provides-Extra: tensorflow
+Requires-Dist: tensorflow; extra == "tensorflow"
+Requires-Dist: pydot; extra == "tensorflow"
+Requires-Dist: graphviz; extra == "tensorflow"
+Provides-Extra: tensorflow-testing
+Requires-Dist: tensorflow; extra == "tensorflow-testing"
+Requires-Dist: keras<3.0; extra == "tensorflow-testing"
+Provides-Extra: testing
+Requires-Dist: InquirerPy==0.3.4; extra == "testing"
+Requires-Dist: aiohttp; extra == "testing"
+Requires-Dist: authlib>=1.3.2; extra == "testing"
+Requires-Dist: fastapi; extra == "testing"
+Requires-Dist: httpx; extra == "testing"
+Requires-Dist: itsdangerous; extra == "testing"
+Requires-Dist: jedi; extra == "testing"
+Requires-Dist: Jinja2; extra == "testing"
+Requires-Dist: pytest<8.2.2,>=8.1.1; extra == "testing"
+Requires-Dist: pytest-cov; extra == "testing"
+Requires-Dist: pytest-env; extra == "testing"
+Requires-Dist: pytest-xdist; extra == "testing"
+Requires-Dist: pytest-vcr; extra == "testing"
+Requires-Dist: pytest-asyncio; extra == "testing"
+Requires-Dist: pytest-rerunfailures<16.0; extra == "testing"
+Requires-Dist: pytest-mock; extra == "testing"
+Requires-Dist: urllib3<2.0; extra == "testing"
+Requires-Dist: soundfile; extra == "testing"
+Requires-Dist: Pillow; extra == "testing"
+Requires-Dist: gradio>=4.0.0; extra == "testing"
+Requires-Dist: numpy; extra == "testing"
+Provides-Extra: torch
+Requires-Dist: torch; extra == "torch"
+Requires-Dist: safetensors[torch]; extra == "torch"
+Provides-Extra: typing
+Requires-Dist: typing-extensions>=4.8.0; extra == "typing"
+Requires-Dist: types-PyYAML; extra == "typing"
+Requires-Dist: types-requests; extra == "typing"
+Requires-Dist: types-simplejson; extra == "typing"
+Requires-Dist: types-toml; extra == "typing"
+Requires-Dist: types-tqdm; extra == "typing"
+Requires-Dist: types-urllib3; extra == "typing"
+
+<p align="center">
+  <picture>
+    <source media="(prefers-color-scheme: dark)" srcset="https://huggingface.co/datasets/huggingface/documentation-images/raw/main/huggingface_hub-dark.svg">
+    <source media="(prefers-color-scheme: light)" srcset="https://huggingface.co/datasets/huggingface/documentation-images/raw/main/huggingface_hub.svg">
+    <img alt="huggingface_hub library logo" src="https://huggingface.co/datasets/huggingface/documentation-images/raw/main/huggingface_hub.svg" width="352" height="59" style="max-width: 100%;">
+  </picture>
+  <br/>
+  <br/>
+</p> 
+
+<p align="center">
+    <i>The official Python client for the Huggingface Hub.</i>
+</p>
+
+<p align="center">
+    <a href="https://huggingface.co/docs/huggingface_hub/en/index"><img alt="Documentation" src="https://img.shields.io/website/http/huggingface.co/docs/huggingface_hub/index.svg?down_color=red&down_message=offline&up_message=online&label=doc"></a>
+    <a href="https://github.com/huggingface/huggingface_hub/releases"><img alt="GitHub release" src="https://img.shields.io/github/release/huggingface/huggingface_hub.svg"></a>
+    <a href="https://github.com/huggingface/huggingface_hub"><img alt="PyPi version" src="https://img.shields.io/pypi/pyversions/huggingface_hub.svg"></a>
+    <a href="https://pypi.org/project/huggingface-hub"><img alt="PyPI - Downloads" src="https://img.shields.io/pypi/dm/huggingface_hub"></a>
+    <a href="https://codecov.io/gh/huggingface/huggingface_hub"><img alt="Code coverage" src="https://codecov.io/gh/huggingface/huggingface_hub/branch/main/graph/badge.svg?token=RXP95LE2XL"></a>
+</p>
+
+<h4 align="center">
+    <p>
+        <b>English</b> |
+        <a href="https://github.com/huggingface/huggingface_hub/blob/main/i18n/README_de.md">Deutsch</a> |
+        <a href="https://github.com/huggingface/huggingface_hub/blob/main/i18n/README_hi.md">हिंदी</a> |
+        <a href="https://github.com/huggingface/huggingface_hub/blob/main/i18n/README_ko.md">한국어</a> |
+        <a href="https://github.com/huggingface/huggingface_hub/blob/main/i18n/README_cn.md">中文（简体）</a>
+    <p>
+</h4>
+
+---
+
+**Documentation**: <a href="https://hf.co/docs/huggingface_hub" target="_blank">https://hf.co/docs/huggingface_hub</a>
+
+**Source Code**: <a href="https://github.com/huggingface/huggingface_hub" target="_blank">https://github.com/huggingface/huggingface_hub</a>
+
+---
+
+## Welcome to the huggingface_hub library
+
+The `huggingface_hub` library allows you to interact with the [Hugging Face Hub](https://huggingface.co/), a platform democratizing open-source Machine Learning for creators and collaborators. Discover pre-trained models and datasets for your projects or play with the thousands of machine learning apps hosted on the Hub. You can also create and share your own models, datasets and demos with the community. The `huggingface_hub` library provides a simple way to do all these things with Python.
+
+## Key features
+
+- [Download files](https://huggingface.co/docs/huggingface_hub/en/guides/download) from the Hub.
+- [Upload files](https://huggingface.co/docs/huggingface_hub/en/guides/upload) to the Hub.
+- [Manage your repositories](https://huggingface.co/docs/huggingface_hub/en/guides/repository).
+- [Run Inference](https://huggingface.co/docs/huggingface_hub/en/guides/inference) on deployed models.
+- [Search](https://huggingface.co/docs/huggingface_hub/en/guides/search) for models, datasets and Spaces.
+- [Share Model Cards](https://huggingface.co/docs/huggingface_hub/en/guides/model-cards) to document your models.
+- [Engage with the community](https://huggingface.co/docs/huggingface_hub/en/guides/community) through PRs and comments.
+
+## Installation
+
+Install the `huggingface_hub` package with [pip](https://pypi.org/project/huggingface-hub/):
+
+```bash
+pip install huggingface_hub
+```
+
+If you prefer, you can also install it with [conda](https://huggingface.co/docs/huggingface_hub/en/installation#install-with-conda).
+
+In order to keep the package minimal by default, `huggingface_hub` comes with optional dependencies useful for some use cases. For example, if you want have a complete experience for Inference, run:
+
+```bash
+pip install "huggingface_hub[inference]"
+```
+
+To learn more installation and optional dependencies, check out the [installation guide](https://huggingface.co/docs/huggingface_hub/en/installation).
+
+## Quick start
+
+### Download files
+
+Download a single file
+
+```py
+from huggingface_hub import hf_hub_download
+
+hf_hub_download(repo_id="tiiuae/falcon-7b-instruct", filename="config.json")
+```
+
+Or an entire repository
+
+```py
+from huggingface_hub import snapshot_download
+
+snapshot_download("stabilityai/stable-diffusion-2-1")
+```
+
+Files will be downloaded in a local cache folder. More details in [this guide](https://huggingface.co/docs/huggingface_hub/en/guides/manage-cache).
+
+### Login
+
+The Hugging Face Hub uses tokens to authenticate applications (see [docs](https://huggingface.co/docs/hub/security-tokens)). To log in your machine, run the following CLI:
+
+```bash
+hf auth login
+# or using an environment variable
+hf auth login --token $HUGGINGFACE_TOKEN
+```
+
+### Create a repository
+
+```py
+from huggingface_hub import create_repo
+
+create_repo(repo_id="super-cool-model")
+```
+
+### Upload files
+
+Upload a single file
+
+```py
+from huggingface_hub import upload_file
+
+upload_file(
+    path_or_fileobj="/home/lysandre/dummy-test/README.md",
+    path_in_repo="README.md",
+    repo_id="lysandre/test-model",
+)
+```
+
+Or an entire folder
+
+```py
+from huggingface_hub import upload_folder
+
+upload_folder(
+    folder_path="/path/to/local/space",
+    repo_id="username/my-cool-space",
+    repo_type="space",
+)
+```
+
+For details in the [upload guide](https://huggingface.co/docs/huggingface_hub/en/guides/upload).
+
+## Integrating to the Hub.
+
+We're partnering with cool open source ML libraries to provide free model hosting and versioning. You can find the existing integrations [here](https://huggingface.co/docs/hub/libraries).
+
+The advantages are:
+
+- Free model or dataset hosting for libraries and their users.
+- Built-in file versioning, even with very large files, thanks to a git-based approach.
+- In-browser widgets to play with the uploaded models.
+- Anyone can upload a new model for your library, they just need to add the corresponding tag for the model to be discoverable.
+- Fast downloads! We use Cloudfront (a CDN) to geo-replicate downloads so they're blazing fast from anywhere on the globe.
+- Usage stats and more features to come.
+
+If you would like to integrate your library, feel free to open an issue to begin the discussion. We wrote a [step-by-step guide](https://huggingface.co/docs/hub/adding-a-library) with ❤️ showing how to do this integration.
+
+## Contributions (feature requests, bugs, etc.) are super welcome 💙💚💛💜🧡❤️
+
+Everyone is welcome to contribute, and we value everybody's contribution. Code is not the only way to help the community.
+Answering questions, helping others, reaching out and improving the documentations are immensely valuable to the community.
+We wrote a [contribution guide](https://github.com/huggingface/huggingface_hub/blob/main/CONTRIBUTING.md) to summarize
+how to get started to contribute to this repository.
+
+

venv/lib/python3.13/site-packages/huggingface_hub-0.36.0.dist-info/RECORD

@@ -0,0 +1,336 @@
+../../../bin/hf,sha256=hYcvtyYu0U20AOsOnRUsPayoH6ntXkSPouKQjbXK41Q,213
+../../../bin/huggingface-cli,sha256=aMwOrmnSAieCYaLVnx97sGJlNQgkQVajQLYeaX_veo0,231
+../../../bin/tiny-agents,sha256=WYW6slshx3NBjSOE1HtLo9IpSHChr9TQYFPYnm2ptzU,223
+huggingface_hub-0.36.0.dist-info/INSTALLER,sha256=zuuue4knoyJ-UwPPXg8fezS7VCrXJQrAP7zeNuwvFQg,4
+huggingface_hub-0.36.0.dist-info/LICENSE,sha256=xx0jnfkXJvxRnG63LTGOxlggYnIysveWIZ6H3PNdCrQ,11357
+huggingface_hub-0.36.0.dist-info/METADATA,sha256=Js6AVj00AvKTW32IGDW5saZNiX3C-Yb1K_QRRkLeca0,14822
+huggingface_hub-0.36.0.dist-info/RECORD,,
+huggingface_hub-0.36.0.dist-info/REQUESTED,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+huggingface_hub-0.36.0.dist-info/WHEEL,sha256=tZoeGjtWxWRfdplE7E3d45VPlLNQnvbKiYnx7gwAy8A,92
+huggingface_hub-0.36.0.dist-info/entry_points.txt,sha256=HIzLhjwPTO7U_ncpW4AkmzAuaadr1ajmYagW5mdb5TM,217
+huggingface_hub-0.36.0.dist-info/top_level.txt,sha256=8KzlQJAY4miUvjAssOAJodqKOw3harNzuiwGQ9qLSSk,16
+huggingface_hub/__init__.py,sha256=AXr4hh9b3lZcbOnyMUV_Y6JXn2Q1iMCVYPnq1FYPhOQ,52675
+huggingface_hub/__pycache__/__init__.cpython-313.pyc,,
+huggingface_hub/__pycache__/_commit_api.cpython-313.pyc,,
+huggingface_hub/__pycache__/_commit_scheduler.cpython-313.pyc,,
+huggingface_hub/__pycache__/_inference_endpoints.cpython-313.pyc,,
+huggingface_hub/__pycache__/_jobs_api.cpython-313.pyc,,
+huggingface_hub/__pycache__/_local_folder.cpython-313.pyc,,
+huggingface_hub/__pycache__/_login.cpython-313.pyc,,
+huggingface_hub/__pycache__/_oauth.cpython-313.pyc,,
+huggingface_hub/__pycache__/_snapshot_download.cpython-313.pyc,,
+huggingface_hub/__pycache__/_space_api.cpython-313.pyc,,
+huggingface_hub/__pycache__/_tensorboard_logger.cpython-313.pyc,,
+huggingface_hub/__pycache__/_upload_large_folder.cpython-313.pyc,,
+huggingface_hub/__pycache__/_webhooks_payload.cpython-313.pyc,,
+huggingface_hub/__pycache__/_webhooks_server.cpython-313.pyc,,
+huggingface_hub/__pycache__/community.cpython-313.pyc,,
+huggingface_hub/__pycache__/constants.cpython-313.pyc,,
+huggingface_hub/__pycache__/dataclasses.cpython-313.pyc,,
+huggingface_hub/__pycache__/errors.cpython-313.pyc,,
+huggingface_hub/__pycache__/fastai_utils.cpython-313.pyc,,
+huggingface_hub/__pycache__/file_download.cpython-313.pyc,,
+huggingface_hub/__pycache__/hf_api.cpython-313.pyc,,
+huggingface_hub/__pycache__/hf_file_system.cpython-313.pyc,,
+huggingface_hub/__pycache__/hub_mixin.cpython-313.pyc,,
+huggingface_hub/__pycache__/inference_api.cpython-313.pyc,,
+huggingface_hub/__pycache__/keras_mixin.cpython-313.pyc,,
+huggingface_hub/__pycache__/lfs.cpython-313.pyc,,
+huggingface_hub/__pycache__/repocard.cpython-313.pyc,,
+huggingface_hub/__pycache__/repocard_data.cpython-313.pyc,,
+huggingface_hub/__pycache__/repository.cpython-313.pyc,,
+huggingface_hub/_commit_api.py,sha256=pGESDsicpWMeZnct-71635KgTfvUoyok_hPl9ZgIIWI,41010
+huggingface_hub/_commit_scheduler.py,sha256=P64poLZoTJnSyR39SN6w5s9bLyngKstWee03fpoVETQ,14660
+huggingface_hub/_inference_endpoints.py,sha256=ahmbPcEXsJ_JcMb9TDgdkD8Z2z9uytkFG3_1o6dTm8g,17598
+huggingface_hub/_jobs_api.py,sha256=OFcbChcXsLvaX4oGumsHscZKAzsueYIhh0Z6Y4ycpio,10883
+huggingface_hub/_local_folder.py,sha256=2iHXNgIT3UdSt2PvCovd0NzgVxTRypKb-rvAFLK-gZU,17305
+huggingface_hub/_login.py,sha256=TWNkZpMPkDuttQ36uoi-ozLQ1IcXVsZ42tbcQ-b-h0Q,20248
+huggingface_hub/_oauth.py,sha256=75ya9toHxC0WRKsLOAI212CrssRjTSxs16mHWWNMb3w,18714
+huggingface_hub/_snapshot_download.py,sha256=b-NzYQcvktsAirIfGQKgzQwu8w0S6lhBTvnJ5S6saw8,16166
+huggingface_hub/_space_api.py,sha256=jb6rF8qLtjaNU12D-8ygAPM26xDiHCu8CHXHowhGTmg,5470
+huggingface_hub/_tensorboard_logger.py,sha256=tUdQzx-wXF4yjoGJG2izqZrn-IPMflMBWMkl1sKYzo0,8420
+huggingface_hub/_upload_large_folder.py,sha256=l2YWLZttOw69EGdihT3y_Nhr5mweLGooZG9L8smNoHY,30066
+huggingface_hub/_webhooks_payload.py,sha256=Xm3KaK7tCOGBlXkuZvbym6zjHXrT1XCrbUFWuXiBmNY,3617
+huggingface_hub/_webhooks_server.py,sha256=RLrQuCHlDH_qUQJQOm11fKFDEhIUR2IxwazuKy-T9Uo,15672
+huggingface_hub/cli/__init__.py,sha256=xzX1qgAvrtAX4gP59WrPlvOZFLuzuTgcjvanQvcpgHc,928
+huggingface_hub/cli/__pycache__/__init__.cpython-313.pyc,,
+huggingface_hub/cli/__pycache__/_cli_utils.cpython-313.pyc,,
+huggingface_hub/cli/__pycache__/auth.cpython-313.pyc,,
+huggingface_hub/cli/__pycache__/cache.cpython-313.pyc,,
+huggingface_hub/cli/__pycache__/download.cpython-313.pyc,,
+huggingface_hub/cli/__pycache__/hf.cpython-313.pyc,,
+huggingface_hub/cli/__pycache__/jobs.cpython-313.pyc,,
+huggingface_hub/cli/__pycache__/lfs.cpython-313.pyc,,
+huggingface_hub/cli/__pycache__/repo.cpython-313.pyc,,
+huggingface_hub/cli/__pycache__/repo_files.cpython-313.pyc,,
+huggingface_hub/cli/__pycache__/system.cpython-313.pyc,,
+huggingface_hub/cli/__pycache__/upload.cpython-313.pyc,,
+huggingface_hub/cli/__pycache__/upload_large_folder.cpython-313.pyc,,
+huggingface_hub/cli/_cli_utils.py,sha256=Nt6CjbkYqQQRuh70bUXVA6rZpbZt_Sa1WqBUxjQLu6g,2095
+huggingface_hub/cli/auth.py,sha256=XSsbU7-_TS5IXdASkgUCdQeoXVG82VUyGYvOS4oLLRs,7317
+huggingface_hub/cli/cache.py,sha256=fQjYfbRUapeHsK10Y6w_Ixu9JKyuZyM7pJzExJGd_2c,15855
+huggingface_hub/cli/download.py,sha256=8b5wqhMYg3X9tar9EEeWdPZk9um1kZTI_WgBqyiatqs,7141
+huggingface_hub/cli/hf.py,sha256=SQ73_SXEQnWVJkhKT_6bwNQBHQXGOdI5qqlTTtI0XH0,2328
+huggingface_hub/cli/jobs.py,sha256=eA6Q7iy_-7vjU4SjYPvn71b2aVo2qt3q-pVxLyXCWqg,44317
+huggingface_hub/cli/lfs.py,sha256=J9MkKOGUW6GjBrKs2zZUCOaAGxpatxsEoSbBjuhDJV8,7230
+huggingface_hub/cli/repo.py,sha256=CuOqQZ7WELLk9Raf3tnyXILt9e93OrlS8Dyxx3BqdQA,10618
+huggingface_hub/cli/repo_files.py,sha256=9oeeQJx8Z0ygbTElw1o5T6dGtRbeolcXENt_ouEBvjk,4844
+huggingface_hub/cli/system.py,sha256=eLSYME7ywt5Ae3tYQnS43Tai2pR2JLtA1KGImzPt5pM,1707
+huggingface_hub/cli/upload.py,sha256=lOHR_JzfM2XL_pYK3Z1HlGnaAI-fw7xGY46Lccvbsy4,14362
+huggingface_hub/cli/upload_large_folder.py,sha256=w4RIW0yZKTnNnhDOB6yISnIo_h_Hy13KwWVzrFzczpY,6164
+huggingface_hub/commands/__init__.py,sha256=AkbM2a-iGh0Vq_xAWhK3mu3uZ44km8-X5uWjKcvcrUQ,928
+huggingface_hub/commands/__pycache__/__init__.cpython-313.pyc,,
+huggingface_hub/commands/__pycache__/_cli_utils.cpython-313.pyc,,
+huggingface_hub/commands/__pycache__/delete_cache.cpython-313.pyc,,
+huggingface_hub/commands/__pycache__/download.cpython-313.pyc,,
+huggingface_hub/commands/__pycache__/env.cpython-313.pyc,,
+huggingface_hub/commands/__pycache__/huggingface_cli.cpython-313.pyc,,
+huggingface_hub/commands/__pycache__/lfs.cpython-313.pyc,,
+huggingface_hub/commands/__pycache__/repo.cpython-313.pyc,,
+huggingface_hub/commands/__pycache__/repo_files.cpython-313.pyc,,
+huggingface_hub/commands/__pycache__/scan_cache.cpython-313.pyc,,
+huggingface_hub/commands/__pycache__/tag.cpython-313.pyc,,
+huggingface_hub/commands/__pycache__/upload.cpython-313.pyc,,
+huggingface_hub/commands/__pycache__/upload_large_folder.cpython-313.pyc,,
+huggingface_hub/commands/__pycache__/user.cpython-313.pyc,,
+huggingface_hub/commands/__pycache__/version.cpython-313.pyc,,
+huggingface_hub/commands/_cli_utils.py,sha256=ePYTIEWnU677nPvdNC5AdYcEB1400L6qYEUxMkVUzME,2329
+huggingface_hub/commands/delete_cache.py,sha256=035yACUtVUIG8tEtc5vexDoFFphzdk5IXkFTlD4WMiw,17738
+huggingface_hub/commands/download.py,sha256=0QY9ho7eiAPvFndBPttGtH6vXNk3r9AioltNwc8h1Z4,8310
+huggingface_hub/commands/env.py,sha256=qv4SmjuzUz9exo4RDMY2HqabLCKE1oRb55cBA6LN9R4,1342
+huggingface_hub/commands/huggingface_cli.py,sha256=gDi7JueyiLD0bGclTEYfHPQWpAY_WBdPfHT7vkqa5v0,2654
+huggingface_hub/commands/lfs.py,sha256=xdbnNRO04UuQemEhUGT809jFgQn9Rj-SnyT_0Ph-VYg,7342
+huggingface_hub/commands/repo.py,sha256=WcRDFqUYKB0Kz0zFopegiG614ot6VOYTAf6jht0BMss,6042
+huggingface_hub/commands/repo_files.py,sha256=ftjLCC3XCY-AMmiYiZPIdRMmIqZbqVZw-BSjBLcZup4,5054
+huggingface_hub/commands/scan_cache.py,sha256=gQlhBZgWkUzH4wrIYnvgV7CA4C7rvV2SuY0x2JCB7g0,8675
+huggingface_hub/commands/tag.py,sha256=4fgQuXJHG59lTVyOjIUZjxdJDL4JZW4q10XDPSo-gss,6382
+huggingface_hub/commands/upload.py,sha256=eAJIig4ljtO9FRyGjiz6HbHS-Q4MOQziRgzjQrl5Koo,14576
+huggingface_hub/commands/upload_large_folder.py,sha256=_1id84BFtbL8HgFRKZ-el_uPrijamz1qWlzO16KbUAc,6254
+huggingface_hub/commands/user.py,sha256=dDpi0mLYvTeYf0fhPVQyEJsn7Wrk6gWvR5YHC6RgebU,7516
+huggingface_hub/commands/version.py,sha256=rGpCbvxImY9eQqXrshYt609Iws27R75WARmKQrIo6Ok,1390
+huggingface_hub/community.py,sha256=exJxrySnXURAijkVOcreuwM5JAuuz2L1xTSDkd223wk,12365
+huggingface_hub/constants.py,sha256=nILseAp4rqLu_KQTZDpPGOhepVAPanD7azbomAvovj0,10313
+huggingface_hub/dataclasses.py,sha256=rjQfuX9MeTXZQrCQC8JvkjpARDehOiSluE7Kz1L7Ueg,17337
+huggingface_hub/errors.py,sha256=D7Lw0Jjrf8vfmD0B26LEvg-JWkU8Zq0KDPJOzFY4QLw,11201
+huggingface_hub/fastai_utils.py,sha256=m7wwWk-TdhIB1CJMigAzzUBP4eLQALutEzwjWf9Ej-o,16755
+huggingface_hub/file_download.py,sha256=-vUQWkPGlzxNqevwU4OzlGOa0ZgvS1U_RGnuO_PwqN8,78957
+huggingface_hub/hf_api.py,sha256=REMm9AFgUtyizI6tkEy6glX2Aa7-TH7-uWhlhl0q0fE,487935
+huggingface_hub/hf_file_system.py,sha256=uLeublBZhWd4309fE3eFHIN8G7RCrX2_6_gr0BYjuzQ,48338
+huggingface_hub/hub_mixin.py,sha256=Ii3w9o7XgGbj6UNPnieW5IDfaCd8OEKpIH1hRkncRDQ,38208
+huggingface_hub/inference/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+huggingface_hub/inference/__pycache__/__init__.cpython-313.pyc,,
+huggingface_hub/inference/__pycache__/_client.cpython-313.pyc,,
+huggingface_hub/inference/__pycache__/_common.cpython-313.pyc,,
+huggingface_hub/inference/_client.py,sha256=9cAIkBFuzFC5f6jVp62MJNDSUcPqxsFluhQLi6FqXdc,157536
+huggingface_hub/inference/_common.py,sha256=dI3OPg0320OOB0FRy_kqftW9F3ghEnBVA5Gi4VaSctg,15778
+huggingface_hub/inference/_generated/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+huggingface_hub/inference/_generated/__pycache__/__init__.cpython-313.pyc,,
+huggingface_hub/inference/_generated/__pycache__/_async_client.cpython-313.pyc,,
+huggingface_hub/inference/_generated/_async_client.py,sha256=DSOAXJ_TxRubPisWnVKzepXalDA7PcE-NG3oczo8iMw,163445
+huggingface_hub/inference/_generated/types/__init__.py,sha256=9WvrGQ8aThtKSNzZF06j-CIE2ZuItne8FFnea1p1u38,6557
+huggingface_hub/inference/_generated/types/__pycache__/__init__.cpython-313.pyc,,
+huggingface_hub/inference/_generated/types/__pycache__/audio_classification.cpython-313.pyc,,
+huggingface_hub/inference/_generated/types/__pycache__/audio_to_audio.cpython-313.pyc,,
+huggingface_hub/inference/_generated/types/__pycache__/automatic_speech_recognition.cpython-313.pyc,,
+huggingface_hub/inference/_generated/types/__pycache__/base.cpython-313.pyc,,
+huggingface_hub/inference/_generated/types/__pycache__/chat_completion.cpython-313.pyc,,
+huggingface_hub/inference/_generated/types/__pycache__/depth_estimation.cpython-313.pyc,,
+huggingface_hub/inference/_generated/types/__pycache__/document_question_answering.cpython-313.pyc,,
+huggingface_hub/inference/_generated/types/__pycache__/feature_extraction.cpython-313.pyc,,
+huggingface_hub/inference/_generated/types/__pycache__/fill_mask.cpython-313.pyc,,
+huggingface_hub/inference/_generated/types/__pycache__/image_classification.cpython-313.pyc,,
+huggingface_hub/inference/_generated/types/__pycache__/image_segmentation.cpython-313.pyc,,
+huggingface_hub/inference/_generated/types/__pycache__/image_to_image.cpython-313.pyc,,
+huggingface_hub/inference/_generated/types/__pycache__/image_to_text.cpython-313.pyc,,
+huggingface_hub/inference/_generated/types/__pycache__/image_to_video.cpython-313.pyc,,
+huggingface_hub/inference/_generated/types/__pycache__/object_detection.cpython-313.pyc,,
+huggingface_hub/inference/_generated/types/__pycache__/question_answering.cpython-313.pyc,,
+huggingface_hub/inference/_generated/types/__pycache__/sentence_similarity.cpython-313.pyc,,
+huggingface_hub/inference/_generated/types/__pycache__/summarization.cpython-313.pyc,,
+huggingface_hub/inference/_generated/types/__pycache__/table_question_answering.cpython-313.pyc,,
+huggingface_hub/inference/_generated/types/__pycache__/text2text_generation.cpython-313.pyc,,
+huggingface_hub/inference/_generated/types/__pycache__/text_classification.cpython-313.pyc,,
+huggingface_hub/inference/_generated/types/__pycache__/text_generation.cpython-313.pyc,,
+huggingface_hub/inference/_generated/types/__pycache__/text_to_audio.cpython-313.pyc,,
+huggingface_hub/inference/_generated/types/__pycache__/text_to_image.cpython-313.pyc,,
+huggingface_hub/inference/_generated/types/__pycache__/text_to_speech.cpython-313.pyc,,
+huggingface_hub/inference/_generated/types/__pycache__/text_to_video.cpython-313.pyc,,
+huggingface_hub/inference/_generated/types/__pycache__/token_classification.cpython-313.pyc,,
+huggingface_hub/inference/_generated/types/__pycache__/translation.cpython-313.pyc,,
+huggingface_hub/inference/_generated/types/__pycache__/video_classification.cpython-313.pyc,,
+huggingface_hub/inference/_generated/types/__pycache__/visual_question_answering.cpython-313.pyc,,
+huggingface_hub/inference/_generated/types/__pycache__/zero_shot_classification.cpython-313.pyc,,
+huggingface_hub/inference/_generated/types/__pycache__/zero_shot_image_classification.cpython-313.pyc,,
+huggingface_hub/inference/_generated/types/__pycache__/zero_shot_object_detection.cpython-313.pyc,,
+huggingface_hub/inference/_generated/types/audio_classification.py,sha256=Jg3mzfGhCSH6CfvVvgJSiFpkz6v4nNA0G4LJXacEgNc,1573
+huggingface_hub/inference/_generated/types/audio_to_audio.py,sha256=2Ep4WkePL7oJwcp5nRJqApwviumGHbft9HhXE9XLHj4,891
+huggingface_hub/inference/_generated/types/automatic_speech_recognition.py,sha256=8CEphr6rvRHgq1L5Md3tq14V0tEAmzJkemh1_7gSswo,5515
+huggingface_hub/inference/_generated/types/base.py,sha256=4XG49q0-2SOftYQ8HXQnWLxiJktou-a7IoG3kdOv-kg,6751
+huggingface_hub/inference/_generated/types/chat_completion.py,sha256=j1Y8G4g5yGs4g7N4sXWbipF8TwkQG0J-ftL9OxejkBw,11254
+huggingface_hub/inference/_generated/types/depth_estimation.py,sha256=rcpe9MhYMeLjflOwBs3KMZPr6WjOH3FYEThStG-FJ3M,929
+huggingface_hub/inference/_generated/types/document_question_answering.py,sha256=6BEYGwJcqGlah4RBJDAvWFTEXkO0mosBiMy82432nAM,3202
+huggingface_hub/inference/_generated/types/feature_extraction.py,sha256=NMWVL_TLSG5SS5bdt1-fflkZ75UMlMKeTMtmdnUTADc,1537
+huggingface_hub/inference/_generated/types/fill_mask.py,sha256=OrTgQ7Ndn0_dWK5thQhZwTOHbQni8j0iJcx9llyhRds,1708
+huggingface_hub/inference/_generated/types/image_classification.py,sha256=A-Y024o8723_n8mGVos4TwdAkVL62McGeL1iIo4VzNs,1585
+huggingface_hub/inference/_generated/types/image_segmentation.py,sha256=vrkI4SuP1Iq_iLXc-2pQhYY3SHN4gzvFBoZqbUHxU7o,1950
+huggingface_hub/inference/_generated/types/image_to_image.py,sha256=snvGbmCdqchxGef25MceD7LSKAmVkIgnoX5t71rdlAQ,2290
+huggingface_hub/inference/_generated/types/image_to_text.py,sha256=OaFEBAfgT-fOVzJ7xVermGf7VODhrc9-Jg38WrM7-2o,4810
+huggingface_hub/inference/_generated/types/image_to_video.py,sha256=bC-L_cNsDhk4s_IdSiprJ9d1NeMGePLcUp7UPpco21w,2240
+huggingface_hub/inference/_generated/types/object_detection.py,sha256=VuFlb1281qTXoSgJDmquGz-VNfEZLo2H0Rh_F6MF6ts,2000
+huggingface_hub/inference/_generated/types/question_answering.py,sha256=zw38a9_9l2k1ifYZefjkioqZ4asfSRM9M4nU3gSCmAQ,2898
+huggingface_hub/inference/_generated/types/sentence_similarity.py,sha256=w5Nj1g18eBzopZwxuDLI-fEsyaCK2KrHA5yf_XfSjgo,1052
+huggingface_hub/inference/_generated/types/summarization.py,sha256=WGGr8uDLrZg8JQgF9ZMUP9euw6uZo6zwkVZ-IfvCFI0,1487
+huggingface_hub/inference/_generated/types/table_question_answering.py,sha256=cJnIPA2fIbQP2Ejn7X_esY48qGWoXg30fnNOqCXiOVQ,2293
+huggingface_hub/inference/_generated/types/text2text_generation.py,sha256=v-418w1JNNSZ2tuW9DUl6a36TQQCADa438A3ufvcbOw,1609
+huggingface_hub/inference/_generated/types/text_classification.py,sha256=FarAjygLEfPofLfKeabzJ7PKEBItlHGoUNUOzyLRpL4,1445
+huggingface_hub/inference/_generated/types/text_generation.py,sha256=28u-1zU7elk2teP3y4u1VAtDDHzY0JZ2KEEJe5d5uvg,5922
+huggingface_hub/inference/_generated/types/text_to_audio.py,sha256=1HR9Q6s9MXqtKGTvHPLGVMum5-eg7O-Pgv6Nd0v8_HU,4741
+huggingface_hub/inference/_generated/types/text_to_image.py,sha256=sGGi1Fa0n5Pmd6G3I-F2SBJcJ1M7Gmqnng6sfi0AVzs,1903
+huggingface_hub/inference/_generated/types/text_to_speech.py,sha256=ROFuR32ijROCeqbv81Jos0lmaA8SRWyIUsWrdD4yWow,4760
+huggingface_hub/inference/_generated/types/text_to_video.py,sha256=yHXVNs3t6aYO7visrBlB5cH7kjoysxF9510aofcf_18,1790
+huggingface_hub/inference/_generated/types/token_classification.py,sha256=iblAcgfxXeaLYJ14NdiiCMIQuBlarUknLkXUklhvcLI,1915
+huggingface_hub/inference/_generated/types/translation.py,sha256=xww4X5cfCYv_F0oINWLwqJRPCT6SV3VBAJuPjTs_j7o,1763
+huggingface_hub/inference/_generated/types/video_classification.py,sha256=TyydjQw2NRLK9sDGzJUVnkDeo848ebmCx588Ur8I9q0,1680
+huggingface_hub/inference/_generated/types/visual_question_answering.py,sha256=AWrQ6qo4gZa3PGedaNpzDFqx5yOYyjhnUB6iuZEj_uo,1673
+huggingface_hub/inference/_generated/types/zero_shot_classification.py,sha256=BAiebPjsqoNa8EU35Dx0pfIv8W2c4GSl-TJckV1MaxQ,1738
+huggingface_hub/inference/_generated/types/zero_shot_image_classification.py,sha256=8J9n6VqFARkWvPfAZNWEG70AlrMGldU95EGQQwn06zI,1487
+huggingface_hub/inference/_generated/types/zero_shot_object_detection.py,sha256=GUd81LIV7oEbRWayDlAVgyLmY596r1M3AW0jXDp1yTA,1630
+huggingface_hub/inference/_mcp/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+huggingface_hub/inference/_mcp/__pycache__/__init__.cpython-313.pyc,,
+huggingface_hub/inference/_mcp/__pycache__/_cli_hacks.cpython-313.pyc,,
+huggingface_hub/inference/_mcp/__pycache__/agent.cpython-313.pyc,,
+huggingface_hub/inference/_mcp/__pycache__/cli.cpython-313.pyc,,
+huggingface_hub/inference/_mcp/__pycache__/constants.cpython-313.pyc,,
+huggingface_hub/inference/_mcp/__pycache__/mcp_client.cpython-313.pyc,,
+huggingface_hub/inference/_mcp/__pycache__/types.cpython-313.pyc,,
+huggingface_hub/inference/_mcp/__pycache__/utils.cpython-313.pyc,,
+huggingface_hub/inference/_mcp/_cli_hacks.py,sha256=KX9HZJPa1p8ngY3mtYGGlVUXfg4vYbbBRs-8HLToP04,3284
+huggingface_hub/inference/_mcp/agent.py,sha256=jqvQwOajY41RIhCtD-XgVfuWbTouSYCQkIWJ1gHRrJQ,4262
+huggingface_hub/inference/_mcp/cli.py,sha256=AmSUT6wXlE6EWmI0SfQgTWYnL07322zGwwk2yMZZlBc,9640
+huggingface_hub/inference/_mcp/constants.py,sha256=kldRfaidXMdyMl_jLosaQomgWDv4shvnFe3dnQNwXSU,2511
+huggingface_hub/inference/_mcp/mcp_client.py,sha256=9rcwOO7L2Ih0oGLkeY9o5gbkwEBmsDkHKf4XAmp4Mvc,16784
+huggingface_hub/inference/_mcp/types.py,sha256=3gq-P_mrmvPI6KWBqjCxavtMPiGz10YXog7wg4oJYAo,941
+huggingface_hub/inference/_mcp/utils.py,sha256=KFsGOC8dytS3VgaugBzibdteWasZ9CAnp83U2SyIlMw,4188
+huggingface_hub/inference/_providers/__init__.py,sha256=UxPnzOdVcJgroPEatuahb4fsHaObUYPrwUCzv5ADCa4,9019
+huggingface_hub/inference/_providers/__pycache__/__init__.cpython-313.pyc,,
+huggingface_hub/inference/_providers/__pycache__/_common.cpython-313.pyc,,
+huggingface_hub/inference/_providers/__pycache__/black_forest_labs.cpython-313.pyc,,
+huggingface_hub/inference/_providers/__pycache__/cerebras.cpython-313.pyc,,
+huggingface_hub/inference/_providers/__pycache__/clarifai.cpython-313.pyc,,
+huggingface_hub/inference/_providers/__pycache__/cohere.cpython-313.pyc,,
+huggingface_hub/inference/_providers/__pycache__/fal_ai.cpython-313.pyc,,
+huggingface_hub/inference/_providers/__pycache__/featherless_ai.cpython-313.pyc,,
+huggingface_hub/inference/_providers/__pycache__/fireworks_ai.cpython-313.pyc,,
+huggingface_hub/inference/_providers/__pycache__/groq.cpython-313.pyc,,
+huggingface_hub/inference/_providers/__pycache__/hf_inference.cpython-313.pyc,,
+huggingface_hub/inference/_providers/__pycache__/hyperbolic.cpython-313.pyc,,
+huggingface_hub/inference/_providers/__pycache__/nebius.cpython-313.pyc,,
+huggingface_hub/inference/_providers/__pycache__/novita.cpython-313.pyc,,
+huggingface_hub/inference/_providers/__pycache__/nscale.cpython-313.pyc,,
+huggingface_hub/inference/_providers/__pycache__/openai.cpython-313.pyc,,
+huggingface_hub/inference/_providers/__pycache__/publicai.cpython-313.pyc,,
+huggingface_hub/inference/_providers/__pycache__/replicate.cpython-313.pyc,,
+huggingface_hub/inference/_providers/__pycache__/sambanova.cpython-313.pyc,,
+huggingface_hub/inference/_providers/__pycache__/scaleway.cpython-313.pyc,,
+huggingface_hub/inference/_providers/__pycache__/together.cpython-313.pyc,,
+huggingface_hub/inference/_providers/__pycache__/zai_org.cpython-313.pyc,,
+huggingface_hub/inference/_providers/_common.py,sha256=brZJ1CUxDKooPdmVlm4cuKjvaW_refVY0Y7CbGQe7e4,12373
+huggingface_hub/inference/_providers/black_forest_labs.py,sha256=FIukZoIFt_FDrTTDfpF-Vko5sXnmH0QvVIsMtV2Jzm8,2852
+huggingface_hub/inference/_providers/cerebras.py,sha256=QOJ-1U-os7uE7p6eUnn_P_APq-yQhx28be7c3Tq2EuA,210
+huggingface_hub/inference/_providers/clarifai.py,sha256=1cEXQwhGk4DRKiPCQUa5y-L6okTo4781EImQC8yJVOw,380
+huggingface_hub/inference/_providers/cohere.py,sha256=O3tC-qIUL91mx_mE8bOHCtDWcQuKOUauhUoXSUBUCZ8,1253
+huggingface_hub/inference/_providers/fal_ai.py,sha256=pCr5qP6R1W1CrEw-_nKdNuP3UqsUi58yL18w4r7mXRo,9989
+huggingface_hub/inference/_providers/featherless_ai.py,sha256=QxBz-32O4PztxixrIjrfKuTOzvfqyUi-cVsw0Hf_zlY,1382
+huggingface_hub/inference/_providers/fireworks_ai.py,sha256=Id226ITfPkOcFMFzly3MW9l-dZl9l4qizL4JEHWkBFk,1215
+huggingface_hub/inference/_providers/groq.py,sha256=JTk2JV4ZOlaohho7zLAFQtk92kGVsPmLJ1hmzcwsqvQ,315
+huggingface_hub/inference/_providers/hf_inference.py,sha256=0yi3cR-EJ4HYx3mSzOsMOTVmvVBkaajTzTfKB8JXQpk,9540
+huggingface_hub/inference/_providers/hyperbolic.py,sha256=OQIBi2j3aNvuaSQ8BUK1K1PVeRXdrxc80G-6YmBa-ns,1985
+huggingface_hub/inference/_providers/nebius.py,sha256=VJpTF2JZ58rznc9wxdk-57vwF8sV2vESw_WkXjXqCho,3580
+huggingface_hub/inference/_providers/novita.py,sha256=HGVC8wPraRQUuI5uBoye1Y4Wqe4X116B71GhhbWy5yM,2514
+huggingface_hub/inference/_providers/nscale.py,sha256=qWUsWinQmUbNUqehyKn34tVoWehu8gd-OZ2F4uj2SWM,1802
+huggingface_hub/inference/_providers/openai.py,sha256=GCVYeNdjWIgpQQ7E_Xv8IebmdhTi0S6WfFosz3nLtps,1089
+huggingface_hub/inference/_providers/publicai.py,sha256=1I2W6rORloB5QHSvky4njZO2XKLTwA-kPdNoauoT5rg,210
+huggingface_hub/inference/_providers/replicate.py,sha256=otVfPkfBtlWrpjQub4V__t7g_w8Ewc7ZU3efiOauW-I,3820
+huggingface_hub/inference/_providers/sambanova.py,sha256=Unt3H3jr_kgI9vzRjmmW1DFyoEuPkKCcgIIloiOj3j8,2037
+huggingface_hub/inference/_providers/scaleway.py,sha256=Jy81kXWbXCHBpx6xmyzdEfXGSyhUfjKOLHuDSvhHWGo,1209
+huggingface_hub/inference/_providers/together.py,sha256=KHF19CS3qXS7G1-CwcMiD8Z5wzPKEKi4F2DzqAthbBE,3439
+huggingface_hub/inference/_providers/zai_org.py,sha256=plGzMZuLrChZvgpS3CCPqI6ImotZZxNLgfxnR7v6tw8,646
+huggingface_hub/inference_api.py,sha256=b4-NhPSn9b44nYKV8tDKXodmE4JVdEymMWL4CVGkzlE,8323
+huggingface_hub/keras_mixin.py,sha256=gDm8PBcTqYhfrEvhu1_ptxzxbVOF3h0wAArn90UyzRA,19547
+huggingface_hub/lfs.py,sha256=v0mTThnULTmFv8MVWfrkQEwkiFXzWWx7xyp2VLf-EPo,17020
+huggingface_hub/py.typed,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+huggingface_hub/repocard.py,sha256=8tmR7SYVQZ4iBFYCmOj0yl6Ohc9Vv136s-KQKkxBq7U,34865
+huggingface_hub/repocard_data.py,sha256=hr4ReFpEQMNdh_9Dx-L-IJoI1ElHyk-h-8ZRqwVYYOE,34082
+huggingface_hub/repository.py,sha256=axZcbAh4ufXEaMgPbrS1WWgvshd-mFvYnRZAZ_yYljQ,54541
+huggingface_hub/serialization/__init__.py,sha256=kn-Fa-m4FzMnN8lNsF-SwFcfzug4CucexybGKyvZ8S0,1041
+huggingface_hub/serialization/__pycache__/__init__.cpython-313.pyc,,
+huggingface_hub/serialization/__pycache__/_base.cpython-313.pyc,,
+huggingface_hub/serialization/__pycache__/_dduf.cpython-313.pyc,,
+huggingface_hub/serialization/__pycache__/_tensorflow.cpython-313.pyc,,
+huggingface_hub/serialization/__pycache__/_torch.cpython-313.pyc,,
+huggingface_hub/serialization/_base.py,sha256=VGQ4Z9Abg2gsL_1rTGSS9p-3tkkG9eaERjlzBTLGkdU,8109
+huggingface_hub/serialization/_dduf.py,sha256=s42239rLiHwaJE36QDEmS5GH7DSmQ__BffiHJO5RjIg,15424
+huggingface_hub/serialization/_tensorflow.py,sha256=Ea3wN1bKgyb_9opj-FtH-WpIp0ptkovKimroZOudX5c,3608
+huggingface_hub/serialization/_torch.py,sha256=dw3RMkr0CYAr_TwPG_rma-ueHBRTXpfEJtrVKAvvtN4,45143
+huggingface_hub/templates/datasetcard_template.md,sha256=W-EMqR6wndbrnZorkVv56URWPG49l7MATGeI015kTvs,5503
+huggingface_hub/templates/modelcard_template.md,sha256=4AqArS3cqdtbit5Bo-DhjcnDFR-pza5hErLLTPM4Yuc,6870
+huggingface_hub/utils/__init__.py,sha256=ORfVkn5D0wuLIq12jjhTzn5_c4F8fRPxB7TG-iednuQ,3722
+huggingface_hub/utils/__pycache__/__init__.cpython-313.pyc,,
+huggingface_hub/utils/__pycache__/_auth.cpython-313.pyc,,
+huggingface_hub/utils/__pycache__/_cache_assets.cpython-313.pyc,,
+huggingface_hub/utils/__pycache__/_cache_manager.cpython-313.pyc,,
+huggingface_hub/utils/__pycache__/_chunk_utils.cpython-313.pyc,,
+huggingface_hub/utils/__pycache__/_datetime.cpython-313.pyc,,
+huggingface_hub/utils/__pycache__/_deprecation.cpython-313.pyc,,
+huggingface_hub/utils/__pycache__/_dotenv.cpython-313.pyc,,
+huggingface_hub/utils/__pycache__/_experimental.cpython-313.pyc,,
+huggingface_hub/utils/__pycache__/_fixes.cpython-313.pyc,,
+huggingface_hub/utils/__pycache__/_git_credential.cpython-313.pyc,,
+huggingface_hub/utils/__pycache__/_headers.cpython-313.pyc,,
+huggingface_hub/utils/__pycache__/_hf_folder.cpython-313.pyc,,
+huggingface_hub/utils/__pycache__/_http.cpython-313.pyc,,
+huggingface_hub/utils/__pycache__/_lfs.cpython-313.pyc,,
+huggingface_hub/utils/__pycache__/_pagination.cpython-313.pyc,,
+huggingface_hub/utils/__pycache__/_paths.cpython-313.pyc,,
+huggingface_hub/utils/__pycache__/_runtime.cpython-313.pyc,,
+huggingface_hub/utils/__pycache__/_safetensors.cpython-313.pyc,,
+huggingface_hub/utils/__pycache__/_subprocess.cpython-313.pyc,,
+huggingface_hub/utils/__pycache__/_telemetry.cpython-313.pyc,,
+huggingface_hub/utils/__pycache__/_typing.cpython-313.pyc,,
+huggingface_hub/utils/__pycache__/_validators.cpython-313.pyc,,
+huggingface_hub/utils/__pycache__/_xet.cpython-313.pyc,,
+huggingface_hub/utils/__pycache__/_xet_progress_reporting.cpython-313.pyc,,
+huggingface_hub/utils/__pycache__/endpoint_helpers.cpython-313.pyc,,
+huggingface_hub/utils/__pycache__/insecure_hashlib.cpython-313.pyc,,
+huggingface_hub/utils/__pycache__/logging.cpython-313.pyc,,
+huggingface_hub/utils/__pycache__/sha.cpython-313.pyc,,
+huggingface_hub/utils/__pycache__/tqdm.cpython-313.pyc,,
+huggingface_hub/utils/_auth.py,sha256=Ixve2vxdftHXXk2R2vfyLzlVoDT39Tkq-Hrou9KCUvw,8286
+huggingface_hub/utils/_cache_assets.py,sha256=kai77HPQMfYpROouMBQCr_gdBCaeTm996Sqj0dExbNg,5728
+huggingface_hub/utils/_cache_manager.py,sha256=XbeYoZMj8_JCl6eqRviHO6DxGSS29r5Pj38xLlao96Y,34364
+huggingface_hub/utils/_chunk_utils.py,sha256=MH7-6FwCDZ8noV6dGRytCOJGSfcZmDBvsvVotdI8TvQ,2109
+huggingface_hub/utils/_datetime.py,sha256=kCS5jaKV25kOncX1xujbXsz5iDLcjLcLw85semGNzxQ,2770
+huggingface_hub/utils/_deprecation.py,sha256=HZhRGGUX_QMKBBBwHHlffLtmCSK01TOpeXHefZbPfwI,4872
+huggingface_hub/utils/_dotenv.py,sha256=RzHqC8HgzVxE-N4DFBcnemvX0NHmXcV0My2ASK0U1OQ,2017
+huggingface_hub/utils/_experimental.py,sha256=3-c8irbn9sJr2CwWbzhGkIrdXKg8_x7BifhHFy32ei8,2470
+huggingface_hub/utils/_fixes.py,sha256=xQV1QkUn2WpLqLjtXNiyn9gh-454K6AF-Q3kwkYAQD8,4437
+huggingface_hub/utils/_git_credential.py,sha256=ao9rq-rVHn8lghSVZEjDAX4kIkNi7bayY361TDSgSpg,4619
+huggingface_hub/utils/_headers.py,sha256=w4ayq4hLGaZ3B7nwdEi5Zu23SmmDuOwv58It78wkakk,8868
+huggingface_hub/utils/_hf_folder.py,sha256=WNjTnu0Q7tqcSS9EsP4ssCJrrJMcCvAt8P_-LEtmOU8,2487
+huggingface_hub/utils/_http.py,sha256=Q7W1YoT2k47duPb9ib_FGPEXwn8B3BtrwNvjF5ZYW_w,25581
+huggingface_hub/utils/_lfs.py,sha256=EC0Oz6Wiwl8foRNkUOzrETXzAWlbgpnpxo5a410ovFY,3957
+huggingface_hub/utils/_pagination.py,sha256=EX5tRasSuQDaKbXuGYbInBK2odnSWNHgzw2tSgqeBRI,1906
+huggingface_hub/utils/_paths.py,sha256=w1ZhFmmD5ykWjp_hAvhjtOoa2ZUcOXJrF4a6O3QpAWo,5042
+huggingface_hub/utils/_runtime.py,sha256=L7SOYezdxKcwd4DovAY0UGY3qt27toXO-QjceIDwExk,11634
+huggingface_hub/utils/_safetensors.py,sha256=GW3nyv7xQcuwObKYeYoT9VhURVzG1DZTbKBKho8Bbos,4458
+huggingface_hub/utils/_subprocess.py,sha256=u9FFUDE7TrzQTiuEzlUnHx7S2P57GbYRV8u16GJwrFw,4625
+huggingface_hub/utils/_telemetry.py,sha256=54LXeIJU5pEGghPAh06gqNAR-UoxOjVLvKqAQscwqZs,4890
+huggingface_hub/utils/_typing.py,sha256=z-134-HG_qJc0cjdSXkmDm3vIRyF5aEfbZgJCB_Qp2Y,3628
+huggingface_hub/utils/_validators.py,sha256=u8AacmA9xCCyer8efmzl1EpQUWTe3zVzsWSJSv3uxTU,9190
+huggingface_hub/utils/_xet.py,sha256=f8qfk8YKePAeGUL6lQiQ1w_3bcs78oWwbeACYdUeg5k,7312
+huggingface_hub/utils/_xet_progress_reporting.py,sha256=JK64hv8orABfNnk1_Wd0YyD_5FfeyVeBvelKpjaNIvs,6169
+huggingface_hub/utils/endpoint_helpers.py,sha256=9VtIAlxQ5H_4y30sjCAgbu7XCqAtNLC7aRYxaNn0hLI,2366
+huggingface_hub/utils/insecure_hashlib.py,sha256=iAaepavFZ5Dhfa5n8KozRfQprKmvcjSnt3X58OUl9fQ,1142
+huggingface_hub/utils/logging.py,sha256=N6NXaCcbPbZSF-Oe-TY3ZnmkpmdFVyTOV8ASo-yVXLE,4916
+huggingface_hub/utils/sha.py,sha256=OFnNGCba0sNcT2gUwaVCJnldxlltrHHe0DS_PCpV3C4,2134
+huggingface_hub/utils/tqdm.py,sha256=xAKcyfnNHsZ7L09WuEM5Ew5-MDhiahLACbbN2zMmcLs,10671

venv/lib/python3.13/site-packages/huggingface_hub-0.36.0.dist-info/entry_points.txt

@@ -0,0 +1,8 @@
+[console_scripts]
+hf = huggingface_hub.cli.hf:main
+huggingface-cli = huggingface_hub.commands.huggingface_cli:main
+tiny-agents = huggingface_hub.inference._mcp.cli:app
+
+[fsspec.specs]
+hf=huggingface_hub.HfFileSystem
+

venv/pyvenv.cfg

@@ -0,0 +1,5 @@
+home = /usr/bin
+include-system-site-packages = false
+version = 3.13.7
+executable = /usr/bin/python3.13
+command = /usr/bin/python -m venv /home/dheena/prerad/venv

volumes/notebooks/.gitignore

@@ -0,0 +1 @@
+.ipynb_checkpoints

volumes/notebooks/etl.ipynb

@@ -0,0 +1,191 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "d12005fa",
+   "metadata": {},
+   "outputs": [
+    {
+     "ename": "",
+     "evalue": "",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[1;31mRunning cells with 'blip' requires ipykernel package.\n",
+      "\u001b[1;31mRun the following command to install 'ipykernel' into the Python environment. \n",
+      "\u001b[1;31mCommand: 'conda install -n blip ipykernel --update-deps --force-reinstall'"
+     ]
+    }
+   ],
+   "source": [
+    "import pandas as pd  "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "id": "e7eebe96",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "part                                                     p10\n",
+       "patient                                            p10000032\n",
+       "scan       [s50414267.txt, s53189527.txt, s53911762.txt, ...\n",
+       "Name: 0, dtype: object"
+      ]
+     },
+     "execution_count": 8,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# one row in the control dictionary\n",
+    "pd.read_json(\"/opt/physionet/control.jsonl\",lines=True).iloc[0]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "id": "1d191897",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "fold                                                        p10\n",
+       "image         /opt/physionet/physionet.org/files/mimic-cxr-j...\n",
+       "original                                       FINAL REPORT\\...\n",
+       "report        /opt/physionet/physionet.org/files/mimic-cxr/2...\n",
+       "patient                                               p10000764\n",
+       "text          findings: pa and lateral views of the chest pr...\n",
+       "indication    indication: unknown year old male with hypoxia...\n",
+       "Name: 0, dtype: object"
+      ]
+     },
+     "execution_count": 10,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# each row contains the labels and metadata needed to train the transformer\n",
+    "example = pd.read_json(\"/opt/physionet/dataset.jsonl\",lines=True).iloc[0]\n",
+    "example"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "id": "dd14ab31",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "                                 FINAL REPORT\n",
+      " EXAMINATION:  CHEST (PA AND LAT)\n",
+      " \n",
+      " INDICATION:  ___M with hypoxia  // ?pna, aspiration.\n",
+      " \n",
+      " COMPARISON:  None\n",
+      " \n",
+      " FINDINGS: \n",
+      " \n",
+      " PA and lateral views of the chest provided.   The lungs are adequately\n",
+      " aerated.\n",
+      " \n",
+      " There is a focal consolidation at the left lung base adjacent to the lateral\n",
+      " hemidiaphragm. There is mild vascular engorgement. There is bilateral apical\n",
+      " pleural thickening.\n",
+      " \n",
+      " The cardiomediastinal silhouette is remarkable for aortic arch calcifications.\n",
+      " The heart is top normal in size.\n",
+      " \n",
+      " IMPRESSION: \n",
+      " \n",
+      " Focal consolidation at the left lung base, possibly representing aspiration or\n",
+      " pneumonia.\n",
+      " \n",
+      " Central vascular engorgement.\n",
+      "\n"
+     ]
+    }
+   ],
+   "source": [
+    "# original text\n",
+    "print(example[\"original\"])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "id": "ea5792df",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "findings: pa and lateral views of the chest provided. the lungs are adequately aerated. there is a focal consolidation at the left lung base adjacent to the lateral hemidiaphragm. there is mild vascular engorgement. there is bilateral apical pleural thickening. the cardiomediastinal silhouette is remarkable for aortic arch calcifications. the heart is top normal in size. impression: focal consolidation at the left lung base, possibly representing aspiration or pneumonia. central vascular engorgement.\n"
+     ]
+    }
+   ],
+   "source": [
+    "# text used as a label for the model\n",
+    "# no indicatin\n",
+    "# no technique\n",
+    "# no comparison\n",
+    "print(example[\"text\"])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "id": "67e5bb00",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "indication: unknown year old male with hypoxia // question pna, aspiration.\n"
+     ]
+    }
+   ],
+   "source": [
+    "# used as a INPUT to the model along with the image\n",
+    "print(example[\"indication\"])"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "blip",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.8 (main, Nov 24 2022, 08:08:27) [Clang 14.0.6 ]"
+  },
+  "vscode": {
+   "interpreter": {
+    "hash": "d2929fa862ca5c20be7df7418b9bcb368752100a819a60622976f7f091b1ba7c"
+   }
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

volumes/physionet/.gitignore

@@ -0,0 +1,2 @@
+*
+!.gitignore