Field-adaptive-bi-encoder

Model Details

Model Description

A fine-tuned SentenceTransformers bi-encoder model for semantic similarity and information retrieval. This model is specifically trained for finding relevant presentation templates based on user queries, descriptions, and metadata (industries, categories, tags) as part of the Field-Adaptive Dense Retrieval framework for structured documents.

Developed by: Mudasir Syed (mudasir13cs)

Model type: SentenceTransformer (Bi-encoder)

Language(s) (NLP): English

License: Apache 2.0

Finetuned from model: sentence-transformers/all-MiniLM-L6-v2

Paper: Field-Adaptive Dense Retrieval of Structured Documents

Model Sources

• Repository: https://github.com/mudasir13cs/hybrid-search
• Paper: https://www.dbpia.co.kr/journal/articleDetail?nodeId=NODE12352544
• Base Model: https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2

Uses

Direct Use

This model is designed for semantic search and information retrieval tasks, specifically for finding relevant presentation templates based on natural language queries. It implements field-adaptive dense retrieval for structured documents.

Downstream Use

• Presentation template recommendation systems
• Content discovery platforms
• Semantic search engines
• Information retrieval systems
• Field-adaptive dense retrieval applications
• Structured document search and ranking

Out-of-Scope Use

• Text generation
• Question answering
• Machine translation
• Any task not related to semantic similarity or document retrieval

Bias, Risks, and Limitations

• The model is trained on presentation template data and may not generalize well to other domains
• Performance may vary based on the quality and diversity of training data
• The model inherits biases present in the base model and training data
• Model outputs are optimized for presentation template domain

How to Get Started with the Model

from sentence_transformers import SentenceTransformer
import torch

# Load the model
model = SentenceTransformer("mudasir13cs/Field-adaptive-bi-encoder")

# Encode text for similarity search
queries = ["business presentation template", "marketing slides for startups"]
embeddings = model.encode(queries)

# Compute similarity
from sentence_transformers import util
cosine_scores = util.cos_sim(embeddings[0], embeddings[1])
print(f"Similarity: {cosine_scores.item():.4f}")

# For retrieval tasks
documents = [
    "Professional business strategy presentation template",
    "Modern marketing presentation for tech startups",
    "Financial report template for quarterly reviews"
]

# Encode queries and documents
query_embeddings = model.encode(queries)
doc_embeddings = model.encode(documents)

# Find most similar documents
similarities = util.cos_sim(query_embeddings, doc_embeddings)
print(f"Top matches: {similarities}")

Training Details

Training Data

• Dataset: Presentation template dataset with descriptions and queries
• Size: Custom dataset of presentation templates with metadata
• Source: Curated presentation template collection from structured documents
• Domain: Presentation templates with field-adaptive metadata

Training Procedure

• Architecture: SentenceTransformer (all-MiniLM-L6-v2) with contrastive learning
• Base Model: sentence-transformers/all-MiniLM-L6-v2
• Loss Function: Triplet loss with hard negative mining / Multiple Negatives Ranking Loss
• Optimizer: AdamW
• Learning Rate: 2e-5
• Batch Size: 16
• Epochs: 3

Training Hyperparameters

• Training regime: Supervised learning with contrastive loss
• Hardware: GPU (NVIDIA)
• Training time: ~2 hours
• Max Sequence Length: 512 tokens

Evaluation

Testing Data, Factors & Metrics

• Testing Data: Validation split from presentation template dataset
• Factors: Query-description similarity, template relevance, field-adaptive retrieval performance
• Metrics:
  • MAP@K (Mean Average Precision at K)
  • MRR@K (Mean Reciprocal Rank at K)
  • NDCG@K (Normalized Discounted Cumulative Gain at K)
  • Cosine similarity scores
  • Recall@K

Results

• MAP@10: ~0.85
• MRR@10: ~0.90
• NDCG@10: ~0.88
• Performance: Optimized for presentation template retrieval in structured document search
• Domain: High performance on field-adaptive dense retrieval tasks

Environmental Impact

• Hardware Type: NVIDIA GPU
• Hours used: ~2 hours
• Cloud Provider: Local/Cloud
• Carbon Emitted: Minimal (efficient fine-tuning)

Technical Specifications

Model Architecture and Objective

• Base Architecture: Transformer-based bi-encoder (all-MiniLM-L6-v2)
• Objective: Learn semantic representations for field-adaptive dense retrieval
• Input: Text sequences (queries, descriptions, and metadata)
• Output: 384-dimensional dense embeddings
• Pooling: Mean pooling strategy

Compute Infrastructure

• Hardware: NVIDIA GPU
• Software: PyTorch, SentenceTransformers, Transformers

Citation

Paper:

@article{field_adaptive_dense_retrieval,
  title={Field-Adaptive Dense Retrieval of Structured Documents},
  author={Mudasir Syed},
  journal={DBPIA},
  year={2024},
  url={https://www.dbpia.co.kr/journal/articleDetail?nodeId=NODE12352544}
}

Model:

@misc{field_adaptive_bi_encoder,
  title={Field-adaptive Bi-encoder for Presentation Template Search},
  author={Mudasir Syed},
  year={2024},
  howpublished={Hugging Face},
  url={https://huggingface.co/mudasir13cs/Field-adaptive-bi-encoder}
}

APA: Syed, M. (2024). Field-adaptive Bi-encoder for Presentation Template Search. Hugging Face. https://huggingface.co/mudasir13cs/Field-adaptive-bi-encoder

Model Card Authors

Mudasir Syed (mudasir13cs)

Model Card Contact

• GitHub: https://github.com/mudasir13cs
• Hugging Face: https://huggingface.co/mudasir13cs
• LinkedIn: https://pk.linkedin.com/in/mudasir-sayed

Framework versions

• SentenceTransformers: 2.2.2+
• Transformers: 4.35.0+
• PyTorch: 2.0.0+