---
library_name: transformers
tags:
  - vision
  - image-segmentation
  - universal-segmentation
  - korean-road
  - oneformer
  - distillation
  - aihub
license: cc-by-4.0
model_name: KoalaSeg-Edge-ViT
---

# KoalaSeg-Edge-ViT 🐨🛣️  
**KoalaSeg = _KOrean lAyered assistive Segmentation_**  

한국 도로 · 보행 환경 전용 **Universal Segmentation** 모델입니다.  
3-중 레이어 마스크(XML 폴리곤 ▶ AIHUB 도로보행 데이터로 학습한 한국 도로 전용 모델 ▶ OneFormer-Cityscapes)를 겹쳐 만든 통합 GT로 파인튜닝한 OneFormer Edge-ViT 학생 버전입니다.

---

## Model Details
| 항목 | 내용 |
|------|------|
| **Developed by** | Team RoadSight |
| **Model type** | Edge-ViT backbone + OneFormer head<br>(semantic-only task token) |
| **Finetuned from** | `shi-labs/oneformer_cityscapes_swin_large` |
| **Framework** | 🤗 Transformers v4.41 / PyTorch 2.3 |
| **License** | CC BY 4.0 |

---

## Training Data
| 출처 | 수량 | 주석 방식 |
|------|------|-----------|
| **AIHUB 도로·보행환경** <br> (도로 차선, 인도, 횡단보도) | 5 615 장 | 공식 pixel-wise GT |
| 자가 촬영 지방도 | 9 042 장 | CVAT XML 폴리곤 |
| Street View 파생 | 3 712 장 | OneFormer-Cityscapes pseudo-mask |
| **총합** | **18 369 장** | 3-중 레이어 합성 → Morph Open/Close + MedianBlur(17 px) |

---

## Speeds & Sizes   *(512 × 512 batch 1)*

| Device | Baseline Cityscapes | Ensemble(3-레이어) | Custom(K-Road) | **KoalaSeg(ft)** |
|--------|--------------------|-------------------|---------------|------------------|
| **A100**   | 3.58 s → 0.28 FPS | 3.74 s → 0.27 FPS | 0.15 s → 6.67 FPS | **0.14 s → 7.25 FPS** |
| **T4**     | 5.61 s → 0.18 FPS | 6.01 s → 0.17 FPS | 0.39 s → 2.60 FPS | **0.31 s → 3.27 FPS** |
| **CPU (i9-12900K)** | 124 s | 150 s | 26.6 s | **18.4 s** |

---

## Evaluation (국내 테스트셋)
| Metric | Baseline | **KoalaSeg** |
|--------|----------|--------------|
| mIoU (전체 클래스) | 0.55 | **0.81** |
| F1 – 도로 vs 인도 | 0.58 | **0.89** |

---

## Quick Start
```python
from transformers import AutoProcessor, AutoModelForUniversalSegmentation
import torch, numpy as np, matplotlib.pyplot as plt
from PIL import Image

model_id = "roadsight/KoalaSeg-Edge-ViT"
proc  = AutoProcessor.from_pretrained(model_id)
model = AutoModelForUniversalSegmentation.from_pretrained(model_id).to("cuda")

img   = Image.open("korean_road.jpg").convert("RGB")
inputs = proc(images=img, task_inputs=["semantic"], return_tensors="pt").to("cuda")
with torch.no_grad():
    out = model(**inputs)

idmap = proc.post_process_semantic_segmentation(out, target_sizes=[img.size[::-1]])[0]
plt.imshow(idmap.cpu()); plt.axis("off"); plt.show()