h-abid/bmvc-gdino-zerowaste

Robust and Label-Efficient Deep Waste Detection

This repository contains fine-tuned model checkpoints and generated pseudo-annotations from the paper:

Robust and Label-Efficient Deep Waste Detection by Hassan Abid, Khan Muhammad, and Muhammad Haris.

For the official PyTorch implementation, detailed installation, training, and evaluation scripts, please refer to the GitHub repository.

Abstract

Effective waste sorting is critical for sustainable recycling, yet AI research in this domain continues to lag behind commercial systems due to limited datasets and reliance on legacy object detectors. In this work, we advance AI-driven waste detection by establishing strong baselines and introducing an ensemble-based semi-supervised learning framework. We first benchmark state-of-the-art Open-Vocabulary Object Detection (OVOD) models on the real-world ZeroWaste dataset, demonstrating that while class-only prompts perform poorly, LLM-optimized prompts significantly enhance zero-shot accuracy. Next, to address domain-specific limitations, we fine-tune modern transformer-based detectors, achieving a new baseline of 51.6 mAP. We then propose a soft pseudo-labeling strategy that fuses ensemble predictions using spatial and consensus-aware weighting, enabling robust semi-supervised training. Applied to the unlabeled ZeroWaste-s subset, our pseudo-annotations achieve performance gains that surpass fully supervised training, underscoring the effectiveness of scalable annotation pipelines. Our work contributes to the research community by establishing rigorous baselines, introducing a robust ensemble-based pseudo-labeling pipeline, generating high-quality annotations for the unlabeled ZeroWaste-s subset, and systematically evaluating OVOD models under real-world waste sorting conditions.

Overview

We introduce a label-efficient framework for waste detection on the industrial ZeroWaste dataset that unifies zero-shot open-vocabulary evaluation, strong supervised baselines, and semi-supervised learning. Optimizing prompts nearly doubles zero-shot mAP for OVOD models, while fine-tuning modern detectors establishes a 51.6 mAP baseline. An ensemble-based soft pseudo-labeling pipeline (WBF + consensus weighting) then exploits unlabeled data to surpass full supervision (up to 54.3 mAP, +2.7–3.7 mAP). Together, these components provide a reproducible benchmark and a scalable annotation strategy for real-world material-recovery facilities.

Updates

• August 26, 2025: Paper is released arXiv
• August 25, 2025: Released code for paper
• July 25, 2025: Accepted in BMVC 2025 🎉

Contents of this Repository

This Hugging Face repository provides the following assets related to the paper:

• Fine-tuned Checkpoints:

  • weights/gdino-swin-t/zerowaste_f_finetuned_best_coco_bbox_mAP.pth (and other similar .pth files)
  • weights/gdino-swin-b/zerowaste_f_finetuned_best_coco_bbox_mAP.pth (and other similar .pth files)
  • weights/gdino-swin-t/zerowaste_semi-sup_best_coco_bbox_mAP.pth (and other similar .pth files)
  • weights/gdino-swin-b/zerowaste_semi-sup_best_coco_bbox_mAP.pth (and other similar .pth files) (For a complete list and details on these models, refer to the GitHub repository)

• Pseudo-annotations: This repository also hosts the ensemble-based pseudo-annotations generated for the unlabeled ZeroWaste-s subset, which include files like:

  • co-detr_swin-l_zerowaste-s_pseudo_annotations.json
  • dino_swin-l_zerowaste-s_pseudo_annotations.json
  • deta_swin-l_zerowaste-s_pseudo_annotations.json
  • gdino-swin-b_zerowaste-s_pseudo_annotations.json

Sample Usage (Evaluation)

You can evaluate our fine-tuned checkpoints on the ZeroWaste-f Test Set. First, ensure huggingface_hub is installed: pip install -U "huggingface_hub>=0.23.0".

Then, download the desired checkpoint (e.g., GroundingDINO Swin-B fine-tuned on ZeroWaste-f) into ./weights/:

huggingface-cli download h-abid/bmvc-gdino-zerowaste \
  --include "weights/gdino-swin-b/zerowaste_f_finetuned_best_coco_bbox_mAP.pth" \
  --local-dir ./

Finally, run the evaluation command:

python external_modules/mmdetection/tools/test.py \
external_modules/mmdetection/configs/grounding_dino/grounding_dino_swin-b_inference_zerowaste_f.py \
weights/gdino-swin-b/zerowaste_f_finetuned_best_coco_bbox_mAP.pth

Results will be stored in ./experiments/. For other models, semi-supervised evaluation, and training instructions, refer to the GitHub repository.

Citation

If you find our work or this repository useful, please consider giving a star ⭐ and citing our paper:

@misc{abid2025robustlabelefficientdeepwaste,
  title         = {Robust and Label-Efficient Deep Waste Detection},
  author        = {Hassan Abid and Khan Muhammad and Muhammad Haris Khan},
  year          = {2025},
  eprint        = {2508.18799},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  url           = {https://arxiv.org/abs/2508.18799}
}

License

This repository is released under the MIT License. Please also respect the license terms of the ZeroWaste dataset.

Acknowledgement

We used the following open-source codebases in our work and gratefully acknowledge the authors for releasing them:

• GroundingDINO for open-vocabulary detection.
• OWLv2 for zero-shot evaluation.
• Ultralytics for YOLO11, RT-DETR, and YOLO-World implementation.
• MMDetection as the training framework for fine-tuning.
• DETR-based variants used in fine-tuning: Co-DETR, DETA, DINO.

We also thank the authors of the ZeroWaste dataset for making the data publicly available, enabling reproducible research in sustainable AI.