ZSE-Cap: A Zero-Shot Ensemble for Image Retrieval and Prompt-Guided Captioning

Abstract

ZSE-Cap is a zero-shot ensemble approach for the EVENTA Challenge at ACM Multimedia 2025, which targets event-aware image retrieval and prompt-guided captioning over the 400K+ OpenEvents V1 dataset.

Without any task-specific fine-tuning, we extract embeddings from CLIP, SigLIP, and DINOv2 and combine them with a custom L2-distance weighted ensemble for retrieval. For captioning, we engineer prompts that inject the retrieved articles as context for a Gemma 3 model, producing event-aware captions that go beyond pixel-level descriptions.

Results

Ranked Top-4 on the private test set:

Pipeline

1. Retrieve. Query image → embeddings from CLIP, SigLIP, and DINOv2 → vector database → ensemble score → top-1 retrieved image and its source article.
2. Caption. Retrieved article + prompt template → Gemma 3 → event-aware enriched caption.

Authors

Duc-Tai Dinh (lead R&D, ductai.dt05@gmail.com), Duc Anh Khoa Dinh — Team ITxTK9, University of Science, VNU-HCM.

References

• arXiv:2507.20564

BibTeX

1
@misc{dinh2025zsecap,
2
  title         = {ZSE-Cap: A Zero-Shot Ensemble for Image Retrieval and Prompt-Guided Captioning},
3
  author        = {Dinh, Duc-Tai and Dinh, Duc Anh Khoa},
4
  year          = {2025},
5
  eprint        = {2507.20564},
6
  archivePrefix = {arXiv},
7
  primaryClass  = {cs.CV},
8
  url           = {https://arxiv.org/abs/2507.20564}
9
}