KPTER: K-Pointer for Temporal Event Retrieval

Abstract

The explosive proliferation of online video content necessitates advanced retrieval systems, as promoted by the Ho Chi Minh AI Challenge (AIC) 2025. This competition comprises three tasks: Known-Item Search (KIS), Visual Question Answering (VQA), and the newly introduced Temporal Retrieval and Alignment of Key Events (TRAKE).

To address these challenges, we propose a comprehensive multimodal retrieval framework. Our system integrates heterogeneous data sources, including semantic embeddings from CLIP and BEiT-3, Optical Character Recognition (OCR), Automatic Speech Recognition (ASR), and open-vocabulary object detection. The architecture features a dual-layer search mechanism. For discrete queries (KIS, VQA), a Multi-modal Search layer retrieves and ranks results from parallel data streams using the Weighted Reciprocal Rank Fusion (WRRF) algorithm. To address the sequential nature of the TRAKE task, we introduce a novel Temporal Search module built upon an efficient K-pointer sequential re-ranking algorithm. This algorithm effectively validates and ranks video segments containing events in a specified temporal order. Finally, we present a competition-oriented user interface designed for real-time interaction, supporting multi-stage query construction and precise temporal refinement tools.

Authors

Khoa Dinh Duc Anh, Duc-Tai Dinh, Trung Nguyen Le Hoang, Nhan Nguyen Thanh — Team WATLERE, University of Science, VNU-HCM.

Highlights

Best Presentation Award at SOICT 2025 (Session 10A: Lifelog Event Retrieval).
Top 12+/80+ teams at AI Challenge HCMC 2025.
Novel K-pointer sequential re-ranking algorithm for the TRAKE temporal retrieval task.
Weighted Reciprocal Rank Fusion (WRRF) for multi-stream fusion across CLIP/BEiT-3 embeddings, OCR, ASR, and open-vocabulary detection.
Competition-oriented interactive UI for multi-stage query construction and temporal refinement.

References

[Springer Nature] KPTER: K-Pointer for Temporal Event Retrieval

SOICT 2025 — The 14th International Symposium on Information and Communication Technology. Presented Saturday, December 13th, 2025.

BibTeX

1
@inproceedings{kpter2025,
2
  title     = {KPTER: K-Pointer for Temporal Event Retrieval},
3
  author    = {Dinh, Khoa Duc Anh and Dinh, Duc-Tai and Nguyen, Trung Le Hoang and Nguyen, Nhan Thanh},
4
  booktitle = {Proceedings of the 14th International Symposium on Information and Communication Technology (SOICT)},
5
  year      = {2025},
6
  note      = {Best Presentation Award}
7
}