Multi-Factor Adaptive Vision Selection for Egocentric Video Question Answering
Abstract
The challenge of interpreting the world from a human perspective in Artificial Intelligence (AI) is particularly evident in egocentric video question answering, which grapples with issues like small object recognition, noise suppression, and spatial-temporal reasoning. To address these challenges, we introduce the Multi-Factor Adaptive vision Selection (MFAS) framework. MFAS integrates a patch partition and merging module for enhanced small object recognition, a prior-guided patch selection module for noise suppression and focused analysis, and a hierarchical aggregation network to aggregate visual semantics guided by questions. Extensive experiments on several public egocentric datasets have validated the effectiveness and generalization of our framework. Code and data are available in https://github.com/Hyu-Zhang/EgoVideoQA.
Cite
Text
Zhang et al. "Multi-Factor Adaptive Vision Selection for Egocentric Video Question Answering." International Conference on Machine Learning, 2024.Markdown
[Zhang et al. "Multi-Factor Adaptive Vision Selection for Egocentric Video Question Answering." International Conference on Machine Learning, 2024.](https://mlanthology.org/icml/2024/zhang2024icml-multifactor/)BibTeX
@inproceedings{zhang2024icml-multifactor,
title = {{Multi-Factor Adaptive Vision Selection for Egocentric Video Question Answering}},
author = {Zhang, Haoyu and Liu, Meng and Liu, Zixin and Song, Xuemeng and Wang, Yaowei and Nie, Liqiang},
booktitle = {International Conference on Machine Learning},
year = {2024},
pages = {59310-59328},
volume = {235},
url = {https://mlanthology.org/icml/2024/zhang2024icml-multifactor/}
}