AoI-MDP: An AoI Optimized Markov Decision Process Dedicated in the Underwater Task (Student Abstract)

Yimian Ding, Jingzehua Xu, Yiyuan Yang, Guanwen Xie, Xinqi Wang, Shuai Zhang

AAAI 2025 pp. 29348-29350

doi:10.1609/AAAI.V39I28.35247 /aaai/2025/ding2025aaai-aoi/

Abstract

Ocean exploration places high demands on autonomous underwater vehicles, especially when there's observation delay. We propose age of information optimized Markov decision process (AoI-MDP) to enhance underwater tasks by modeling observation delay as signal delay and including it in the state space. AoI-MDP also introduces wait time in the action space and integrates AoI with reward functions, optimizing information freshness and decision-making using reinforcement learning. Simulations show AoI-MDP outperforms the standard MDP, demonstrating superior performance, feasibility, and generalization in underwater tasks. To accelerate relevant research, we have made the codes available as open-source at https://github.com/Xiboxtg/AoI-MDP.

AAAI Semantic Scholar

Cite

Text

Ding et al. "AoI-MDP: An AoI Optimized Markov Decision Process Dedicated in the Underwater Task (Student Abstract)." AAAI Conference on Artificial Intelligence, 2025. doi:10.1609/AAAI.V39I28.35247

Markdown

[Ding et al. "AoI-MDP: An AoI Optimized Markov Decision Process Dedicated in the Underwater Task (Student Abstract)." AAAI Conference on Artificial Intelligence, 2025.](https://mlanthology.org/aaai/2025/ding2025aaai-aoi/) doi:10.1609/AAAI.V39I28.35247

BibTeX

@inproceedings{ding2025aaai-aoi,
  title     = {{AoI-MDP: An AoI Optimized Markov Decision Process Dedicated in the Underwater Task (Student Abstract)}},
  author    = {Ding, Yimian and Xu, Jingzehua and Yang, Yiyuan and Xie, Guanwen and Wang, Xinqi and Zhang, Shuai},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year      = {2025},
  pages     = {29348-29350},
  doi       = {10.1609/AAAI.V39I28.35247},
  url       = {https://mlanthology.org/aaai/2025/ding2025aaai-aoi/}
}