Offline Reinforcement Learning for Community-Acquired Pneumonia Management: A Feasibility Study

Abstract

Community-acquired pneumonia (CAP) remains a leading cause of hospital admission and mortality requiring dynamic clinical decision making as patients’ conditions evolve. In this work, we formulate the management of CAP as a sequential decision-making problem and utilise reinforcement learning (RL) as a framework for discovering improved treatment strategies. We leverage a large-scale repository of routinely collected hospital data from the PIONEER hub and conduct an offline RL investigation under real-world complexities such as irregular sampling, missingness and variable treatment patterns. Through an extensive data transformation pipeline, we construct state-action trajectories suitable for RL and then train and evaluate policies via conservative Q-learning and fitted Q-evaluation, achieving initial—though modest—improvements in reducing 30-day mortality. In addition to these preliminary outcomes, our findings underscore the need for refined offline RL methods and rigorous validation to fully realize the potential of using large routine healthcare databases like PIONEER for clinical decision support.