Risks and Safety Considerations for Foundation Model-Based Autonomous Agents' Interaction with the Environment

Abstract

Foundation Model (FM) agents are increasingly deployed across diverse environments, from web automation to physical and medical systems. While their ability to interact autonomously enhances efficiency, it also introduces significant safety risks, including unauthorized access, data breaches, and system disruptions. Existing research on FM agent safety remains fragmented, lacking a comprehensive classification of risks across different domains. This paper addresses this gap by systematically categorizing risks into web, computer, and physical domains and proposing targeted mitigation strategies. Our framework aids researchers, developers, and policymakers in designing safer FM systems and establishing regulatory guidelines. By highlighting potential hazards and preventive measures, this work contributes to ensuring that FM agents operate securely while maximizing their transformative potential.