Agent SOpen-source GUI agent framework that lets an LLM use your computer like a human via an Agent-Computer Interface.

Agent S is an open-source GUI agent framework that enables large language models (LLMs) to interact with computers like humans via an Agent-Computer Interface. The framework allows LLMs to learn from past experiences and perform complex tasks autonomously on a computer. Agent S is designed for single-monitor screens and supports Linux, Mac, and Windows platforms. The framework has achieved state-of-the-art results on various benchmarks, including OSWorld, WindowsAgentArena, and AndroidWorld. Agent S3, the latest version, has surpassed human-level performance on OSWorld with a score of 72.60%. It has also demonstrated strong zero-shot generalization capabilities. Agent S provides a flexible and modular architecture for building GUI agents. The framework includes a library called gui-agents, which allows users to easily integrate Agent S with their applications. The library supports multiple platforms and provides a simple installation process. The development of Agent S is focused on advancing the capabilities of autonomous GUI agents. The framework has the potential to be used in various applications, including automation, AI research, and computer vision. However, users should exercise caution when running Agent S, as it controls the computer by running Python code.