ScreenAgent开源视觉语言模型代理，通过鼠标/键盘规划和执行控制计算机GUI。

4.4 (5)

审阅者 Daniel Nikulshyn·更新 2026年7月

Open Source VLM Agent GUI Automation Computer Use Research Mouse & Keyboard Multimodal Desktop

概览

ScreenAgent 是一个开源的视觉语言模型（VLM）代理，旨在通过鼠标和键盘操作控制计算机GUI。它启用了与实时计算机屏幕的交互，通过观察截图并执行操作。该项目包括了规划-执行-反思的流程，指导代理完成多步任务。它是为了解决教会代理使用计算机的挑战而创建的，需要能力如任务规划、图像理解和可视定位。 ScreenAgent 数据集，这个涵盖了各种每日计算机任务的数据集，是手动注释的，以支持代理的学习。项目由用于控制桌面、数据集、模型工人进行推理、以及培训代码组成。它支持基础鼠标和键盘操作，并且可以应用于不同的桌面操作系统和应用。 ScreenAgent 的方法是通用的，并且不依赖具体的 API，使其具有多样性。

主要功能

鼠标和键盘操作执行
规划-执行-反思流程完成任务
支持多种桌面操作系统和应用
手动注释ScreenAgent数据集支持多种任务
VLM代理用于GUI交互

价格

模型: Freemium
分类: AI Agent Development Frameworks
评分: 4.4 / 5 (5)

使用场景

自动化重复性桌面流程

使用视觉语言模型代理规划并执行鼠标和键盘操作，处理 GUI 应用中的多步任务，无需编写脚本进行交互。

视觉语言代理研究

利用开源代码库来研究、评估和扩展可察觉屏幕并操作计算机的视觉-语言模型代理

跨应用任务执行

将代理指示来规划和完成跨 GUI 程序的任务，通过屏幕理解来导航窗口和控件

可访问性和辅助控制

通过代理来驱动自然语言对计算机 GUI 的控制，使用户可以通过代理来执行操作，它会解释屏幕并采取行动

优点 & 缺点

优点

通用性跨不同桌面操作系统和应用
通过规划-执行-反思流程实现复合任务
支持基本鼠标和键盘操作进行GUI交互
手动数据集注释提供全面任务覆盖

缺点

需要VNC服务器或类似设置进行远程桌面连接
复合任务规划和图像理解能力
依赖于 ScreenAgent 数据集的质量和覆盖范围

评测

4.4

5 个评分的平均值。

登录以留下评测。

Margaret Whitfield

Apr 23, 2026

Skeptical, then convinced

I went in skeptical — most tools in this space overpromise. It actually delivers on the integrations, and it saves real time caught me off guard. The docs could be deeper is why this isn't a perfect score, still, I'd recommend giving it a real trial.

Camille Laurent

Mar 7, 2026

Does the job

Pretty happy overall. The automation just works and the value for money is strong. A few rough edges remain can be annoying, but no dealbreakers — I'd recommend it to a friend without hesitating.

Diego Fernández

Jan 23, 2026

Years in this space

I've evaluated a lot of these over the years. What stands out here is the core workflow — handled better than most — and support is responsive. Pricing gets steep at scale is my one real gripe. Worth the time if this is your use case.

Wei Chen

Jan 10, 2026

Compared a few options

Evaluated this against two competitors. Where it wins: the automation and it is genuinely easy to set up. Where it lags: pricing gets steep at scale. On balance the feature set — especially the onboarding — justifies the 4 stars for our use case.

Rina Desai

Jul 29, 2025

Use it every day

Honestly didn't expect to like it this much. The integrations is exactly what I needed, and the value for money is strong. but I reach for it almost every day now and it just clicks.

问答

What are typical use cases for ScreenAgent?

It is suited to GUI automation tasks where an agent needs to perceive the screen and act via mouse/keyboard—such as automating desktop workflows, testing applications, or building research prototypes for computer-use agents.

What is ScreenAgent and what can it do?

ScreenAgent is an open-source Visual Language Model (VLM) agent that controls computer GUIs. It plans and executes mouse and keyboard actions to automate on-screen tasks across desktop applications.

How much does ScreenAgent cost?

ScreenAgent is open-source, so the software itself is freely available. You may still incur costs for the underlying VLM (if using a paid model) and the hardware required to run it.