Best AR/VR (2026)

1

Sima

Generalist AI agent that follows natural language instructions inside 3D virtual environments.

4.8 (4)

· freemium

Full review

Sima (Scalable Instructable Multiworld Agent) is a research-grade AI agent designed to operate across a wide range of 3D virtual environments, including commercial video games and research simulators. Rather than being trained for a single title, it learns general skills that transfer between worlds by mapping natural language instructions to keyboard and mouse actions, just as a human player would. Developed as part of efforts to build more capable embodied agents, Sima focuses on grounded instruction following: a user types a command such as 'turn left', 'climb the ladder', or 'collect the resource', and the agent attempts to carry it out using only on-screen visual input. This makes it a testbed for studying how language, perception, and action can be combined in complex, open-ended 3D worlds. Sima is primarily a research project rather than a consumer product, and is most relevant to AI researchers, game developers, and teams exploring embodied agents, simulation-based training, and human-AI interaction in interactive environments.

• Generalist agent across multiple 3D environments
• Natural language instruction following
• Vision-based perception of the game screen
• Keyboard and mouse action output
• Transfer of skills between different worlds
• Research-oriented benchmarking across game tasks