ManifestReal-time cost observability and routing for AI agents and applications, enabling multi-provider LLM inference optimization.

Manifest is an open-source platform designed to help users manage and optimize their AI inference costs by providing a routing layer between AI agents or applications and various large language model (LLM) providers. It addresses the challenge of high AI bills and the complexity of efficiently using multiple LLM services by putting users in control of their model consumption and expenditure. The tool functions by allowing users to connect their autonomous agents, applications, or third-party harnesses to Manifest. They then add their preferred LLM providers, which can include API key-based services (like OpenAI, Anthropic, Mistral), existing monthly subscriptions (e.g., Anthropic, GitHub Copilot), custom OpenAI- or Anthropic-compatible endpoints, and even local models running on personal infrastructure via Ollama, LM Studio, or llama.cpp. Once connected, Manifest enables users to define routing rules, select specific models and providers for different queries, and set up fallbacks. This allows for dynamic model selection based on cost, performance, or availability. For instance, it can prioritize using quotas from a pre-paid subscription and automatically fall back to pay-as-you-go models when limits are exceeded. The platform also offers real-time visualization of spending, helping users track every dollar spent across their AI operations. A standout capability is Manifest's "AUTO-FIX" feature, which attempts to remediate common LLM request failures before they reach the agent. This includes fixing issues like deprecated or not-found models, wrong parameters, malformed requests, and exceeded context windows, aiming to prevent downtime and improve request success rates. Manifest is built with flexibility in mind, supporting a wide array of AI applications, personal agents, and workflows. It is available as a cloud version for ease of onboarding or a self-hosted Docker deployment, reflecting its open-source nature. This approach aims to make AI more affordable and accessible, from individual developers to established enterprises, by offering tools to reduce costs without compromising quality or locking users into a single provider.