概览
主要功能
- 应用内模型浏览器和下载器
- 任何可安装模型的本地聊天界面
- 兼容 OpenAI API 的本地服务
- GPU 加速和可配置的 inference 设置
- GGUF 和 MLX 模型格式支持
- 本地检索支持
价格
- 模型
- Free
- 评分
- 4.3 / 5 (6)
使用场景
私人离线 AI 聊天
与 Llama、Mistral 或 Qwen 等开源 LLM 进行完全的私人聊天,保持对话和敏感数据的完整隐私,根本不需要依赖云服务
本地兼容 OpenAI API
建立一个本地的服务,模仿 OpenAI API 来为流程和应用程序提供基于本地模型的机器人服务,仅需替换现有的云端点就能继续使用
在敏感文档中实施 Q&A
使用内置的 Document Q&A 来本地询问私有文件,非常适合无法将敏感材料上传到外部服务的研究人员或专业人员
在开源模型中进行试验
浏览、下载和测试 Hugging Face 的 GGUF 和 MLX 模型范围,并具有可配置的 GPU 加速,适合开发人员评估模型表现
优点 & 缺点
优点
- 用户设备上全线操作,保持数据隐私
- 免费用于个人用途,无需创建账户
- 支持多个支持 GGUF 格式的开源模型
- 包括兼容 OpenAI API 的本地服务
- 支持多个平台
- 本地 GPU 加速
缺点
- 硬件性能对线性能有很大影响
- 庞大的模型需要大量的 RAM 和磁盘空间
- 没有上游的商业云模型的功能
- 商业使用可能需要单独的许可证
评测
6 个评分的平均值。
登录以留下评测。
Solid for our team
We rolled this out across the team last quarter and cross-platform support for Windows, macOS, and Linux. Local chat interface for any installed model fits neatly into how we already work, and gPU acceleration and configurable inference settings removed a step we used to do by hand. Large models require significant RAM and disk space, which is the main caveat, but it has held up under daily use.
Compared a few options
Evaluated this against two competitors. Where it wins: document chat with local retrieval and supports many open-source models in GGUF format. On balance the feature set — especially local chat interface for any installed model — justifies the 5 stars for our use case.
Does the job
Pretty happy overall. OpenAI-compatible local API server just works and includes a local OpenAI-compatible API server. but no dealbreakers — I'd recommend it to a friend without hesitating.
Use it every day
Honestly didn't expect to like it this much. Document chat with local retrieval is exactly what I needed, and includes a local OpenAI-compatible API server. I do wish performance depends heavily on local hardware, but I reach for it almost every day now and it just clicks.
Use it every day
Honestly didn't expect to like it this much. GPU acceleration and configurable inference settings is exactly what I needed, and free for personal use with no account required. I do wish performance depends heavily on local hardware, but I reach for it almost every day now and it just clicks.
Solid for our team
We rolled this out across the team last quarter and fully offline, keeping data private on the user's device. GPU acceleration and configurable inference settings fits neatly into how we already work, and support for GGUF and MLX model formats removed a step we used to do by hand. Commercial use may require a separate license, which is the main caveat, but it has held up under daily use.
问答
暂无问题 — 来当第一个提问的人吧。
提问
Model Serving 的替代品
APIPASS API Marketplace
Model Serving
统一的市场,让您通过单一集成点连接多个 API。
Fast360
Model Serving
开源平台,用于对 PDF 转 Markdown 转换的 OCR 模型进行基准测试
LlamaCloud
Model Serving
完善的文档解析和索引平台,用于建立准确的RAG和代理工作流。
Eidolon AI
Model Serving
开源框架,快速构建和部署企业 AI 代理。
E2B
Model Serving
安全的云沙箱,运行 AI 生成代码与自主代理
FloppyData
Model Serving
高速住宅和移动代理,适用于网页抓取和数据采集
Groq
Model Serving
专注于高性能AI推理解决方案,提供硬件和软件平台用于快速AI应用部署。









