Ollama 内置兼容 OpenAI Chat Completions API
OpenAI compatibility · Ollama Blog
Ollama 宣布内置兼容 OpenAI Chat Completions API，可在本地使用更多工具和应用。使用时只需把请求的 hostname 改为 http://localhost:11434，即可沿用 OpenAI 的调用格式，官方还给出了 cURL 和 Microsoft 开源多智能体框架 Autogen 的接入示例。该支持目前为实验性，后续改进方向仍在考虑中。
推荐理由
AI 评分 54
Ollama 官方说明其本地推理服务已兼容 OpenAI Chat Completions API，读者可据此判断现有工具链能否直接迁移到本地模型。
主题
正文
当前显示原文，中文全文译文尚未就绪。
February 8, 2024
Ollama now has built-in compatibility with the OpenAI Chat Completions API, making it possible to use more tooling and applications with Ollama locally.
Setup
Start by downloading Ollama and pulling a model such as Llama 2 or Mistral:
Usage
cURL
To invoke Ollama’s OpenAI compatible API endpoint, use the same OpenAI format and change the hostname to http://localhost:11434:
Then make the following two edits in app/api/chat/route.ts to update the chat example to use Ollama:
Next, run the app:
Finally, open the example app in your browser at http://localhost:3000:
Autogen
Autogen is a popular open-source framework by Microsoft for building multi-agent applications. For this, example we’ll use the Code Llama model:
Install Autogen:
Then create a Python script example.py to use Ollama with Autogen:
Lastly, run the example to have the assistant write the code to plot a chart:
More to come
This is initial experimental support for the OpenAI API. Future improvements under consideration include:
GitHub issues are welcome! For more information, see the OpenAI compatibility docs.