斯坦福 Hazy Research 提出 Minions 协议，让本地小模型与云端大模型协作
Minions: where local and cloud LLMs meet · Ollama Blog
斯坦福 Hazy Research 实验室等研究者提出 Minions 方法，让 Llama 3.2 等本地小模型与 GPT-4o 等云端大模型协作，把部分 LLM 负载转移到消费级设备，以降低云端成本。其中 Minion 协议让云端模型与单个本地模型自由对话直至达成结果，远程成本降低 30.4 倍，同时保留 87% 的云端模型性能；MinionS 协议由云端模型把任务拆成小块子任务、由小模型并行处理，远程成本降低 5.7 倍，保留 97.9% 的性能。论文附带开源代码，可通过 Ollama 和 OpenAI API Key 运行 demo。
推荐理由
精选AI 评分 60
论文给出本地小模型与云端大模型协作的两种协议，并附开源代码，可据此评估端云协同的成本与质量取舍。
主题
正文
当前显示原文，中文全文译文尚未就绪。
February 25, 2025
Avanika Narayan, Dan Biderman, and Sabri Eyuboglu from Christopher Ré’s Stanford Hazy Research lab, along with Avner May, Scott Linderman, James Zou, have developed a way to shift a substantial portion of LLM workloads to consumer devices by having small on-device models (such as Llama 3.2 with Ollama) collaborate with larger models in the cloud (such as GPT-4o).
This new paper with accompanying open-source code aims to reduce cloud costs with minimal or no quality degradation in two protocol configurations:
Minion: cloud model freely chats with a single local model with access to data, until the two reach a solution
Achieves 30.4x reduction in remote costs, while maintaining 87% of cloud model performance; MinionS: cloud model decomposes the task into bit-sized subtasks to be performed on chunks of the context. Small LLMs solve these tasks in parallel
Achieves 5.7x reduction in remote costs, while maintaining 97.9% of cloud model performance
Clone the repository:
Optionally, create a virtual environment with your favorite package manager (e.g. conda, venv, uv, etc.):
Next, install the Python package and dependencies:
If you don’t have it yet, install Ollama and Meta’s Llama 3.2 model:
Lastly, create an OpenAI API Key for the cloud model.
Running the demo app
The provided streamlit app runs an interactive demo of both the Minion and MinionS protocols. To start it, run:
A browser window will open with instructions on entering your OpenAI API key, selecting a local model, and running either Minion or MinionS:
Example code
To run Minion or MinionS programmatically with Python, the minions package can be used.
Minion
First create a file named example.py and add the following contents:
Then run the example:
MinionS
With a few modifications, the same code can be used to run the MinionS protocol:
After making the modifications, re-run the example:
Minions GitHub repository; Blog post; Paper