Ollama 发布 Secure Minions，实现本地模型与前沿模型的加密协作

June 3, 2025

Three months ago, Stanford’s Hazy Research lab introduced Minions (ICML 2025), an open-source research project that connects local Ollama models (such as Google’s gemma3:4b) to frontier models in the cloud (such as GPT-4o). In Minions, the raw context stays local and can only be accessed by the local LLM. The frontier model orchestrates local LLMs and aggregates their outputs. By sending fewer tokens to the cloud, the protocol reduces cloud costs by 5x-30x while achieving 98% of frontier model accuracy.

Cost savings aside, local-first setups (like Ollama) have a big privacy upside: sensitive context never leaves the device. But in the original Minions protocol, some information still goes to the cloud, and that information can be sensitive.

Avanika Narayan and Dan Biderman from Stanford’s Hazy Research lab then asked: can we encrypt an entire local-remote communication protocol end-to-end? Even from the cloud provider itself?

Their team built a comprehensive security protocol that builds around the new “confidential computing mode” introduced with NVIDIA’s Hopper H100 GPUs.

Here’s how it works:

The local device and the H100 GPU exchange keys; The GPU proves it’s genuine and running in secure mode via remote attestation; Once verified, the H100 becomes a secure enclave: all memory and computation are encrypted, and even root users can’t access plaintext; The local LLM messages are encrypted before being sent to the GPU enclave, where they’re safely decrypted and processed by the cloud LLM. Its outputs are again encrypted before being sent back to the local client

No plaintext is exposed - during transmission or remote LLM inference.

Even with long prompts (~8k tokens) and large models like Qwen-32B, the overhead is minimal: less than 1% added latency.

Confidential LLM collaboration is no longer theoretical — it’s here!

For full technical details, see the HazyResearch blog post.

Clone the repository:

Optionally, create a virtual environment with your favorite package manager (e.g. conda, venv, uv, etc.):

Next, install the Python package and dependencies:

If you don’t have it yet, install Ollama and Google’s Gemma 3 model:

Running the secure protocol in app

The provided streamlit app runs an interactive demo of both the Minion and MinionS protocols. To start it, run:

A browser window will open. Select your Remote Provider to be “Secure”. Set the Secure Endpoint URL to http://20.57.33.122:5056. Set your local client to Ollama and select the model you would like to use.

Example code

To run the secure protocol programmatically in Python, the minions package can be used.

First create a file named example.py and add the following contents:

Run the example:

Minions GitHub repository; Hazy Research blog post