Self-hosted models that keep data private and costs fixed.
Paul Scanlon·Nov 12, 2026
·3 min read
You can now route agent requests to self-hosted open-weight models using a custom model gateway. Give the gateway a custom ID, then use it as the model prefix in your agent's config: private/ollama/gpt-oss:20b.
Using a hosted provider service can complicate HIPAA or GDPR compliance (unless data retention agreements are in place). Self-hosting removes the third party, preventing sensitive data from reaching models outside of your network.
Costs may also be reduced as you're only paying for infrastructure, not tokens, and you control which models your agents can use.
To hide external providers from Studio's model picker, set AUTO_BLOCK_EXTERNAL_PROVIDERS=true in your .env so users can only select models from your own gateways.
Get started
Install the latest @mastra/core and an AI SDK provider for your gateway's API:
npm install @mastra/core@latest @ai-sdk/openai-compatible@3.0.84@mastra/core@1.82.0 or later, added in PR #18153.
Run a local model
This example uses Ollama and gpt-oss:20b. Running ollama serve starts the server, with an OpenAI-compatible API at http://localhost:11434/v1:
brew install ollama
ollama pull gpt-oss:20b
ollama serveCreate a gateway
Create a gateway that sends every model call to the local Ollama server:
id: The model ID prefix, e.g.privateinprivate/ollama/gpt-oss:20b.name: The gateway's display name.fetchProviders(): Returns the providers and models the gateway offers, listed in Studio's model picker.buildUrl(): Returns the base URL for model calls.getApiKey(): Returns the API key to send with each call (not required for local models).resolveLanguageModel(): Creates the language model for each call.
import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
import type { MastraModelGatewayInterface } from "@mastra/core/llm";
export const privateGateway: MastraModelGatewayInterface = {
id: "private",
name: "Private Gateway",
fetchProviders: async () => {
return {
ollama: {
name: "Ollama",
models: ["gpt-oss:20b"],
apiKeyEnvVar: [],
gateway: "private",
url: "http://localhost:11434/v1"
}
};
},
buildUrl: () => {
return "http://localhost:11434/v1";
},
getApiKey: async () => {
return "";
},
resolveLanguageModel: ({ modelId, providerId, apiKey }) => {
return createOpenAICompatible({
name: providerId,
apiKey,
baseURL: "http://localhost:11434/v1"
}).chatModel(modelId);
}
};Register the gateway
Add the gateway to your Mastra instance:
export const mastra = new Mastra({
// ...
gateways: { privateGateway }
});Use the model
Reference the model by [gatewayId]/[provider]/[model]:
export const researchAgent = new Agent({
id: "research-agent",
name: "Research Agent",
instructions: /* ... */,
model: "private/ollama/gpt-oss:20b"
});Hide external providers in Studio
Add the variable to your .env:
AUTO_BLOCK_EXTERNAL_PROVIDERS=trueFor more information and full configuration options, see:
- Custom model gateways
- When to use custom gateways
- Showing only custom gateways in Studio
- Gateway-owned authentication
- Model pricing
Paul ScanlonTechnical Product Marketing ManagerPaul Scanlon sits between Developer Education and Product Marketing at Mastra. Previously, he was a Technical Product Marketing Manager at Neon and worked in Developer Relations at Gatsby, where he created educational content and developer experiences.
All articles by Paul Scanlon →More on Models & Routing
All postsBlogAug 18, 2026AI SDK v7 support in Mastra
Image generation, reasoning control, multimodal prompts, stream handlers, and UI helpers.
ArticleAug 16, 2026LLM vs generative AI: key differences and how to choose
LLM vs generative AI explained: how they differ in scope, architecture, and use cases, plus how to choose the right approach for your app.
ArticleAug 15, 2026Generative AI examples: real-world uses by industry
See generative AI examples across healthcare, finance, retail, and software, plus how developers build, evaluate, and ship gen AI apps.