Ollama 支持嵌入模型，可构建 RAG 应用

April 8, 2024

Ollama supports embedding models, making it possible to build retrieval augmented generation (RAG) applications that combine text prompts with existing documents or other data.

What are embedding models?

Embedding models are models that are trained specifically to generate vector embeddings: long arrays of numbers that represent semantic meaning for a given sequence of text:

The resulting vector embedding arrays can then be stored in a database, which will compare them as a way to search for data that is similar in meaning.

Example embedding models

Usage

To generate vector embeddings, first pull a model:

Next, use the REST API, Python or JavaScript libraries to generate vector embeddings from the model:

Ollama also integrates with popular tooling to support embeddings workflows such as LangChain and LlamaIndex.

Example

This example walks through building a retrieval augmented generation (RAG) application using Ollama and embedding models.

Step 1: Generate embeddings

Create a file named example.py with the contents:

Step 2: Retrieve

Next, add the code to retrieve the most relevant document given an example prompt:

Step 3: Generate

Lastly, use the prompt and the document retrieved in the previous step to generate an answer!

Then, run the code:

Llama 2 will answer the prompt What animals are llamas related to? using the data:

Coming soon

More features are coming to support workflows that involve embeddings:

Batch embeddings: processing multiple input data prompts simultaneously; OpenAI API Compatibility: support for the /v1/embeddings OpenAI-compatible endpoint; More embedding model architectures: support for ColBERT, RoBERTa, and other embedding model architectures