Ollama 博客：如何为 Code Llama 编写提示词

September 9, 2023

Two weeks ago the Code Llama model was released by Meta with three variations:

Instruct; Code completion; Python

This guide walks through the different ways to structure prompts for Code Llama for its different variations and features.

Examples below use the 7 billion parameter model with 4-bit quantization, but 13 billion and 34 billion parameter models were made available as well. This guide uses the open-source Ollama project to download and prompt Code Llama, but these prompts will work in other model providers and runtimes too.

Instruct

The instruct model was trained to output human-like answers to questions. This model variation is the easiest to use and will behave closest to ChatGPT, with answer questions including both natural language and code:

Code reviews can often be time consuming for teams. Code Llama can help by providing a second set of eyes on a commit or pull request for simple bugs:

Writing unit tests often requires quite a bit of boilerplate code. Code Llama can help:

The standard code completion model will simply continue to generate subsequent tokens based on the provided prompt. This variation works best if you’re only looking to generate code, such as having the model write a function after seeing the function signature or comment:

Fill-in-the-middle (FIM), or more briefly, infill is a special prompt format supported by the code completion model can complete code between two already written code blocks. This is especially effective for inserting code within an existing function such as below:

With Code Llama, infill prompts require a special format that the model expects.

To use this with existing code, split the code before and after in the example above the into parts: the prefix, and the suffix. For example, for our LCM example above:

Prompt

While results will vary, you should get something like this:

Response

Note: the model may return at the end of the result. This is a special token in the response that represents the end of the response similar to , and

Python

As a thank you to the community and tooling that created the model, the authors of Code Llama included a Python variation which is fine-tuned on 100B additional Python tokens, making it a good model to use when working on machine learning-related tooling, or any other Python code:

Cody has an experimental version that uses Code Llama with infill support; Continue supports Code Llama as a drop-in replacement for GPT-4; Fine-tuned versions of Code Llama from the Phind and WizardLM teams; Open interpreter can use Code Llama to generate functions that are then run locally in the terminal