The most surprising thing about Ollama’s Modelfile system templates is that they don’t actually "template" anything in the way you’d expect from a programming language. Instead, they define a new model by composing existing ones and layering on specific instructions.
Let’s see this in action. Imagine you want a model specifically for writing Python code, but you want it to always use a specific set of imports and follow a certain style guide. You can create a Modelfile for this.
First, let’s look at a base model. We’ll use llama3 as an example.
ollama run llama3
>>> print("Hello, world!")
print("Hello, world!")
Now, let’s create a Modelfile to instruct llama3 to always include import os and import sys when generating Python code.
Create a file named Modelfile with the following content:
FROM llama3
SYSTEM """
You are a helpful AI assistant that writes Python code.
Always include the following imports at the beginning of your code:
import os
import sys
Ensure your code follows PEP 8 guidelines.
"""
Now, you can create a new model from this Modelfile:
ollama create python-coder -f Modelfile
This command tells Ollama to build a new model named python-coder using the instructions in Modelfile. The FROM llama3 line specifies that this new model is based on the existing llama3 model. The SYSTEM instruction provides the prompt that will be prepended to every user query.
Let’s test our new python-coder model:
ollama run python-coder
>>> write a python function to read a file line by line
import os
import sys
def read_file_line_by_line(filepath):
"""
Reads a file line by line and returns its content as a list of strings.
Args:
filepath (str): The path to the file.
Returns:
list: A list of strings, where each string is a line from the file.
Returns an empty list if the file cannot be read or is empty.
"""
lines = []
if os.path.exists(filepath):
try:
with open(filepath, 'r') as f:
lines = f.readlines()
except Exception as e:
print(f"Error reading file: {e}")
return lines
if __name__ == '__main__':
# Example usage:
# Create a dummy file for testing
dummy_filename = "my_test_file.txt"
with open(dummy_filename, "w") as f:
f.write("This is line 1.\n")
f.write("This is line 2.\n")
f.write("This is line 3.\n")
file_content = read_file_line_by_line(dummy_filename)
if file_content:
print(f"Content of {dummy_filename}:")
for line in file_content:
print(line.strip())
# Clean up the dummy file
if os.path.exists(dummy_filename):
os.remove(dummy_filename)
Notice how the output automatically includes import os and import sys at the beginning, even though we didn’t explicitly ask for them in our prompt to ollama run. This is because the SYSTEM instruction from our Modelfile is automatically applied to every interaction with the python-coder model.
The Modelfile system template is essentially a way to create specialized versions of existing models. You start with a base model (e.g., llama3, mistral, codellama) and then layer on custom instructions, context, or even prompt engineering techniques via the SYSTEM and PARAMETER directives. This allows you to tailor a model’s behavior without needing to fine-tune it, which is a much more involved process.
Here’s a breakdown of the key directives you’ll commonly use in a Modelfile:
FROM: Specifies the base model to use. This is mandatory.SYSTEM: Defines the system prompt that will be prepended to every user prompt. This is where you set the persona, rules, or context for the model.PARAMETER: Allows you to override or set specific generation parameters for the model. For example, you can settemperatureto0.1for more deterministic output ortop_pto0.9.TEMPLATE: Defines the prompt template for chat models. This is more advanced and allows you to control how user and system messages are formatted.
The SYSTEM directive is where the magic of custom prompts truly lies. When you define a SYSTEM prompt in your Modelfile, Ollama injects that text before your actual user prompt when sending it to the underlying base model. This means the base model always receives your custom instructions as part of its context.
A crucial detail often overlooked is how the SYSTEM prompt interacts with subsequent PARAMETER directives within the same Modelfile. If you set a parameter like temperature in the SYSTEM section and then again in a PARAMETER section, the last definition for that parameter will take precedence. Ollama processes the Modelfile sequentially.
The next step after mastering custom system prompts is exploring how to combine multiple base models or even use TEMPLATE directives for more intricate chat-like interactions.