How do you customize a local Ollama model?

You write a Modelfile — a plain text file starting with a FROM instruction naming the base model, followed by PARAMETER lines (like temperature or num_ctx) and an optional SYSTEM instruction defining a persona. Running "ollama create -f Modelfile" builds a new, customized model from it.

← Back to the Introduction level

AI & ML — INTERMEDIATE

Ollama for Engineers: Modelfiles, Parameters & the REST API

“Download and run a model” is the whole story at the Introduction level. Actually building something with Ollama means customizing that model’s behavior and calling it from your own code — both covered here.

The Quick Answer

Ollama models get customized through a Modelfile — a small text file that names a base model, sets parameters like temperature and context size, and optionally defines a system persona. Once built, that model is called either from the terminal or, for real applications, through Ollama’s local REST API at http://localhost:11434 — the same server your terminal was talking to all along.

The Modelfile, Instruction by Instruction

A Modelfile is plain text, and Ollama’s own documented example covers the three instructions you’ll use most:

FROM llama3.2
# sets the temperature to 1 [higher is more creative, lower is more coherent]
PARAMETER temperature 1
# sets the context window size to 4096, this controls how many tokens the
# LLM can use as context to generate the next token
PARAMETER num_ctx 4096

# sets a custom system message to specify the behavior of the chat assistant
SYSTEM You are Mario from super mario bros, acting as an assistant.

FROMnames the existing model this new one is built on top of — you’re customizing behavior, not training a model from scratch. PARAMETER lines set runtime settings; num_ctx in particular is the exact same context window concept from earlier in this series, just exposed as a setting you control. SYSTEM is a fixed instruction baked into the model itself, so every conversation starts with that persona already in place.

Building and running it from that file:

ollama create mario-assistant -f ./Modelfile
ollama run mario-assistant

Source: Ollama, “Modelfile Reference”

Try building one yourself below — pick a temperature and a persona, and see the exact Modelfile that combination produces.

Calling Ollama From Code

The ollama run command is really just a terminal client talking to a local server. That same server exposes a REST API on localhost:11434, which is what any real application actually calls:

curl http://localhost:11434/api/generate -d '{
  "model": "llama3.2",
  "prompt": "What is the capital of France?",
  "stream": false
}'

By default, stream is true, and Ollama sends back one JSON object per generated token as a newline-delimited stream — the same token-by-token generation covered in How LLMs Work, just visible directly in the API response instead of hidden behind a chat UI. Setting "stream": false, as above, waits for generation to finish and returns the complete response in one JSON object instead.

Fun Fact

A Modelfile isn’t case-sensitive, and its instructions can technically appear in any order — the convention of putting FROM first is just for human readability, not a requirement Ollama enforces.

Test Yourself

What does the FROM instruction in a Modelfile specify?

What does the PARAMETER num_ctx instruction control?

What command builds a model from a Modelfile?

What is the default Ollama REST API endpoint for generating a response?

What happens if you send a request to /api/generate without setting "stream": false?

Ready for production concerns — handling concurrent requests, and why GPU offloading can make or break your throughput? Continue to the Advanced level →