Ollama Function Calling: Tool Use in Local LLMs (2026)

Function calling in local LLMs, particularly with Ollama, isn’t about the LLM executing code; it’s about the LLM describing what code it wants to execute and what arguments it needs.

Let’s see it in action. Imagine you have a local LLM running via Ollama, and you want it to tell you the current weather. You’d define a "tool" (a function) that your application can actually run.

{
  "name": "get_current_weather",
  "description": "Get the current weather in a given location",
  "parameters": {
    "type": "object",
    "properties": {
      "location": {
        "type": "string",
        "description": "The city and state, e.g. San Francisco, CA"
      },
      "unit": {
        "type": "string",
        "enum": ["celsius", "fahrenheit"],
        "description": "The temperature unit to use"
      }
    },
    "required": ["location"]
  }
}

Now, you send a prompt to Ollama like: "What’s the weather like in Boston, MA?" along with this tool definition. The LLM, instead of trying to guess the weather, will respond with a JSON object that looks something like this:

{
  "tool_calls": [
    {
      "id": "call_abc123",
      "type": "function",
      "function": {
        "name": "get_current_weather",
        "arguments": "{\"location\": \"Boston, MA\", \"unit\": \"fahrenheit\"}"
      }
    }
  ]
}

Your application receives this. It sees tool_calls, extracts the name (get_current_weather) and arguments ({"location": "Boston, MA", "unit": "fahrenheit"}). It then actually runs your get_current_weather function with those parameters. Let’s say your function fetches this data and returns {"temperature": "72", "unit": "fahrenheit"}.

You then send that result back to Ollama in a subsequent API call, like this:

{
  "messages": [
    {"role": "user", "content": "What's the weather like in Boston, MA?"},
    {
      "role": "assistant",
      "content": null,
      "tool_calls": [
        {
          "id": "call_abc123",
          "type": "function",
          "function": {
            "name": "get_current_weather",
            "arguments": "{\"location\": \"Boston, MA\", \"unit\": \"fahrenheit\"}"
          }
        }
      ]
    },
    {
      "role": "tool",
      "tool_call_id": "call_abc123",
      "content": "{\"temperature\": \"72\", \"unit\": \"fahrenheit\"}"
    }
  ],
  "tools": [ /* ... same tool definition as before ... */ ]
}

Ollama now uses the result of your tool execution to formulate a natural language response to the user, such as: "The current weather in Boston, MA is 72 degrees Fahrenheit."

The core problem this solves is grounding LLM responses in real-world, executable actions. LLMs are great at understanding intent and generating text, but they can’t directly interact with your systems, databases, or the internet. Function calling provides a structured bridge. You define the capabilities your application has, and the LLM learns to request those capabilities with the correct parameters.

Internally, Ollama (and other LLM providers supporting this) uses a fine-tuned model. This fine-tuning teaches the model to recognize when a user’s request can be fulfilled by one of the provided tools and to output the structured tool_calls JSON when that’s the case. The enum in the tool definition is crucial; it constrains the LLM’s output for that parameter to only the specified values, preventing it from hallucinating units like "kelvin" if you only support "celsius" and "fahrenheit."

The "tool use" isn’t a single LLM call; it’s a conversational turn. The LLM first suggests the tool call, your application executes it, and then you feed the result back to the LLM for it to finalize the user-facing answer. This multi-turn interaction is key.

A subtle but critical aspect of function calling is how the LLM handles ambiguity or missing information. If you ask "What’s the weather?" without specifying a location, the LLM, if prompted correctly and given the tool definition, might respond with a request for clarification using the tool’s parameter schema. For instance, it might output:

{
  "tool_calls": [
    {
      "id": "call_xyz789",
      "type": "function",
      "function": {
        "name": "get_current_weather",
        "arguments": "{\"location\": null}"
      }
    }
  ]
}

Or, more typically, the LLM might simply generate a natural language question like, "Sure, which location would you like the weather for?" This behavior is influenced by the model’s training and how you structure the initial prompt and the tool definitions. The key is that the LLM is guided to identify what information is needed based on the tool’s required parameters.

The next step is understanding how to chain multiple tool calls together to accomplish more complex tasks.