Ollama.js: Integrate Local LLMs in Node.js Apps (2026)

Ollama.js is a Node.js library that lets you run large language models (LLMs) locally on your machine and integrate them into your applications.

Imagine you’re building a Node.js app, maybe a chatbot or a content generator, and you want to leverage the power of LLMs without sending your data to a cloud API. That’s where Ollama.js comes in. It acts as a bridge, allowing your Node.js code to communicate with the Ollama runtime, which in turn manages and runs your downloaded LLMs.

Here’s a quick look at how it works in practice. First, you need Ollama installed and running on your system. You can download it from ollama.com. Once installed, you can pull an LLM, for example, Llama 3:

ollama pull llama3

Now, in your Node.js project, you’ll install the ollama package:

npm install ollama

Then, you can start interacting with the model:

import ollama from 'ollama';

async function generateText() {
  const response = await ollama.chat({
    model: 'llama3',
    messages: [{ role: 'user', content: 'Why is the sky blue?' }],
  });
  console.log(response.message.content);
}

generateText();

This code snippet does a few things:

1. It imports the ollama library.
2. It defines an asynchronous function generateText.
3. Inside the function, it calls ollama.chat, specifying the llama3 model and a user message.
4. The response object contains the model’s reply, which is then printed to the console.

The core problem Ollama.js solves is democratizing LLM access. Instead of relying on expensive, rate-limited, and privacy-concerned cloud services, you can run powerful models on your own hardware. This is crucial for applications dealing with sensitive data, requiring offline functionality, or needing fine-grained control over model behavior and costs.

Internally, Ollama.js doesn’t run the LLM itself. Instead, it communicates with the Ollama server process that you run separately. Ollama handles the heavy lifting: downloading models, managing their execution (often leveraging GPU acceleration), and exposing an API. Ollama.js simply makes calls to this local Ollama API using a familiar Node.js interface. The library handles the serialization and deserialization of requests and responses, abstracting away the HTTP communication.

The primary levers you control are the model you choose, the messages you send (including system prompts for guiding behavior), and various parameters that influence the generation process. These parameters, passed within the options object of the ollama.chat or ollama.generate methods, allow you to tune the output. For instance:

• temperature: Controls randomness. Higher values (e.g., 0.8) make output more creative, while lower values (e.g., 0.2) make it more deterministic.
• top_k: Limits the number of highly probable next tokens to consider.
• top_p: Nucleus sampling, another way to control randomness by considering tokens that cumulatively make up a certain probability mass.
• num_predict: Sets a maximum number of tokens to generate.

Here’s an example with some of these options:

import ollama from 'ollama';

async function generateCreativeText() {
  const response = await ollama.chat({
    model: 'llama3',
    messages: [{ role: 'user', content: 'Write a short, whimsical poem about a teapot.' }],
    options: {
      temperature: 0.9,
      num_predict: 100,
      top_k: 40,
    },
  });
  console.log(response.message.content);
}

generateCreativeText();

This allows you to experiment and find the sweet spot for your application’s needs, whether it’s factual accuracy or imaginative storytelling.

A common point of confusion is understanding the difference between the Ollama server and the Ollama.js library. The library is just a client. If the Ollama server isn’t running or isn’t accessible, the library calls will fail, even if the library itself is installed correctly. You might see errors related to connection refused or timeouts, indicating the Node.js process can’t reach the Ollama API endpoint, which defaults to http://localhost:11434.

Beyond simple chat completions, Ollama.js also supports streaming responses, which is essential for interactive applications like chatbots where you want to see text appear as it’s generated, rather than waiting for the entire response. You enable this by setting stream: true in the options. The library then yields chunks of the response as they become available.

import ollama from 'ollama';

async function streamResponse() {
  const stream = await ollama.chat({
    model: 'llama3',
    messages: [{ role: 'user', content: 'Explain quantum entanglement in simple terms.' }],
    stream: true,
  });

  for await (const chunk of stream) {
    process.stdout.write(chunk.message.content);
  }
  console.log('\n--- End of stream ---');
}

streamResponse();

This for await...of loop elegantly handles the streaming chunks, printing them directly to standard output as they arrive.

The next step in integrating LLMs locally is exploring model quantization and fine-tuning.