LangChain doesn’t actually run your LLM calls; it just orchestrates them, which is why you can swap out OpenAI for Anthropic or even a local Llama 2 model with minimal code changes.
Here’s a simple chain in action:
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
# 1. Initialize the LLM
llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0.7)
# 2. Define the prompt template
prompt = ChatPromptTemplate.from_messages([
("system", "You are a helpful assistant. Translate the user's text to French."),
("user", "{text}")
])
# 3. Define the output parser
output_parser = StrOutputParser()
# 4. Create the chain
chain = prompt | llm | output_parser
# 5. Run the chain
result = chain.invoke({"text": "Hello, how are you?"})
print(result)
Running this will give you:
Bonjour, comment allez-vous?
This looks simple, but it’s the foundation. The | operator is the core of LangChain’s composability. It takes the output of the component on its left and passes it as input to the component on its right. This allows you to string together prompts, LLMs, parsers, and other components (like retrieval tools or other chains) into complex workflows.
The ChatPromptTemplate is a structured way to build prompts. It allows you to define different message types (system, human, AI) and inject variables. StrOutputParser simply takes the LLM’s response (which is typically a AIMessage object) and extracts the plain text content.
LangChain’s real power comes from its ability to create agents. An agent is an LLM that can use tools. It receives a user query, decides which tool (if any) to use, executes that tool, observes the tool’s output, and then repeats the process until it has an answer.
Consider an agent that can search the web:
from langchain_openai import ChatOpenAI
from langchain_community.tools import DuckDuckGoSearchRun
from langchain.agents import AgentExecutor, create_tool_calling_agent
from langchain_core.prompts import ChatPromptTemplate
# Initialize the LLM
llm = ChatOpenAI(model="gpt-4o", temperature=0)
# Define the tool
search = DuckDuckGoSearchRun()
# Define the agent prompt
# This prompt guides the LLM on how to use tools.
# It includes placeholders for tool definitions and the user's input.
prompt_template = ChatPromptTemplate.from_messages([
("system", "You are a helpful assistant. Use the available tools to answer the user's question."),
("user", "{input}"),
("placeholder", "{agent_scratchpad}"), # This is where the agent records its thought process and tool outputs
])
# Create the agent
agent = create_tool_calling_agent(llm, [search], prompt_template)
# Create the agent executor
agent_executor = AgentExecutor(agent=agent, tools=[search], verbose=True)
# Run the agent
response = agent_executor.invoke({"input": "What is the weather like in London today?"})
print(response["output"])
When you run this, you’ll see verbose=True output showing the agent’s step-by-step thinking: it recognizes it needs to search, formulates a search query, executes the search tool, reads the search results, and then formulates the final answer.
The agent_scratchpad is crucial. It’s where the LLM "writes down" its intermediate steps. When the agent decides to use a tool, it outputs a special "tool call" message. The AgentExecutor intercepts this, runs the actual tool, and then feeds the tool’s output back into the agent_scratchpad for the LLM’s next turn. This loop continues until the LLM decides it has enough information to answer the original question.
The true magic of LangChain agents lies in their ability to reason about which tools to use and how to use them. The create_tool_calling_agent function, particularly with models like GPT-4o that support tool calling natively, significantly simplifies this. The LLM itself generates the arguments needed for the tool based on the user’s prompt and its understanding of the tool’s signature. You’re not explicitly coding the logic for "if the user asks about weather, use the weather tool." The LLM figures that out.
Most people focus on the Runnable interface and | chaining, but the AgentExecutor is where the dynamic decision-making happens. It’s the LLM acting as a controller, not just a text generator. The create_tool_calling_agent abstracts away much of the complex prompt engineering that used to be required to get an LLM to reliably use tools.
The next frontier is memory. How does an agent remember previous turns in a conversation, or maintain context across multiple tool uses?