Building Production AI Agents with LangChain: A Practical Guide

Last updated: October 13, 2026. Code updated for LangChain v1 (create_agent and middleware) and checked against the official LangChain documentation on this date.

Short answer: To build a production AI agent with LangChain, create it with create_agent, give it a small set of well-described tools, and add middleware for retries, call limits, human approval, and PII handling. Then trace every run with LangSmith, test against an evaluation set, and deploy on a platform that fits your traffic. The rest of this guide shows each step with working patterns.

AI agents go beyond chatbots: they take action. They can search databases, call APIs, make decisions, and run multi-step workflows without step-by-step human direction. But moving from prototype to production means handling errors gracefully, adding observability, and staying reliable when the model makes unexpected choices.

This guide covers building production-ready AI agents with LangChain, from basic tool integration to deployment patterns. It reflects the checklist the Propelius AI agents team uses before an agent goes live: limits, guardrails, tracing, and evals first, then features.

What Are AI Agents?

An AI agent is a language model that runs in a loop, deciding which tools to call and using the results until a task is complete. An agent can:

• Reason: Analyze a user query and plan steps
• Act: Call external tools (APIs, databases, functions)
• Observe: Process tool results and decide the next action
• Iterate: Continue until the task is complete

Example flow: A user asks, "What's my account balance and should I invest more this month?"

1. Agent calls the get_account_balance() tool
2. Receives result: $5,200
3. Agent calls the get_monthly_expenses() tool
4. Receives result: $3,800
5. Agent reasons: surplus of $1,400
6. Returns a recommendation with the supporting data

LangChain Agent Types

In LangChain v1, almost every production agent starts with create_agent, which builds a tool-calling agent on top of LangGraph. The older agent types (ReAct prompts, OpenAI Functions agents, initialize_agent) are legacy APIs and shouldn't be used for new work.

Recommendation: Start with create_agent. Native tool calling is more reliable than prompt-based tool selection, and middleware gives you production controls without custom loop code.

LangChain vs Other Agent Frameworks

LangChain is a strong default when you want provider flexibility and fine-grained control. Here's how it compares with other popular frameworks:

Building a Basic Agent

Building a LangChain agent takes two steps: define tools the model can call, then create the agent with create_agent.

Step 1: Define Tools

Tools are Python functions decorated with @tool. The model reads each tool's name, docstring, and type hints to decide when and how to call it.

import os
import requests
from langchain.tools import tool

@tool
def search_company_data(query: str) -> str:
    """Search the internal company knowledge base for product info, pricing, and policies."""
    resp = requests.post(
        "https://api.internal.example.com/search",
        json={"query": query},
        timeout=10,
    )
    resp.raise_for_status()
    return resp.json()["answer"]

@tool
def send_email(to: str, subject: str, body: str) -> str:
    """Send an email to a user. Use only when the user asks to be notified."""
    if not to.endswith("@yourcompany.com"):
        return "Error: emails can only be sent to company addresses."
    # Call your email provider's API here.
    return f"Email sent to {to}"

@tool
def get_weather(location: str) -> str:
    """Get the current weather summary for a city."""
    resp = requests.get(
        "https://api.weather.example.com/v1/current",
        params={"location": location, "key": os.environ["WEATHER_API_KEY"]},
        timeout=10,
    )
    resp.raise_for_status()
    return resp.json()["summary"]

Tool design rules:

• Clear, verb-based names (search_company_data, not company_searcher)
• Detailed docstrings: the model uses them to decide when to call each tool
• Type hints for every parameter
• Return strings or simple JSON: models read concise text best
• Timeouts and validation inside the tool, so one slow API or bad input can't stall the agent

Step 2: Create the Agent

create_agent takes a model, tools, and an optional system prompt. Models are referenced as "provider:model" strings, so you can switch providers without changing the rest of your code.

from langchain.agents import create_agent

tools = [search_company_data, send_email, get_weather]

agent = create_agent(
    model="openai:gpt-5.5",  # any supported provider:model string
    tools=tools,
    system_prompt="You are a helpful operations assistant. Be concise and accurate.",
)

result = agent.invoke(
    {"messages": [{"role": "user", "content": "What's the weather in London? Email me a summary."}]}
)
print(result["messages"][-1].content)

Production-Ready Patterns

A production agent needs four safeguards on top of the basic loop: error handling, guardrails, observability, and cost control. In LangChain v1, most of them are built-in middleware imported from langchain.agents.middleware.

1. Error Handling and Retries

Models are non-deterministic and APIs fail, so agents can error out, call the wrong tool, or get stuck in loops. Use retry and fallback middleware rather than wrapping the whole agent in a retry:

• ToolRetryMiddleware retries failed tool calls with exponential backoff.
• ModelRetryMiddleware retries failed model calls.
• ModelFallbackMiddleware switches to a backup model if the primary one fails.

from langchain.agents import create_agent
from langchain.agents.middleware import (
    ModelFallbackMiddleware,
    ModelRetryMiddleware,
    ToolRetryMiddleware,
)

agent = create_agent(
    model="openai:gpt-5.5",
    tools=tools,
    middleware=[
        ToolRetryMiddleware(max_retries=3, backoff_factor=2.0, initial_delay=1.0),
        ModelRetryMiddleware(max_retries=3, backoff_factor=2.0, initial_delay=1.0),
        ModelFallbackMiddleware("openai:gpt-5.4-mini"),
    ],
)

2. Guardrails and Validation

Guardrails stop agents from taking harmful or expensive actions. Use three layers:

• Validate inside tools: reject dangerous inputs in the tool itself, as send_email does above.
• Require human approval for sensitive tools with HumanInTheLoopMiddleware (see Human-in-the-Loop below).
• Limit expensive tools with ToolCallLimitMiddleware, and handle sensitive data with PIIMiddleware.

from langchain.agents.middleware import PIIMiddleware, ToolCallLimitMiddleware

agent = create_agent(
    model="openai:gpt-5.5",
    tools=tools,
    middleware=[
        ToolCallLimitMiddleware(tool_name="search_company_data", run_limit=5),
        PIIMiddleware("credit_card", strategy="mask", apply_to_input=True),
    ],
)

Never give an agent tools it should not use. The safest guardrail is not exposing destructive operations (such as deleting data) as tools at all.

3. Observability and Logging

Tracing every agent run lets you see which tools were called, with what inputs, how long each step took, and what it cost. Enable LangSmith tracing with environment variables, then tag runs so you can filter them later:

export LANGSMITH_TRACING=true
export LANGSMITH_API_KEY="your-api-key"
export LANGSMITH_PROJECT="customer-support-agent"

result = agent.invoke(
    {"messages": [{"role": "user", "content": "Help me with my order #12345"}]},
    config={
        "tags": ["production", "customer_support"],
        "metadata": {"user_id": user_id},
    },
)

Track these metrics for every run: tools called, number of model calls, latency, token usage, cost, and whether the task succeeded.

4. Cost Control

Agents use tokens quickly because each step re-sends the conversation and tool results. Control spend with:

• Cap model calls per run with ModelCallLimitMiddleware(run_limit=5) to prevent runaway loops.
• Route by difficulty: use a smaller, cheaper model for simple tasks and a flagship model only for complex reasoning.
• Cache tool results: if a user asks for the same data twice, don't call the API twice.
• Summarize long conversations with SummarizationMiddleware so context doesn't grow without limit.
• Monitor per-user spend and alert when a user exceeds a daily budget.

For more techniques, see our guide to AI cost optimization: cutting LLM token spend.

Advanced Patterns

Multi-Agent Systems

Multi-agent systems split work between specialized agents, each with its own tools and instructions:

• Researcher agent: searches docs and summarizes findings
• Writer agent: drafts emails and reports
• Executor agent: calls APIs and updates records
• Coordinator agent: routes tasks to the right specialist

In LangChain, build this with LangGraph's supervisor pattern or Deep Agents subagents. CrewAI and Microsoft Agent Framework are alternatives. Start with a single agent and split only when one agent's tool list or instructions become unmanageable.

Human-in-the-Loop

Human-in-the-loop pauses the agent before a sensitive tool runs, so a person can approve, edit, or reject the action. HumanInTheLoopMiddleware handles this and requires a checkpointer to save the paused state:

from langchain.agents import create_agent
from langchain.agents.middleware import HumanInTheLoopMiddleware
from langchain.tools import tool
from langgraph.checkpoint.memory import InMemorySaver  # use a database-backed saver in production
from langgraph.types import Command

@tool
def send_invoice(customer_id: str, amount: float) -> str:
    """Send an invoice to a customer. Runs only after human approval."""
    # Call your billing API here.
    return f"Invoice for {amount} sent to customer {customer_id}"

agent = create_agent(
    model="openai:gpt-5.5",
    tools=[send_invoice, search_company_data],
    middleware=[
        HumanInTheLoopMiddleware(
            interrupt_on={
                "send_invoice": {"allowed_decisions": ["approve", "edit", "reject"]},
                "search_company_data": False,  # safe, no approval needed
            }
        )
    ],
    checkpointer=InMemorySaver(),
)

config = {"configurable": {"thread_id": "invoice-123"}}
agent.invoke({"messages": [{"role": "user", "content": "Invoice ACME for $500"}]}, config=config)

# After a reviewer approves:
agent.invoke(Command(resume={"decisions": [{"type": "approve"}]}), config=config, version="v2")

Use human approval for payments, invoices, outbound messages to customers, and any irreversible change. For defending agents against malicious instructions hidden in content, see AI agent security: preventing prompt injection attacks.

Deployment Options

Choose a deployment platform based on how long each agent run takes, how much traffic you expect, and whether runs must pause and resume.

How to choose:

• Low or unpredictable traffic with short runs: AWS Lambda or Cloud Run, so you pay nothing when idle.
• Runs that wait for human approval or take many minutes: a host with durable state, such as LangSmith deployment or your own service with a database-backed checkpointer.
• Sustained high traffic or strict compliance: Kubernetes in your own cloud account.

Build Production AI Agents With Propelius

Propelius Technologies' AI Agents & Automation team designs, builds, and operates LangChain agents for SaaS products. We set up the production essentials from day one: tool design, guardrails and human approval, tracing, evaluation sets, and cost controls. Talk to the Propelius team about your agent, or read 5 ways AI automation cuts SaaS operating costs.

References

• LangChain. Agents. Accessed October 13, 2026.
• LangChain. Built-in middleware. Accessed October 13, 2026.
• LangChain. Human-in-the-loop. Accessed October 13, 2026.
• LangSmith. Observability quickstart. Accessed October 13, 2026.
• OpenAI. API pricing. Accessed October 13, 2026.
• Yao, S., et al. ReAct: Synergizing Reasoning and Acting in Language Models. arXiv, 2022.

FAQs

How much do production AI agents cost per request?

A typical multi-step agent run costs from under a cent to a few cents in model fees, depending on the model and token usage. For example, a run that uses 8,000 input tokens and 1,000 output tokens costs about:

• $0.003 at OpenAI's lowest-cost tier ($0.20 / $1.20 per million input/output tokens, October 2026)
• $0.028 at the balanced tier ($2 / $12 per million)
• $0.07 at the flagship tier ($5 / $30 per million)

At 50,000 runs a month, that's roughly $140, $1,400, or $3,500. Route simple tasks to cheaper models and cache tool results to cut costs.

What causes agents to fail or loop infinitely?

Agents usually fail or loop because of unclear tool descriptions, unexpected tool output, or no limit on how many steps they can take. To fix this:

• Write detailed tool docstrings and clear names.
• Validate tool inputs and outputs, and return clear error messages.
• Cap model calls with ModelCallLimitMiddleware (for example, run_limit=5).
• Monitor steps per run; if runs regularly need more than three, your tools are probably poorly designed.

When should you use an agent vs a simple chain?

Use a chain (or a fixed workflow) when the steps are always the same, and an agent when the model must decide which tools to call and in what order.

• Chain: "Summarize this document" always runs the same steps.
• Agent: "Research this topic and email me a summary" needs the model to decide to search, synthesize, then send.

Agents cost more and are slower because of the extra reasoning steps, so use them only when you need that flexibility.

How do you test AI agents reliably?

Test AI agents in three layers: unit tests for each tool, integration tests for the agent, and evaluation sets for quality.

• Unit tests: test each tool independently with mocked inputs.
• Integration tests: run fixed queries and check that the expected tools are called in the expected order.
• Evaluations: define test cases in LangSmith (query, expected tool use, success criteria), run them regularly, and track the pass rate.

We recommend reaching 85%+ consistency before production, and re-running the evaluation set after every model or prompt change.

How do you reduce agent latency?

Agents are slower than single model calls because each step waits for a model response and a tool result. To speed them up:

• Let the model call independent tools in parallel.
• Stream responses so users see progress immediately.
• Cache tool results.
• Use a faster, smaller model for simple steps.
• Keep tool APIs fast and set timeouts.
• Pre-warm instances in serverless environments.

Which LLM should I use for a production agent?

Use the smallest model that passes your evaluation set. Start by testing a flagship model to establish a quality baseline, then try cheaper and faster models on the same tests. Because LangChain uses "provider:model" strings, switching models is a one-line change.

Do I need LangGraph to build agents with LangChain?

No separate setup is needed: create_agent is built on LangGraph, so you get its persistence and streaming automatically. Use LangGraph directly only when you need a custom workflow with explicit steps and branches.