If you’re building serious LLM-powered applications, you’ve likely come across the Lang* ecosystem: LangChain, LangGraph, and LangSmith. Each tool solves different challenges in developing, orchestrating, and monitoring LLM workflows. But how do you know which one is the right fit for your use case—or when to use them together? Let’s break down the strengths of each tool and offer a straightforward guide to help you choose (or combine) them effectively.
A Primer
LangChain
What It Does
LangChain is a powerful framework that makes it easier to build and deploy applications powered by LLMs. It’s more or less the Swiss army knife of the Lang* ecosystem.
LangChain provides a modular architecture that lets developers combine prompts, models, and tools into robust chains of operations. Whether you’re developing a prototype or scaling an existing product, LangChain helps you handle everything from prompt management to tool integrations, without having to build everything from scratch.
One of LangChain’s most compelling features is its support for retrieval-augmented generation (RAG). RAG allows developers to incorporate external knowledge sources, such as vector databases or document search engines, into their LLM applications (screenshot source). With LangChain, it’s straightforward to combine retrieval modules with LLM prompts, enabling your app to generate contextually relevant answers based on real-time information rather than relying solely on the model’s training data. This is especially valuable for use cases like chatbots, question-answering systems, and enterprise knowledge management.
The data you have at their disposal can be public or proprietary, structured or unstructured. These RAG pipelines are very flexible. Below is a simplified example of a RAG pipeline you might see for an app that summarizes content (screenshot source).
LangChain’s modular design allows users to mix and match components. Possible components include:
- Prompt templates: allow you to craft reusable, consistent instructions for your LLM
- Chains: allow you to link together prompts, models, and tools to build step-by-step workflows
- Agents: give your app the ability to decide which tool or model to call based on a user’s input
- Memory: keeps track of conversations so your app can have context
LangChain is ideal for chatbots, document Q&A, summarization, or any app where you want to build, experiment, and iterate quickly.
Similar Tools
Other, similar tools to LangChain include:
- Haystack: search and RAG pipelines
- LlamaIndex: document Q&A and RAG pipelines
- Microsoft PromptFlow: low/no-code visual designer for LLM apps
LangGraph
What It Does
LangGraph provides a blueprint for building more complex structures. It’s perfect when your app needs to handle branching logic, like a conversation that can go in different directions depending on what the user says. Think of it like building a choose-your-own-adventure book, but with AI.
If you’ve ever worked with a network graph, you may be familiar with the concept of nodes and edges. I had to get really comfortable with structuring data in such a way as to support nodes and edges when I was building my AI Strategy tool. Likewise, LangGraph works with this same structure. In fact, LangGraph builds out workflows using this structure, allowing you to define tasks as nodes and connect them with edges (transitions) to manage steps and decisions. This structure makes it easy to visualize and control complex processes that go beyond simple sequential chains.
LangGraph also supports branching and loops, giving your application the ability to revisit steps, handle alternative paths, and dynamically adapt to different scenarios. This is particularly valuable in use cases like multi-turn conversations or advanced decision-making pipelines.
For example, this flowchart illustrates a LangGraph-powered refund processing agent, as detailed in LangSmith’s agent evaluation tutorial. The workflow initiates at the __start__ node and proceeds to the gather_info node, where the agent extracts customer and purchase information from the conversation. Based on the extracted data, the flow branches: if sufficient purchase details (like Invoice ID or Invoice Line IDs) are available, it moves to the refund node to process the refund; if adequate customer information (such as name and phone number) is provided, it transitions to the lookup node to search for purchase history; otherwise, it concludes at the __end__ node, prompting the user for more information. This structure enables the agent to handle various refund scenarios efficiently by dynamically routing requests based on the information provided (screenshot source).
State management is another key feature of LangGraph. It allows you to track and update data across different steps of your application’s workflow. This means that intermediate results—like user inputs, lookup results, or tool outputs—can be stored and accessed as the workflow progresses. By maintaining this context, LangGraph ensures that your application can handle complex, multi-step interactions without losing track of important details, such as user preferences, partial answers, or processing status.
And, although you can use LangGraph without LangChain, it integrates with LangChain, allowing you to incorporate LangChain components inside your graph. Think of it as building blocks that click together, letting you leverage the best of both tools to create powerful and modular AI applications.
LangGraph is ideal for customer support agents, data pipelines, complex decision trees. Basically anywhere your app needs to juggle a variety of scenarios.
Similar Tools
Other, similar tools to LangGraph include:
- Microsoft Autogen: low/no-code visual designer for building multi-agent workflows with LLMs
- FlowiseAI: low/no-code visual designer for LLM workflows
- CrewAI: orchestrates multi-agent workflows with roles, e.g., planner, researcher, summarizer, etc
LangSmith
What It Does
No matter how slick your app looks, you’ll eventually hit a snag—like a prompt that suddenly misbehaves or an output that makes zero sense. LangSmith is your backstage pass to debugging, testing, and monitoring everything that happens inside your LLM app.
With LangSmith, you can trace every step your app takes, from the initial input to the final output (screenshot source). For example, you can visualize the flow of your LLM application, including intermediate steps like tool calls and external service interactions. This tracing functionality is like peeking inside the AI’s brain to understand how decisions are made and where improvements might be needed.
You can also evaluate your app’s outputs using LangSmith. This feature lets you test your LLM application against benchmarks or human-annotated judgments to ensure that the results meet your desired quality standards. It’s especially helpful for regression testing, so you can maintain consistent performance even as your application evolves.
LangSmith helps you monitor key metrics such as token usage, costs, latency, and error rates. With real-time dashboards and alerting mechanisms, you can minimize the risk of being caught off guard by unexpected performance issues or usage spikes. These dashboards help ensure your application remains reliable, efficient, and cost-effective.
Last but definitely not least, LangSmith offers robust prompt management tools. You can version different prompts, test them systematically, and analyze their performance to determine which one delivers the best results. This is invaluable for refining your prompt engineering strategies and maximizing your application’s effectiveness.
LangSmith is ideal for any app you plan to deploy—especially when you want to identify and fix bugs and ensure everything runs smoothly.
Similar Tools
Other, similar tools to LangSmith include:
- Weights & Biases: ML experiment tracking and monitoring
- PromptLayer: prompt observability and versioning
- EvidentlyAI: model monitoring and performance tracking
A Simple Flowchart
I created a flowchart to help you decide which might work best for your app. Just keep in mind all of these tools can be combined for greater brawn. ?
Tip: It’s a pretty simple app, but you can zoom and drag the nodes (the rectangles) or the canvas.
Leave a Reply