Learning LangChain: Building AI and LLM Applications with LangChain and LangGraph

by Mayo Oshin, Nuno Campos

If you're looking to build production-ready AI applications that can reason and retrieve external data for context-awareness, you'll need to master LangChain—a popular development framework and platform for building, running, and managing agentic applications. LangChain is used by several leading companies, including Zapier, Replit, Databricks, and many more. This guide is an indispensable resource for developers who understand Python or JavaScript but are beginners eager to harness the power of AI.



Authors Mayo Oshin and Nuno Campos demystify the use of LangChain through practical insights and in-depth tutorials. Starting with basic concepts, this book shows you step-by-step how to build a production-ready AI agent that uses your data.

Harness the power of retrieval-augmented generation (RAG) to enhance the accuracy of LLMs using external up-to-date dataDevelop and deploy AI applications that interact intelligently and contextually with usersMake use of the powerful agent architecture with LangGraphIntegrate and manage third-party APIs and tools to extend the functionality of your AI applicationsMonitor, test, and evaluate your AI applications to improve performanceUnderstand the foundations of LLM app development and how they can be used with LangChain

Genres: Artificial Intelligence, Technical, Programming

294 pages, Paperback

Published March 25, 2025

About the author

Black belt researcher separating signal from noise to feed your curious brain @mayooshin.com/newsletter

Reviews

[Wilson Jimenez · Rating 4 out of 5]

Preface
- transformer neural network architecture, engine behind LLMs predictive capabilities



1. LLM Fundamentals with LangChain

- use ChatPromptTemplate() to provide prompt, context, question
- use llm.with_structured_output() to get json back as an answer
- llm.invoke() transforms a single input to output
* llm.batch() multiple ones
* there is also llm.stream()
- with LCEL parallelisation is automatic

2. RAG Part 1: Indexing Your Data

- chapter about indexing and storing data into embeddings
- Recursive Abstractive Processing for Tree-Organized Retrieval (RAPTOR) to recursively create an embeddings tree, helps to reduce hallucinations

3. RAG Part II: Chatting with Your Data

- ret = db.as_retriever()
* ret.invoke('user query')
* abstracts the user's query embedding logic, performs similarity comparison and returns relevant documents
^ pass a k argument to signify the number of documents to return, the larger the number, the larger the prompt and the slower the application will be
- Rewrite-Retrieve-Read strategy to rewrite insufficient user queries and prevent the model from getting distracted
- Multi-Query Retrieval to formulate multiple versions of the user's query into parallel retreival prompts for the vector store
* to overcome the limitations of distance-based similarity
* parallel retrieved documents can be flatten and passed as context to final llm answering agent
^ retriever.batch() to run queries in parallel
- Reciprocal Rank Fusion (RRF) can be used to rank documents pulled by multiple queries higher and bubbled them up
* the premise being that repeated documents bear more weight in the current user query context hence making them the first items in the retrieved doc list
- Hypothetical Document Embeddings (HyDE) is used to generate a hypothetical document from the user query
* the premise being that finding relevant documents based on this hyDE instead of the user query might be more effective
- Query Routing: have an intermediate step choosing the correct datasource to pull data from when having multiple ones
* an llm is tasked with determining where to fetch documents from based on the user query before passing to the llm answering the question
* this is logical routing
- semantic routing involves creating multiple prompts, embedding them, embedding the user question, and use cosine similarity to pick the most similar prompt

- use SelfQueryRetriever() to translate user questions to structured queries like SQL

- create_sql_query_chain() to create multi steps of npl to SQL to execution

4. Using LangGraph to Add Memory to Your Chatbot

- LangGraph is recommended for chatbots, ie when needing to keep state (memory)
- graph.get_graph().draw_mermaid_png() to print out the graph
- graph = builder.compile(checkpointer=MemorySaver()) saves state automatically
* there are checkpointers for posgress, redis, and others
- pass {'configurable': 'thread_id': 'foo-bar'} to allow independent conversations state
- graph.update_state(thread1, [HumanMessage('foo bar')]) appends to the state for later use
- trim_messages() to clean up memory and keep context from overgrowing
- filter_messages() to fetch types of messages by id, type, or name, eg all human ones
- merge_message_runs() merges messages of the same type, some models like some Anthropic's, don't support consecutive messages of the same type

5. Cognitive Architectures with LangGraph

- LLM Cognitive Architectures are the models of human reasoning and their implementations in computers
- defined as the recipes of steps taken by an LLM app

6. Agent Architectures

- ToolNode() is a wrapper around tools to call, error handling, and return back to llm
* to use with tools_condition as conditional edge
- several tools can increase inaccuracy in choosing the correct ones to call
* can be addressed by a RAG step to fetch and pass along the relevant tools only
- agentic: giving the llm the ability to choose between multiple options by using external information

- ReAct: Reason + Act, combines reasoning (CoT) and act.
* Consists of few-shot task solving trajectories, with human-written reasoning traces and actions, plus environment observations in response to actions
^ Agentic architecture where the model decides to reason, act, in a loop, and also when to break out of the loop and provide the final answer

7. Agents II

- Reflection: having an llm analise its past outputs and choices
* consists of taking the ai response and passing it as a human message to an analyser llm for review
* the main llm is passed back the reviewer llm's response wrapped as a human message
* this is necessary as dialogue-tuned llms are trained on pairs of ai-human messages, passing them all as ai messages can lead to poor performance
- you can either have a fixed number of iterations, or have the reflect step make the decision to end the revision process, or both
- subgraphs can be used as part of another graph
* like when you need a set of nodes to be used in multiple graphs
* or when multiple teams work on different parts of the graph independently
- if the parent and subgraph share the same schema fields you can add the subgraph as a node to the parent
* if not, you need a node with a function that calls the subgraph, since both extra fields passed to the subgraph and extra fields returned back to the parent graph are ignored
^ hence you need a function to transform state before or after
- Multi agent strategies
* Network: each agent can communicate with the rest, any agent can decide what other agent to call
* Supervisor: a single agent decides which agent to call, the supervisor can be enhanced with tools
* Hierarchical: supervisor of supervisors
* Custom: each agent communicates with a subset of agents, parts of the flow are deterministic and only select agents can decide which agents to call next
- in the supervisor strategy, subagents can also decide to terminate and reply back to the user

8. Patterns to Make the Most of LLMs

- some llm apps objectives:
* latency: fast responses
* autonomy: minimise human in the loop interruptions
* variance: minimise variations between invocations
- Agent: high agency, low reliability
- Chain: low agency, high reliability
- agency vs reliability its a trade-off, can be seen as a frontier, ideally we want to move the frontier outward in all directions, ie increase agency for the same level of reliability
* ways to achieve this
^ streaming: helps perceived speed
^ structured outputs: makes llm more likely to adhere to a schema
^ humn in the loop: can benefit high agency architectures by interrupting, approving, forking, undoing
^ double texting modes: the longer the answer takes, the more likely a human might send it new input before the prev has completed
- streaming updates have different modes, which can be combined by passing a list
* updates: default mode, yield each update to the shared state
* values: yield state of the graph on every change, ie after each set of nodes finishes executing, useful for displaying outputs that closely mirror the shape of the graph
* debug: checkpoints, task events, task results
- to use human-in-the-loop, need to add a checkpointer, usually MemorySaver(), which persists state after each step
* pass a second config argument to invoke() or astream() to identify the current thread to save/restore memory
* can then continue from any of the prev steps without starting from a blank slate by combining new input with saved state from the checkpointer
- the authorize pattern will pause execution, ask user for confirmation (eg before calling a tool) before proceeding
* graph.astream(input, config, interrupt_before=['tools']), tools is the node name
- to resume, call the graph with null input
* graph.astream(None, config, interrupt_before=['tools'])
- to restart, call it with new input, it'll restart from the first node and include the current state
* to drop the current state, just change the thread_id
- graph.update_state() to edit state before resuming
* this creates a new checkpoint
- fork: graph.get_state_history(config) to browse the history of all past states and get alternative answers
* returns an iterator, might need to convert to list via list comprehension

9. Deployment: Launching Yout AI Application into Production

- LangGraph Platform enables horizontal scaling of task queues, servers and a postgres checkpointer
* supports HITL, double texting, async background jobs, cron jobs, monitoring, a visual playground
- streaming modes
* values: steam state of graph after each super-step is executed
* messages: stream each token (good for chat apps) and full messages at the end of execution, this requires the graph to have a messages key

10. Testing: Evaluation, Monitoring, and Continuous Improvement

- three stages of testing
* design: assertions executed at runtime and fed back to the llm for self-correction
* preprod: tests ran before deployment with the purpose of fixing any regressions before releasing
* prod: monitor released app to catch errors before the users, identify issues and feed back to the design and preprod phases
- add a grading/validation, use an llm to check the answer is valid and absent of hallucinations
- 3 types of evaluation
* human evaluators
* heuristic evaluators: hardcoded functions and assertions, like comparing output to a predefined ground truth
- llm as-a-judge evaluator: integrating human grading rules into an llm
^ use straightforward prompts that can be easily understood and replicated by a human, avoid telling the llm to provide a range score like 1-10 with vague distinctions between scores
^ effective method to grade nlp outputs by passing it to an llm for judgement and evaluation
^ test different rounds of prompt engineering
- regression testing in llms does not always yields 100%, need to compare results of 2 or more test runs since output can vary significantly due to model updates, changes in data distribution
- agents should be tested in 3 different levels
* response: output accuracy
* single step: any single, important step like tool call is performed
^ can be used to test where the app is underperforming
* trajectory: the llm follows the correct rag, tools, llms, path

[Guru · Rating 3 out of 5]

A concise primer by OG LangChain contributors. Available on O'Reilly Learning.