State And Handoffs
How agents remember work across steps, resume after a crash without repeating irreversible actions, and hand off to a person without losing any context.
State is what makes an agent resumable
A single model call is stateless. A production agent almost never is. It runs multiple steps, may pause for approval, and must survive a crash without redoing irreversible actions. State is the record of where a run is and what has already happened, and getting it right is what separates a robust workflow from one that corrupts data on the first restart. The test is brutal and simple: kill the process mid-run, restart it, and check whether the run resumes correctly without duplicating any action.
What to persist
| What | Why | Typical home |
|---|---|---|
| Run inputs and a stable run ID | Find and replay any execution | runs table, keyed by run_id |
| Step results (checkpoints) | A resume does not repeat completed steps | LangGraph checkpointer (Postgres) |
| Pending actions | A human can decide days later with full context | pending_actions table plus notification |
| Executed side effects | A retried step recognizes work already done | executed_actions table with idempotency key |
| Status | System knows whether to resume, escalate, or leave alone | running, waiting, failed, done enum on the run |
Durable state with a checkpointer
In LangGraph, persistence is a checkpointer attached at compile time. Every node transition writes a checkpoint keyed by a thread_id, so any run can be resumed from its last completed step. In-memory checkpointers are for tests only; production uses Postgres:
from langgraph.checkpoint.postgres import PostgresSaver
DB_URI = "postgresql://agent:secret@db:5432/agent_state"
with PostgresSaver.from_conn_string(DB_URI) as checkpointer:
checkpointer.setup() # creates checkpoint tables, run once
graph = builder.compile(checkpointer=checkpointer)
config = {"configurable": {"thread_id": "triage-tkt_48213"}}
result = graph.invoke(
{"ticket_id": "tkt_48213", "ticket_body": body},
config,
)
The thread_id is derived from the trigger, not generated randomly. That means a duplicate webhook delivery lands on the same thread instead of spawning a second run, and support can find the run for any ticket in one query.
Checkpoints give you replay for free
Durable checkpoints do more than survive crashes. Because every step's state is stored, you can fetch the history of any thread and inspect exactly what the state looked like before the step that went wrong, which turns postmortems from speculation into reading. You can also fork a past checkpoint to test a fix: rerun the failing step with a corrected prompt against the exact state it saw in production. Get comfortable with graph.get_state(config) and graph.get_state_history(config) early; they are the debugger for your agent, and teams that skip them end up rebuilding a worse version by hand.
Handoffs to people are interrupts, not restarts
A handoff is a controlled transfer of a run from the agent to a human, usually for approval or because the agent hit an escalation trigger. In LangGraph this is the interrupt function: the node pauses, the payload you pass is surfaced to your approval UI, and the run resumes exactly where it stopped when someone responds, even days later:
from langgraph.types import interrupt, Command
def human_review(state: TriageState) -> dict:
decision = interrupt({
"ticket_id": state["ticket_id"],
"draft": state["draft"],
"why_stopped": "reply drafts require approval",
"decision_needed": "approve, edit, or reject",
})
if decision["action"] == "approve":
return {"status": "approved"}
if decision["action"] == "edit":
return {"draft": decision["draft"], "status": "approved"}
return {"status": "rejected"}
# Later, when the reviewer clicks approve in your UI:
graph.invoke(Command(resume={"action": "approve"}), config)
Two details matter in production. First, interrupt requires a checkpointer, because the paused state has to live somewhere durable. Second, when the run resumes, the node restarts from its beginning, so any code before the interrupt call runs again. Keep side effects out of nodes that interrupt, or guard them with idempotency keys.
Idempotency is not optional
If a run crashes after sending an email but before recording that it sent, a naive resume sends the email twice. Give every external action an idempotency key derived from the run and step, and record execution atomically with the action:
import hashlib, json
def idempotency_key(run_id: str, step: str, payload: dict) -> str:
raw = f"{run_id}:{step}:" + json.dumps(payload, sort_keys=True)
return hashlib.sha256(raw.encode()).hexdigest()
def send_reply(run_id: str, to: str, body: str) -> None:
key = idempotency_key(run_id, "send_reply", {"to": to, "body": body})
inserted = db.execute(
"INSERT INTO executed_actions (key, run_id, action) "
"VALUES (%s, %s, 'send_reply') ON CONFLICT (key) DO NOTHING",
(key, run_id),
)
if inserted.rowcount == 0:
return # already sent on a previous attempt, skip
email_api.send(to=to, body=body, idempotency_key=key)
Many APIs (Stripe, most email providers) accept an idempotency key directly; pass yours through so the protection extends across their retries as well as your own. Assume any step can be attempted more than once and design so that a second attempt is safe.
Handoffs between agents are API contracts
Larger systems split work across specialized agents: one classifies, another drafts, another checks policy. Each handoff is a boundary where you should validate the payload and log the transfer, because a malformed object passed silently between agents is a common source of confusing failures. The cheapest reliable pattern is a Pydantic model per handoff:
class ClassifierToDrafter(BaseModel):
run_id: str
urgency: str
category: str
customer_id: str
context_refs: list[str] # provenance, not raw blobs
def hand_off(payload: dict) -> ClassifierToDrafter:
msg = ClassifierToDrafter.model_validate(payload) # raises on garbage
log.info("handoff", run_id=msg.run_id, edge="classifier->drafter")
return msg
Note context_refs rather than the retrieved documents themselves. Passing provenance instead of raw blobs keeps handoff payloads small enough to log in full, and a logged handoff you can read six weeks later is worth more than one you had to truncate. The receiving agent re-reads what it needs.
What a good handoff payload contains
Both kinds of handoff - to a person and to another agent - fail in the same way: the receiver gets a conclusion without the means to evaluate it. A human handed "Approve this refund?" with a draft and nothing else either approves everything or opens four other tabs, and both outcomes are bad.
Five fields make a handoff usable. Every one of them answers a question the receiver would otherwise have to go and find:
{
"run_id": "triage-tkt_48213",
"why_stopped": "Refund over the auto-approve ceiling ($50)",
"what_i_did": ["read customer record", "read last 5 tickets",
"matched KB article kb_218"],
"what_i_want": {"action": "issue_refund", "amount_cents": 7400,
"order_id": "ord_8823ka91"},
"what_i_could_not_check": ["knowledge_base_unavailable"],
"decision_needed": "approve | edit amount | reject",
"expires_at": "2026-08-08T17:00:00Z"
}
- why_stopped is the field people leave out and the one reviewers read first. "Because a rule said so, and here is the rule" turns a judgment call into a check.
- what_i_did is the evidence trail. Without it the reviewer cannot tell a well-researched proposal from a guess.
- what_i_could_not_check is the degradation list from the previous lesson, carried through. A reviewer who knows the knowledge base was down reviews differently, and should.
- expires_at is what stops a paused run living forever, which is the next problem.
The state nobody plans for: waiting forever
Interrupts make pausing easy, and that is the trap. The reviewer goes on holiday. The Slack message is missed. Six weeks later somebody finds forty runs in waiting and nobody knows whether the underlying tickets were handled another way. The paused run is not the failure - the absence of a policy for paused runs is.
Decide three things, in advance, for every gate you add:
- How long may it wait? An hour for a customer-facing reply, a week for an internal report. Write the number down; it belongs next to the gate in the design.
- What happens at the deadline? There are only three honest answers, and you must pick one per gate: escalate to a second person, take the safe default action (usually "do nothing and tell the customer a human will follow up"), or fail the run loudly. Silence is not on the list.
- Who is told? A run that expires without anybody hearing about it is data loss with extra steps.
The mechanism is unglamorous - a scheduled job that scans for runs in waiting past their deadline and applies the policy:
def sweep_expired_runs(now: datetime) -> None:
for run in db.query_runs(status="waiting", expires_before=now):
policy = GATE_POLICY[run.gate] # explicit, per gate
if policy == "escalate":
notify(run.escalation_target, run)
db.set_run(run.id, expires_at=now + timedelta(hours=4))
elif policy == "safe_default":
graph.invoke(Command(resume={"action": "reject",
"reason": "review timed out"}),
{"configurable": {"thread_id": run.id}})
else: # "fail_loudly"
db.set_run(run.id, status="failed", error="review timed out")
alert(run)
The dashboard that matters alongside it is one line: how many runs are waiting, and what is the age of the oldest one? That single number catches an abandoned review queue faster than any error-rate alert, because an abandoned queue produces no errors at all.
Test the crash, do not assume it
"Resumes correctly after a crash" is an assertion, and assertions in this area are wrong more often than not - usually because a side effect sits before an interrupt in the same node, and the resume re-runs it. This is cheap to actually test:
def test_resume_does_not_duplicate_side_effects(graph, config, fake_email):
# 1. Run until the review interrupt.
graph.invoke({"ticket_id": "tkt_1", "ticket_body": "..."}, config)
state = graph.get_state(config)
assert state.next, "expected the run to be paused at review"
# 2. Simulate the process dying and coming back: same thread_id,
# a completely fresh graph object, nothing shared in memory.
fresh = build_graph_with_checkpointer()
fresh.invoke(Command(resume={"action": "approve"}), config)
# 3. The assertion that actually matters.
assert fake_email.send_count == 1
Run that test for every action that costs money, sends a message, or changes a record in a system you do not own. It is the test most teams write only after the incident that made it necessary, and the incident is invariably more expensive than the afternoon.
One habit makes it much more likely to pass: keep every node either side-effecting or interrupting, never both. If a node might pause, it does nothing irreversible; the irreversible work goes in the node after the gate. That constraint costs you one extra node and removes a whole category of duplicate-action bug.
How much state is too much
The opposite failure is real too. Teams that get burned once start persisting everything - every intermediate string, every retrieved document, every full model response - and end up with a state object nobody can read, a checkpoint table growing faster than the business, and personal data sitting in a store nobody thought of as a database.
Three questions keep it proportionate:
- Would a resume need this? If not, it is a log line, not state. Logs are cheap, searchable, and easy to expire; state is a contract.
- Can it be a reference? Store
kb_218, not the article. Store the ticket ID, not the ticket. References stay valid, blobs go stale, and a stale blob is worse than a missing one because it looks authoritative. - Would you be comfortable if this were read back to the customer? Checkpoints frequently contain personal data, which means retention rules and deletion requests apply to them exactly as they do to your main database. Decide the retention period when you create the table, not when somebody asks for their data to be deleted.
Setup: durable state in one file, with the step everyone forgets
Persistence is the part of this lesson you should not take on trust, because the failure mode is invisible until a crash. Here it is end to end, small enough to run in a minute. Package versions are the current releases as of August 2026; pin them, since the checkpointer packages version independently of the core library.
pip install "langgraph==1.2.10" "langgraph-checkpoint-sqlite==3.1.1"
# for a real deployment, use Postgres instead:
# pip install "langgraph-checkpoint-postgres==3.1.1"
SQLite is right for learning and for a single-process job. Postgres is what you use the moment more than one worker can pick up a run, because two processes writing checkpoints to a file on a local disk is not a design, it is a race. The API surface is close enough that swapping later is a small change; the operational difference is total.
# durable.py - run with: python durable.py (twice - see below)
import sqlite3
from typing import TypedDict
from langgraph.graph import StateGraph, START, END
from langgraph.checkpoint.sqlite import SqliteSaver
class State(TypedDict, total=False):
ticket_id: str
steps: list
status: str
def step_one(state: State) -> dict:
print("running step_one")
return {"steps": (state.get("steps") or []) + ["one"], "status": "one"}
def step_two(state: State) -> dict:
print("running step_two")
return {"steps": (state.get("steps") or []) + ["two"], "status": "done"}
conn = sqlite3.connect("checkpoints.sqlite", check_same_thread=False)
checkpointer = SqliteSaver(conn)
# The step everyone forgets. Without it the first write fails with a
# sqlite3.OperationalError about a missing table, and the message does
# not mention checkpointing at all.
checkpointer.setup()
builder = StateGraph(State)
builder.add_node("step_one", step_one)
builder.add_node("step_two", step_two)
builder.add_edge(START, "step_one")
builder.add_edge("step_one", "step_two")
builder.add_edge("step_two", END)
graph = builder.compile(checkpointer=checkpointer)
config = {"configurable": {"thread_id": "tkt_9f2"}}
if __name__ == "__main__":
existing = graph.get_state(config)
if existing.values:
print("resuming; state already on disk:", existing.values)
else:
print("fresh run")
print(graph.invoke({"ticket_id": "tkt_9f2"}, config))
Run it twice. The first run prints "fresh run" and executes both nodes. The second prints the state it recovered from disk before doing anything - and that is the whole point: the run's identity is the thread_id, not the process. Delete checkpoints.sqlite and you are back to a fresh run, which is a useful thing to know when a resume behaves strangely in development.
Three things about that file generalise to any real deployment:
- The
thread_idmust be derived, never generated. Deriving it from the triggering entity - the ticket ID, the order ID - means a duplicate webhook resumes the existing run instead of starting a second one. A random UUID per invocation gives you persistence with none of the benefit. - The checkpointer belongs to the deployment, not the graph. Compile the graph once and pass a checkpointer chosen by environment: SQLite locally, Postgres in production. Hard-coding it is how test runs end up writing to the production checkpoint table.
- Setup is a migration. The
setup()call creates tables. In production that belongs in your deploy step, run once and deliberately, not on every process start where a cold start can race itself.
Inspecting a stuck run without guessing
Once runs are durable, the commonest incident is not a crash - it is a run that is simply sitting there, and the operator's first question is "sitting where, and since when?" Two calls answer it, and both belong in a small script you keep next to the service.
# inspect.py - python inspect.py tkt_9f2
import sys
from durable import graph
config = {"configurable": {"thread_id": sys.argv[1]}}
snapshot = graph.get_state(config)
print("next node(s) :", snapshot.next or "(none - the run is finished)")
print("status :", snapshot.values.get("status"))
print("values :", snapshot.values)
print("\n--- history, newest first ---")
for i, past in enumerate(graph.get_state_history(config)):
print(i, past.next, past.values.get("status"))
if i > 10:
break
The field that resolves most incidents is snapshot.next. If it is empty, the run finished and whatever you are waiting for was never going to arrive from here - look downstream. If it names a node, the run is genuinely paused there, and the history tells you when it arrived and what it had at that point. That is a two-minute diagnosis instead of an afternoon of reading application logs.
| What you observe | What it means | Action |
|---|---|---|
| An error about a missing table on the first write | setup() never ran against this database | Run it as a deploy step. The error text will not mention checkpointing, so this one costs people an hour the first time |
get_state returns empty values for a run you know exists | Wrong thread_id, or a different database than the one the run wrote to | Print the resolved connection string. In development this is nearly always a stale local SQLite file |
| Every invocation starts from the beginning | A new thread_id per call, or the graph was compiled without a checkpointer | Derive the id from the triggering entity; assert the compiled graph has a checkpointer at startup |
snapshot.next names a node, and nothing is moving | A gate is waiting on a human or an external event that never came | This is the expiry-policy case. Resume with a safe default, escalate, or fail loudly - but decide, do not leave it |
| Resume works, but an email went out twice | A side effect sits before the interrupt in the same node | Split the node. Side-effecting or interrupting, never both |
| Checkpoint storage growing much faster than run volume | Whole API responses are being written into state | Store references. Add a retention job in the same change, before it becomes a data-protection question |
A handoff contract, written out
The lesson argues that every handoff needs a validated contract. Here is one filled in, because "define a schema" is the kind of advice that gets nodded at and skipped. This is the packet a triage agent hands to a refunds agent - or to a person - and every field exists because its absence caused a question somebody had to go and answer manually.
class Handoff(BaseModel):
# identity: how the receiver finds everything else
run_id: str # the thread_id, so the trail is followable
source: str # "triage-agent" - who is handing over
created_at: datetime
# the ask: what the receiver is expected to do
action_requested: str = Field(pattern="^(refund|escalate|close)$")
reason: str = Field(min_length=20) # long enough to be a real sentence
# the evidence: what the sender based this on
ticket_id: str
customer_id: str | None # None is legitimate and visible
evidence: list[str] # ids, not blobs: ["kb_12", "tkt_88"]
degraded: list[str] # what was unavailable at the time
# the limits: what the receiver must not exceed
max_refund_cents: int = Field(ge=0, le=50_00)
requires_human: bool
Four properties make this a contract rather than a message. It carries the run identity, so a receiver can pull the full state rather than being told a story about it. It carries evidence as references, which stay valid, instead of copied text, which goes stale. It carries the degradation list, so the receiver knows the sender was working with a partial picture - the single most commonly omitted field and the one that most often turns into "why did it say that?". And it carries an explicit ceiling, so authority is transferred with a limit attached rather than assumed.
Validate it on the way out and again on the way in. Validating twice looks redundant and is not: the sender's check catches its own bugs, and the receiver's check catches version skew after somebody deploys a change to one side and not the other. That second case is the one that bites in a system where two agents are deployed separately, and it is silent unless somebody asserts.
SQLite or Postgres: deciding rather than defaulting
The checkpointer choice gets made by accident more often than it gets made, usually by whichever line was in the tutorial. The two questions that actually decide it:
| Question | SQLite is fine if... | You need Postgres if... |
|---|---|---|
| How many processes can touch a run? | Exactly one, always - a single worker or a local script | More than one, ever, including during a deploy where old and new overlap |
| What happens if the machine is replaced? | Nothing you would miss - the file is disposable | Paused runs must survive. A file on a container's disk does not |
The second row catches people running on managed platforms, where the filesystem is frequently ephemeral. Everything works in testing, then a routine deploy replaces the container and every run waiting on a human approval quietly ceases to exist - with no error, because nothing failed. The runs simply are not there any more, and the only symptom is a review queue that got shorter overnight.
A practical middle path: use SQLite locally so development needs no database, select the checkpointer from an environment variable, and add a startup assertion that refuses to boot in production without the production backend. Two lines, and they close the gap between "we meant to switch it" and "we switched it".
def make_checkpointer(env: str):
if env == "production":
from langgraph.checkpoint.postgres import PostgresSaver
url = os.environ["CHECKPOINT_DATABASE_URL"] # KeyError is correct here
return PostgresSaver.from_conn_string(url)
from langgraph.checkpoint.sqlite import SqliteSaver
return SqliteSaver(sqlite3.connect("checkpoints.sqlite",
check_same_thread=False))
The bare os.environ[...] is deliberate. A missing checkpoint database in production should stop the process at boot with a loud error, not fall back to a local file that appears to work for an afternoon and then evaporates.
Common mistakes
- Keeping state only in memory, so a crash loses the run or replays actions.
- Handing off with no context, forcing the human to reconstruct what happened.
- Skipping idempotency keys, then duplicating emails, charges, or records on retry.
- Putting side effects before an interrupt call, so every resume re-executes them.
What you should be able to do now
- Attach a Postgres checkpointer to a LangGraph graph and resume a run by thread_id.
- Pause a run for human approval with interrupt and resume it with Command(resume=...).
- Derive idempotency keys from run and step, and gate every external write on them.
- Define a validated Pydantic contract for every handoff between agents.
- Pass the kill-and-restart test: no lost runs, no duplicated actions.
- Set an expiry and a named policy for every gate, and monitor the age of the oldest waiting run.
Where this goes next
This is the second and last free lesson. Together with the five-part architecture you now have the two foundations: a system you can reason about, and a state model that survives a crash and a human being on holiday. Write the kill-and-restart test against your own agent before you read anything else - it either passes or it teaches you something today.
The paid lessons cover the rest of production. Failure classification and bounded retries with backoff. Evaluation sets so a prompt or model change can be shown to help. Tracing and structured logging that make a bad run reconstructable months later. Cost ceilings per run so a looping agent cannot spend a weekend's budget. Prompt-injection defence where untrusted content meets real tool permissions. Deployment, rollback, and the runbook for the night an agent starts behaving oddly. Every lesson ships runnable code and the repo.
If the goal is a working system rather than the skill set, we build these as projects - see AI automation services, or start with the free automation audit, which includes an honest read on whether a process needs an agent at all. The idempotency, retry and backoff and observability entries are the short versions of three things assumed above.
Leave your email below and both free lessons arrive with their code - the StateGraph, the checkpointer setup, the interrupt-and-resume flow, and the idempotency helper - as files you can run rather than snippets to retype.
Key takeaways
What to remember from this lesson
- The test for state is brutal and simple: kill the process mid-run, restart it, and check the run resumes without duplicating any action.
- Derive thread_id from the trigger, never randomly. A duplicate webhook then lands on the same thread instead of spawning a second run.
- A resumed node restarts from its beginning, so any code before an interrupt call runs again. Keep side effects out of interrupting nodes or guard them with idempotency keys.
- Idempotency keys derived from run and step, recorded atomically with the action, are what stop a crash-after-send from sending twice. Pass yours through to APIs that accept one.
- get_state and get_state_history are the debugger for your agent: inspect the exact state before the failing step, and fork a past checkpoint to test a fix against real production state.
Try it yourself
Stop a retried webhook from charging twice
This is the single most expensive bug in agent engineering. The fixture is a fake payments gateway and a ledger table - no network, no real money. A delivery service retried the same booking-deposit webhook. Write the idempotency check that makes the second attempt a no-op that still returns the original receipt.
Try it yourself
Resume a crashed run without redoing completed steps
A run for an accounting firm's invoice-chase agent died after two of four steps. The checkpoint fixture below is what a LangGraph checkpointer would have persisted. Write the resume logic: skip what is already done, run only what is left, and prove which steps actually executed.
Check yourself
The three state mistakes that corrupt data
You derive thread_id with a fresh UUID on every webhook. What breaks?
A node sends an email, then calls interrupt() to wait for approval. The run is resumed a day later. What happens?
What is the honest test that your state design works?
Answer every question to mark.
Exercise
Pass the kill-and-restart test
- 1Attach a Postgres checkpointer to a LangGraph graph and run it once with a thread_id derived from a fake trigger ID rather than a random value.
- 2Add a node that performs one visible side effect - append a line to a file, or insert a row - and one that pauses with interrupt for approval.
- 3Start a run, let it pause at the interrupt, then kill the process. Restart and resume it with Command(resume=...). Confirm the run completes.
- 4Count the side effects. If the line was appended twice, you have reproduced the bug the lesson warns about: the side effect sat before the interrupt in a node that restarts from its beginning.
- 5Fix it with an idempotency key derived from run_id and step, recorded with an INSERT ... ON CONFLICT DO NOTHING, and skip the action when the insert affects zero rows.
- 6Repeat the kill-and-restart cycle three times. The side effect must appear exactly once across all three.
- 7Call get_state_history for the thread and read the state as it was immediately before your interrupt node.
You are done when Three kill-and-restart cycles on the same thread_id leave exactly one side effect recorded, the run reaches its end state after each resume, and you can print the checkpoint history and point at the state the interrupting node saw.
Companion files
Checkpointing, interrupts, and idempotency as code
The Postgres checkpointer setup, the interrupt-based handoff node, the idempotency-key helper, and a demo graph you can actually kill mid-run - everything the exercise above asks you to build.
The free-preview pack includes 8 files of runnable code and config for State And Handoffs, plus the other free lessons of this course. No email needed.
Free preview pack - 16 KB zipCheck yourself
Answer before you reveal
Your resume duplicates an email every time. Where is the bug, given that the node code looks correct?Show answer
The send sits before the interrupt call in an interrupting node. A resumed node restarts from its beginning, so everything before the interrupt re-executes. Move the side effect out, or gate it on an idempotency key.
Why does interrupt require a checkpointer?Show answer
Because the paused state has to live somewhere durable - a reviewer may respond days later, and the run must resume from exactly where it stopped.
Why validate the payload passed from one agent to the next when both are your own code?Show answer
Because a malformed object passed silently across a handoff is a common source of confusing failures - the error surfaces far from its cause. A Pydantic model per handoff raises at the boundary and gives you a place to log the transfer.
