Pattern 01
Hybrid retrieval, not pure vector
BM25 catches exact-phrase queries that embeddings miss. Embeddings catch paraphrases. Use both; rerank.
Agent playbook · Agency agent
Agency Client Reporting AI Agent
Create an agent that compiles campaign results, highlights client-ready insights, drafts reports, and prepares account managers.
Tier-1 inboxRetrieval groundedRefusal patternsKill switch
Architecture
A 6-step loop. No more, no less.
┌──────────────┐ ┌──────────────────┐ ┌─────────────────────┐
│ inbound │──▶ │ router / intent │──▶ │ retrieval (hybrid) │
│ (ticket, │ │ classify │ │ bm25 + embeddings │
│ webhook) │ └──────────────────┘ └─────────┬───────────┘
└──────────────┘ │ │
▼ ▼
┌─────────────────┐ ┌─────────────────────┐
│ guard / PII │ │ refusal classifier │
│ detect & │ ◀── │ grounded? cited? │
│ redact │ │ in scope? │
└─────────────────┘ └─────────┬───────────┘
│
▼
┌────────────────────┐
│ reply + citations │
│ emit trace + eval │
└────────┬───────────┘
│
▼
┌────────────────────┐
│ human handoff / │
│ ticket close │
└────────────────────┘Six steps. Each one has an eval. Each one logs a trace. The kill switch sits in front of the router, so we can stop the world at any point without a deploy.
The four patterns
Picks that change the outcome.
Pattern 02
Refusal as a system step
A separate classifier decides “can this be answered grounded in our docs?” before the agent ever speaks.
Pattern 03
Citations on every reply
No citation, no send. Every customer-visible answer references the doc section that authorized it. Audit is one click.
Pattern 04
Kill switch in front, rollout dial behind
A flag that disables the route without a deploy. A rollout dial at 1%, 10%, 50%, 100%. The first time you’ll need it, you’ll be glad.
Related playbooks
Other agents in the catalog.
Want the build