🍳 Cookbook: Building a RAG Agent (Chat with Docs)

One of the most powerful agent patterns is RAG (Retrieval-Augmented Generation)—giving your agent access to a custom knowledge base (PDFs, internal wikis, or product manuals) that wasn't in its training data.

🎯 Goal

Build a "Manual Reader" agent that can answer questions about a specific PDF user manual.

---

🏗️ Phase 1: The RAG Tool

Instead of hardcoding knowledge, we build a Tool that queries a Vector Database.

1. Install Dependencies

You'll need a vector store. We'll use chromadb for a local embedded database.

poetry add chromadb sentence-transformers

2. Create the Knowledge Tool

In app/tools/knowledge.py:

import chromadb
from app.tools.base import tool

# Initialize client (persistent storage)
client = chromadb.PersistentClient(path="./knowledge_db")
collection = client.get_or_create_collection("product_manuals")

@tool(name="search_manuals", category="knowledge")
def search_manuals(query: str) -> str:
    """
    Searches the product manuals for relevant information.
    Use this when the user asks specific questions about device operation.
    """
    results = collection.query(
        query_texts=[query],
        n_results=3
    )

    # Format results for the LLM
    context = ""
    for doc in results['documents'][0]:
        context += f"- {doc}\n"

    return context

---

📥 Phase 2: Ingestion Script

You need a way to get data into the database. Create a standalone script ingest.py:

import chromadb

client = chromadb.PersistentClient(path="./knowledge_db")
collection = client.get_or_create_collection("product_manuals")

documents = [
    "To reset the device, hold the power button for 10 seconds.",
    "The battery light flashes red when charge is below 5%.",
    "Standard warranty covers the device for 2 years."
]

print("Ingesting data...")
collection.add(
    documents=documents,
    ids=["doc1", "doc2", "doc3"]
)
print("Done!")

Run it once to prime your database:

python ingest.py

---

🤖 Phase 3: The Specialist Agent

Now create an agent configured to prioritize this tool.

# app/agents/support.py
from app.agents.base import BaseAgent
from app.tools.knowledge import search_manuals

class SupportAgent(BaseAgent):
    def __init__(self):
        super().__init__(
            name="support_bot",
            # We explicitly tell it to use the tool in the system prompt
            system_prompt="""
            You are a helpful support assistant. 
            ALWAYS use the 'search_manuals' tool before answering questions.
            Do not guess. If the tool returns no results, say you don't know.
            """,
            tools=[search_manuals]
        )

---

🚀 Phase 4: Run It

# Register your new agent in the CLI or run via orchestrator
poetry run agentic orchestrate "How do I reset the device?"

What happens: 1. Agent receives: "How do I reset...?" 2. Agent thinks: I need to look this up. 3. Agent calls: search_manuals("reset device") 4. Tool returns: "To reset... hold button 10s." 5. Agent answers: "You can reset the device by holding the power button for 10 seconds."

---

💡 Pro Tip: Semantic Router

For advanced setups, don't just use RAG as a tool. Use Semantic Routing in the Orchestrator to automatically route all technical questions to the RAG Agent, while routing chit-chat to a basic LLM.

# app/services/router.py
async def route_request(input_text):
    if is_technical_question(input_text):
        return await SupportAgent().run(input_text)
    else:
        return await ChatAgent().run(input_text)