PyWrapAI — Complete Tutorial

This tutorial takes you from zero to a fully working AI chatbot system step by step. By the end you will have built an application that calls an LLM, assembles conversation history and RAG context, calls tools through a full ReAct agent loop, and tracks the token cost of every call.

What You Are Building

Your App
  │
  ├─ PyWrapAI          ← calls the LLM, tracks tokens, handles retries
  │
  └─ PyWrapAI-Graph    ← assembles the full message list (system prompt + history + query)

The two libraries are installed and used separately. Each has a single, clear responsibility.

Part 1 — PyWrapAI: Calling an LLM

Installation

pip install anthropic          # or: pip install openai / google-generativeai

PyWrapAI itself has no mandatory dependencies — only the provider SDK you choose to use.

Your First Call

from pywrapai import LLM

llm      = LLM(provider="anthropic", model="claude-haiku-4-5-20251001")
response = llm.chat("What is the capital of France?")

print(response.content)
# → Paris is the capital of France.

print(response.usage.input_tokens)   # → 14
print(response.usage.output_tokens)  # → 9
print(response.model)                # → claude-haiku-4-5-20251001

llm.chat() always returns an LLMResponse object with three fields:

.content — the text the model produced
.usage — a TokenUsage with .input_tokens and .output_tokens
.model — the exact model string returned by the API

Switching Providers

The same code works for all four supported providers. Only the constructor changes.

# OpenAI
llm = LLM(provider="openai", model="gpt-4o-mini")

# Anthropic
llm = LLM(provider="anthropic", model="claude-haiku-4-5-20251001")

# Google Gemini
llm = LLM(provider="gemini", model="gemini-2.0-flash")

# Ollama (runs locally — no API key required)
llm = LLM(provider="ollama", model="llama3.2")

Your application code that calls llm.chat() never changes — only the provider.

API Keys

PyWrapAI reads API keys from environment variables automatically.

Provider	Environment variable
OpenAI	`OPENAI_API_KEY`
Anthropic	`ANTHROPIC_API_KEY`
Gemini	`GOOGLE_API_KEY`
Ollama	(none — runs locally)

You can also pass the key directly:

llm = LLM(provider="anthropic", api_key="sk-ant-...")

Token Tracking

Token usage is tracked automatically on every chat() call. Access the tracker through llm.tokens.

llm = LLM(provider="openai", model="gpt-4o-mini")

llm.chat("Tell me about Python.")
llm.chat("And about JavaScript?")

print(llm.tokens.total_input)   # total input tokens across both calls
print(llm.tokens.total_output)  # total output tokens across both calls
print(llm.tokens.total_cost)    # total cost in USD (float)
print(llm.tokens.summary())     # formatted report

llm.tokens.summary() prints:

── Token Usage Summary ──────────────────────────────
  Calls         : 2
  Input tokens  : 47
  Output tokens : 381
  Total tokens  : 428
  Total cost    : $0.000296
────────────────────────────────────────────────────

To see individual call records:

for call in llm.tokens.calls:
    print(call.model, call.input_tokens, call.output_tokens, f"${call.cost:.6f}")

Cost Tracking

Cost is calculated from a built-in pricing table (USD per 1 million tokens). You can override it with a JSON file:

# {
#   "gpt-4o-mini": { "input": 0.15, "output": 0.60 },
#   "my-custom-model": { "input": 1.00, "output": 3.00 }
# }

llm = LLM(provider="openai", model="gpt-4o-mini", pricing="./my_pricing.json")

User prices take priority. Models not in your file keep the built-in defaults. Unknown models return $0.00 — the library never crashes on an unknown model.

You can also call get_cost() directly:

from pywrapai import get_cost

cost = get_cost("gpt-4o-mini", input_tokens=500, output_tokens=200)
print(f"${cost:.6f}")

Multi-Turn Conversations

For a conversation with history, pass a list of Message objects:

from pywrapai import LLM, Message, Role

llm = LLM(provider="anthropic")

messages = [
    Message(Role.USER,      "My name is Alice and I love hiking."),
    Message(Role.ASSISTANT, "That's wonderful, Alice! Where do you hike?"),
    Message(Role.USER,      "What's my name and hobby?"),
]

response = llm.chat(messages)
print(response.content)
# → Your name is Alice and you love hiking!

In practice you will not build this list manually. PyWrapAI-Graph handles that automatically.

System Prompts

A system prompt is a persistent instruction that governs all responses in a conversation. You can set it at LLM creation time:

llm = LLM(
    provider="anthropic",
    system_prompt="You are a helpful customer support agent for Acme Corp. "
                  "Be concise and professional. Never discuss competitors."
)

response = llm.chat("How do I reset my password?")

The system prompt is automatically prepended to every call, in position 0 of the message list.

Prompt Templates

PromptTemplate lets you define reusable prompts with named placeholders:

from pywrapai import LLM, PromptTemplate

llm = LLM(provider="openai")

summarise = PromptTemplate("Summarise the following {document_type} in {language}:\n\n{text}")

response = llm.chat(
    summarise.render(
        document_type = "legal contract",
        language      = "plain English",
        text          = "Whereas the party of the first part..."
    )
)

render() returns a plain string. PromptTemplate raises a ValueError if you forget a variable.

Structured Output

When you need the LLM to return JSON matching a specific schema, use chat_structured(). Pass a Pydantic model class as the schema:

from pydantic import BaseModel
from pywrapai import LLM

class SentimentResult(BaseModel):
    sentiment: str        # "positive", "negative", or "neutral"
    confidence: float     # 0.0 to 1.0
    summary: str

llm    = LLM(provider="anthropic")
result = llm.chat_structured(
    "The new product launch was a disaster. Sales are down 40%.",
    schema=SentimentResult,
)

print(result.sentiment)    # → "negative"
print(result.confidence)   # → 0.95
print(result.summary)      # → "The product launch was unsuccessful..."
print(type(result))        # → <class 'SentimentResult'>

You can also pass a plain dict schema if you do not use Pydantic:

schema = {"name": "string", "year": "integer", "country": "string"}
result = llm.chat_structured("Tell me about Marie Curie", schema=schema)
print(result["name"])    # → "Marie Curie"
print(result["year"])    # → 1867

Under the hood, chat_structured() injects the schema as a JSON instruction into the system message, calls the LLM, strips any markdown fences, parses the JSON, and validates it against the Pydantic model.

Retry and Fallback

Network errors and API timeouts happen. Configure retries and a fallback provider:

from pywrapai import LLM

backup = LLM(provider="anthropic", model="claude-haiku-4-5-20251001")

llm = LLM(
    provider    = "openai",
    model       = "gpt-4o-mini",
    max_retries = 3,        # retry up to 3 times before falling back
    retry_delay = 1.0,      # first retry after 1s, then 2s, then 4s (exponential)
    fallback    = backup,   # if all retries fail, try this LLM instead
)

response = llm.chat("Hello")
# If OpenAI fails 3 times → automatically tries Anthropic

The fallback LLM benefits from its own retry settings too.

Response Caching

Identical requests return the cached response without calling the API. Useful for development and for reducing costs on repeated queries.

llm = LLM(provider="openai", cache=True)

r1 = llm.chat("What is 2+2?")
r2 = llm.chat("What is 2+2?")   # ← returned from cache, no API call

print(llm.cache.size)    # → 1
llm.cache.clear()        # wipe all cached entries

The cache key is a SHA-256 hash of seven call parameters: model name, full message list, temperature, max_tokens, system_prompt, tools, and tool_choice. All seven must match for a cache hit. This means a tool-enabled call and a plain text call with the same prompt are never accidentally served each other’s response. You can also pass a pre-built ResponseCache with a custom capacity:

from pywrapai import ResponseCache

cache = ResponseCache(max_size=500)
llm   = LLM(provider="openai", cache=cache)

Streaming

Stream responses chunk by chunk as they arrive from the API:

llm = LLM(provider="anthropic")

for chunk in llm.stream("Write a short poem about the ocean."):
    print(chunk, end="", flush=True)
print()

Each chunk is a small string. Token usage IS tracked for stream() calls — after the generator is exhausted, the provider reads the final token counts from the response metadata and records them in the TokenTracker.

Async

All async operations use the same methods with an a prefix:

import asyncio
from pywrapai import LLM

llm = LLM(provider="anthropic")

async def main():
    # Async chat
    response = await llm.achat("What is Python?")
    print(response.content)

    # Async streaming
    async for chunk in llm.astream("Tell me a story"):
        print(chunk, end="", flush=True)

asyncio.run(main())

Internally, achat() uses the provider’s native async client (AsyncOpenAI, AsyncAnthropic, client.aio for Gemini, httpx.AsyncClient for Ollama). It never blocks the event loop. All sync features (retry, cache, fallback) work identically in the async path.

Part 2 — PyWrapAI-Graph: Assembling the Full Message

In a real AI application, each LLM call needs:

A system prompt (persisted across the session)
Conversation history (past user + assistant turns)
The current user message

PyWrapAI-Graph assembles all three in the correct order and calls PyWrapAI. Your application code never builds this list manually.

Setting Up

# No extra pip install needed — GraphLLM wraps PyWrapAI's LLM

Create a prompt file and put it in system_prompts_registry/:

system_prompts_registry/
    support-agent-v1.txt
    finance-bot-v2.txt

support-agent-v1.txt:

You are a friendly and professional customer support agent for Acme Corp.
Help users with product questions, account issues, and technical problems.
Always be concise, empathetic, and solution-focused.

GraphLLM — Basic Usage

from pywrapai import LLM
from pywrapai_graph import GraphLLM

llm = LLM(provider="anthropic", model="claude-haiku-4-5-20251001")

graph = GraphLLM(
    llm         = llm,
    enable_system_prompt_versioning = True,
    prompt_file = "support-agent-v1.txt",
)

response = graph.chat("I can't log in to my account.")
print(response.content)

The Message Assembly Order

When you call graph.chat("I can't log in"), GraphLLM builds this exact message list before calling the LLM:

[System Prompt]     ← loaded from support-agent-v1.txt
[History Turn 1]    ← previous user message (if history is provided)
[History Turn 1]    ← previous assistant response
[History Turn N…]
[Current message]   ← "I can't log in"

This is the correct structure that produces coherent, context-aware conversations.

Manual History Injection

from pywrapai import Message, Role

history = [
    Message(Role.USER,      "I ordered a laptop last week."),
    Message(Role.ASSISTANT, "Thank you! I can see your order #12345 for a laptop."),
]

response = graph.chat("Where is my order?", history=history)
# The model knows about order #12345 because history was injected

In production, your application fetches history from a database and passes it here — how you persist and load that history is entirely up to you.

PromptRegistry — Loading Prompt Files

PromptRegistry is what GraphLLM uses internally to load prompts. You can use it directly:

from pywrapai_graph import PromptRegistry

registry = PromptRegistry()
prompt   = registry.load("support-agent-v1.txt")
print(prompt)

available = registry.list_files()
print(available)   # ["finance-bot-v2.txt", "support-agent-v1.txt"]

Token Access via GraphLLM

GraphLLM proxies all LLM properties:

print(graph.tokens.summary())    # same as llm.tokens.summary()
print(graph.model)               # "claude-haiku-4-5-20251001"
print(graph.provider)            # "anthropic"
print(graph.system_prompt)       # loaded prompt text

Part 3 — Tool Calling (Phase 1.6)

Tool calling lets the LLM request that your code run a function and send the result back. This is the foundation for agents — the ReAct loop (Agent class, Phase 3) automates the back-and-forth, but this section shows the explicit flow so you understand what is happening.

Defining Tools

Use the @tool decorator in PyWrapAI-Graph. It reads the function’s type hints and docstring at decoration time — no JSON Schema written by hand.

from pywrapai_graph import ToolRegistry, tool

registry = ToolRegistry()

@tool
def get_weather(city: str) -> str:
    """Get the current weather for a city."""
    # In production, call a real weather API here
    return f"16°C, partly cloudy in {city}"

@tool
def search_docs(query: str, limit: int = 5) -> str:
    """Search the internal knowledge base."""
    return f"Found {limit} results for '{query}'"

registry.register(get_weather)
registry.register(search_docs)

print(registry.names)   # ["get_weather", "search_docs"]

Decorated functions still work as normal callables:

get_weather(city="London")   # "16°C, partly cloudy in London"

Calling the LLM with Tools

Pass registry.to_schema() — the neutral format — to any chat() method. Every provider converts this internally; your code never changes between providers.

from pywrapai import LLM
from pywrapai_graph import GraphLLM

llm   = LLM(provider="anthropic", model="claude-haiku-4-5-20251001")
graph = GraphLLM(llm=llm)

response = graph.chat(
    "What is the weather in London today?",
    tools       = registry.to_schema(),   # ← always use to_schema(), not a provider-specific method
    tool_choice = "auto",                 # let the model decide; "required" forces a tool call
)

Handling the Response

When the model decides to call a tool, response.content is "" and response.tool_calls is a list of ToolCall objects.

from pywrapai import Message, Role, ToolResult

if response.tool_calls:
    tc = response.tool_calls[0]
    print(tc.name)       # "get_weather"
    print(tc.arguments)  # {"city": "London"}

    # Execute the tool
    result = registry.execute(tc)
    print(result.content)   # "16°C, partly cloudy in London"

    # Build the follow-up conversation
    messages = [
        Message(Role.USER,      "What is the weather in London today?"),
        Message(Role.ASSISTANT, "", tool_calls=response.tool_calls),
        Message(Role.TOOL,      result.content,
                tool_result=ToolResult(tc.id, tc.name, result.content)),
    ]

    # Call again — model converts the tool result into a natural reply
    final = graph.chat(messages)
    print(final.content)
    # → "The weather in London is currently 16°C and partly cloudy."
else:
    print(response.content)   # model answered without calling a tool

Tool Calling Through Both Layers

The same tools= parameter works at every level:

# Layer 1 — LLM directly
response = llm.chat("Weather in Paris?", tools=registry.to_schema())

# Layer 2 — GraphLLM (system prompt + history + tools)
response = graph.chat("Weather in Paris?", tools=registry.to_schema())

stream() Does Not Support tools=

Passing tools= to stream() or GraphLLM.stream() raises ValueError. Tool-enabled conversations always use chat(). This is by design — streaming interleaved tool-use deltas requires delta assembly logic that belongs in the Phase 3 agent loop.

# This raises ValueError:
for chunk in graph.stream("Weather?", tools=registry.to_schema()):
    ...

# Correct for tool-enabled conversations:
response = graph.chat("Weather?", tools=registry.to_schema())

Error Handling in Tools

If your tool function raises an exception, registry.execute() catches it and returns the exception message as ToolResult.content instead of crashing. The model sees the error and can retry with different arguments or ask for clarification.

@tool
def lookup_order(order_id: str) -> str:
    """Look up an order by ID."""
    if not order_id.startswith("ORD-"):
        raise ValueError(f"Invalid order ID format: {order_id}")
    return f"Order {order_id}: in transit, arriving Thursday."

result = registry.execute(ToolCall(id="c1", name="lookup_order", arguments={"order_id": "12345"}))
print(result.content)   # "Error: Invalid order ID format: 12345"
# The model sees this and can ask the user to provide a valid order ID.

Part 4 — RAG Pipeline: Giving the LLM a Knowledge Base

RAG (Retrieval-Augmented Generation) lets the LLM answer questions from your own documents. Before each LLM call, the most relevant passages are retrieved and injected into the user’s message automatically.

What You Need

pip install openai numpy   # Embedder uses openai; InMemoryVectorStore uses numpy

numpy is only required for InMemoryVectorStore. If you implement a custom BaseVectorStore that does not use numpy, the dependency is optional.

Step 1 — Build the Knowledge Base

from pywrapai       import Embedder
from pywrapai_graph import (
    Document, Chunker,
    InMemoryVectorStore, VectorRetriever,
)

# 1a. Create the embedder (calls OpenAI embed API)
embedder = Embedder(provider="openai", model="text-embedding-3-small")

# 1b. Create the store and retriever
store     = InMemoryVectorStore()
retriever = VectorRetriever(embedder=embedder, store=store, k=4)

# 1c. Add your documents
docs = [
    Document("Paris is the capital of France, founded in 987 AD."),
    Document("The Eiffel Tower is 330 metres tall and located in Paris."),
    Document("France adopted the Euro as its currency in 2002."),
    Document("The Louvre museum is the world's largest art museum."),
]

chunker = Chunker(chunk_size=500, overlap=50)
n = retriever.add_documents(docs, chunker=chunker)
print(f"Added {n} chunks to the store.")   # → Added 4 chunks to the store.

add_documents() does three things in one call:

Splits each Document into Chunk objects using the Chunker
Embeds all chunks in a single batched API call
Adds the embedded chunks to the store

Choosing a Vector Store Backend

InMemoryVectorStore is the simplest option — no dependencies, no disk I/O — but vectors are lost when the process exits. For production use, swap it for a persistent backend. All backends implement BaseVectorStore so swapping is a one-line change.

from pywrapai_graph import (
    InMemoryVectorStore,   # RAM only — no extra deps
    SQLiteVectorStore,     # persisted to a .db file — no extra deps
    ChromaVectorStore,     # HNSW ANN — pip install chromadb
    FAISSVectorStore,      # high-perf index files — pip install faiss-cpu
    PGVectorStore,         # PostgreSQL + pgvector — pip install psycopg2-binary pgvector
    QdrantVectorStore,     # Qdrant :memory:/disk/Docker/Cloud — pip install qdrant-client
)

# SQLite — zero extra deps, survives restarts
store = SQLiteVectorStore("./knowledge_base.db")

# Chroma — embedded HNSW, millions of vectors
store = ChromaVectorStore(path="./chroma_db", collection="pywrapai_docs")

# FAISS — two files on disk, exact cosine search, ~1M vectors without approximation
store = FAISSVectorStore(index_path="./docs.faiss", meta_path="./docs_meta.pkl")

# pgvector — PostgreSQL, server-side HNSW, Stage 2 production target
store = PGVectorStore(connection_string="postgresql://user:pass@localhost:5432/mydb")

# Qdrant — four deployment modes via the same API
store = QdrantVectorStore(collection="docs", location=":memory:")         # in-process
store = QdrantVectorStore(collection="docs", location="./qdrant_data")    # local disk
store = QdrantVectorStore(collection="docs", url="http://localhost:6333") # Docker

Pass the store to VectorRetriever exactly as you would InMemoryVectorStore:

retriever = VectorRetriever(embedder=embedder, store=store, k=4)
retriever.add_documents(docs, chunker=Chunker(chunk_size=400, overlap=50))

Important — Qdrant local disk mode: Qdrant holds an exclusive file lock on the data directory. Call store.close() before opening a second instance pointing to the same path.

Important — PGVectorStore tests: Live tests require a running PostgreSQL server. Set PYWRAPAI_TEST_PG_URL=postgresql://user:pass@host/db to enable them; otherwise they are skipped.

Step 2 — Query with GraphLLM

from pywrapai       import LLM
from pywrapai_graph import GraphLLM

llm   = LLM(provider="openai")
graph = GraphLLM(
    llm       = llm,
    retriever = retriever,   # ← enables RAG
    rag_k     = 3,           # retrieve up to 3 chunks per query
)

response = graph.chat("What is the capital of France?")
print(response.content)
# → "The capital of France is Paris, which was founded in 987 AD."

Behind the scenes, graph.chat() does:

Embeds the query "What is the capital of France?" via the retriever’s embedder
Searches the store for the top 3 most similar chunks
Formats the chunks into the context template and replaces the user message
Calls llm.chat() with the augmented message list

Step 3 — Combine with Manual History

RAG and conversation history compose the same way any other graph.chat() call does — pass history= and PyWrapAI-Graph injects the retrieved context into just the current turn:

history: list[Message] = []

r1 = graph.chat("What currency does France use?", history=history)
history += [Message(Role.USER, "What currency does France use?"),
            Message(Role.ASSISTANT, r1.content)]

r2 = graph.chat("Tell me more about it.", history=history)   # knows this is about France

print(r1.content)   # "France uses the Euro, adopted in 2002."

Whatever you persist to your own store, save only the original user text — never the <context> block:

Turn 1 user: "What currency does France use?" (NOT the <context> block)
Turn 1 assistant: "France uses the Euro, adopted in 2002."
Turn 2 user: "Tell me more about it."

Step 4 — Disable RAG for Specific Calls

# Greeting — no retrieval needed
response = graph.chat("Hello!", rag=False)

# Normal query — RAG active
response = graph.chat("What museums are in Paris?")

Pass rag=False to skip retrieval for messages that don’t need document context (greetings, confirmations, administrative messages).

Step 5 — Use the Retriever as a Tool (Phase 3 bridge)

from pywrapai_graph import retrieval_tool, ToolRegistry

# Expose the retriever as a callable tool for the agent loop
registry = ToolRegistry()
t = retrieval_tool(
    retriever   = retriever,
    name        = "search_knowledge_base",
    description = "Search the knowledge base for relevant context.",
)
registry.register(t)

# The LLM can now call "search_knowledge_base" as a function
response = graph.chat(
    "What do you know about Eiffel Tower height?",
    tools = registry.to_schema(),
)

This is useful when you want the LLM to decide when to retrieve, rather than always injecting context. The Phase 3 Agent class will drive this loop automatically.

Checking Embedding Cost

# Embedding cost is tracked separately from chat cost
print(f"Embed cost: ${embedder.tokens.total_cost:.6f}")
print(f"Chat cost:  ${llm.tokens.total_cost:.6f}")
print(f"Total:      ${embedder.tokens.total_cost + llm.tokens.total_cost:.6f}")

Using a Custom Embedder Provider

# Gemini embeddings (free tier)
embedder = Embedder(provider="gemini", model="models/text-embedding-004")

# Ollama local embeddings (zero cost, requires ollama serve)
embedder = Embedder(provider="ollama", model="nomic-embed-text")

All three providers produce vectors. Only the dimension differs (OpenAI: 1536, Gemini: 768, Ollama: varies by model). The InMemoryVectorStore handles any dimension — do not mix embedder providers within the same store.

Note for Anthropic users: Anthropic does not offer a public embedding API. If your whole stack uses Anthropic for chat, you still need a separate provider for embedding. OpenAI text-embedding-3-small ($0.02/M tokens) requires an OPENAI_API_KEY, or use local Ollama nomic-embed-text at zero cost with no API key. Passing provider="anthropic" to Embedder raises InvalidRequestError immediately.

Part 5 — Putting It All Together

Here is a complete working script using both libraries, with the application managing its own in-memory history:

import os
from pywrapai        import LLM, Message, Role
from pywrapai_graph  import GraphLLM

# ── 1. Create the LLM engine ───────────────────────────────────────────────
llm = LLM(
    provider    = "anthropic",
    model       = "claude-haiku-4-5-20251001",
    max_retries = 2,
)

# ── 2. Wrap with GraphLLM (adds system prompt + history assembly) ──────────
graph = GraphLLM(
    llm         = llm,
    enable_system_prompt_versioning = True,
    prompt_file = "support-agent-v1.txt",
)

# ── 3. Chat — the app owns the history list ────────────────────────────────
history: list[Message] = []

while True:
    user_input = input("You: ").strip()
    if not user_input:
        continue
    if user_input.lower() == "quit":
        break

    response = graph.chat(user_input, history=history)
    print(f"Bot: {response.content}")
    print(f"     [{response.usage.input_tokens} in / {response.usage.output_tokens} out]")

    history.append(Message(Role.USER, user_input))
    history.append(Message(Role.ASSISTANT, response.content))

Each turn:

GraphLLM prepends the system prompt from support-agent-v1.txt
GraphLLM inserts history between the system prompt and the current message
LLM calls the Anthropic API with the full assembled message list
The app appends the new user message and response to history itself

For anything beyond a single process — a web backend, multiple users, restarts — you’ll want to persist history somewhere (a database, Redis, a file) keyed by whatever identifies the conversation, and load it back before each graph.chat() call.

Part 6 — Complete Project Structure

my-ai-app/
├── system_prompts_registry/
│   ├── support-agent-v1.txt
│   └── finance-bot-v1.txt
├── main.py                      ← your application
└── .env
    ANTHROPIC_API_KEY=sk-ant-...

Loading .env in Your App

# At the top of main.py, before any LLM code
from pathlib import Path
import os

for line in Path(".env").read_text().splitlines():
    line = line.strip()
    if line and not line.startswith("#") and "=" in line:
        k, _, v = line.partition("=")
        os.environ.setdefault(k.strip(), v.strip())

Part 7 — Multi-User Applications

The library design scales directly to multi-user web applications. Each user gets their own user_id, and each conversation gets its own session_id — you decide how those map to stored history.

# In a FastAPI or Flask endpoint:
from pywrapai         import LLM
from pywrapai_graph   import GraphLLM

# Create ONCE at startup — shared across ALL requests
# LLM and GraphLLM are stateless at call-time: the cache and tracker
# accumulate across users, which is exactly what you want.
llm   = LLM(provider="anthropic", model="claude-haiku-4-5-20251001", cache=True)
graph = GraphLLM(llm=llm, enable_system_prompt_versioning=True, prompt_file="support-agent-v1.txt")

def handle_message(user_id: str, session_id: str, message: str) -> str:
    # load_history() is your own function — a DB call, a Redis lookup, etc.
    history  = load_history(user_id, session_id, max_turns=10)
    response = graph.chat(message, history=history)
    save_history(user_id, session_id, message, response.content)
    return response.content

Key conversations by (user_id, session_id) in whatever store you use. Different users never see each other’s history. Multiple sessions for the same user are also isolated.

Part 8 — Phase 3: The Agent ReAct Loop

Parts 1–4 showed how to call an LLM, assemble messages, and add RAG. This part shows how to wire the full ReAct (Reason + Act) loop so the model can call tools automatically, see the results, and keep reasoning until it has a complete answer — all in one agent.run() call.

What the Agent Does

Without the Agent, you wrote the tool loop manually:

1. response = llm.chat(..., tools=schema)
2. if response.tool_calls: result = registry.execute(tc)
3. append result messages
4. response = llm.chat(messages_with_tool_results)
5. repeat until plain text

Agent.run() does all of that automatically.

Installation — no new dependencies

The Agent class is part of pywrapai_graph. No extra packages.

Define Your Tools

from pywrapai_graph import tool, ToolRegistry

registry = ToolRegistry()

@tool
def get_weather(city: str) -> str:
    """Get the current weather for a city."""
    # In a real app: call a weather API
    return f"16°C, cloudy in {city}"

@tool
def search_docs(query: str, limit: int = 5) -> str:
    """Search internal documentation for a query."""
    return f"Found {limit} results for '{query}': ..."

registry.register(get_weather)
registry.register(search_docs)

Build the Agent

from pywrapai       import LLM
from pywrapai_graph import GraphLLM, Agent, AgentConfig

llm   = LLM(provider="openai", model="gpt-4o-mini")
graph = GraphLLM(llm=llm).with_system_prompt(
    "You are a helpful assistant. Always call a tool to look up real data before answering."
)

config = AgentConfig(
    max_turns         = 8,
    tool_error_policy = "return_to_model",  # model sees errors and can recover
)

agent = Agent(graph=graph, registry=registry, config=config)

GraphLLM(llm=llm).with_system_prompt(text) is the correct construction pattern. There is no system_prompt= kwarg in the GraphLLM constructor.

Run the Agent

from pywrapai_graph import AgentResult, AgentMaxTurnsError

try:
    result: AgentResult = agent.run("What's the weather in London and Paris?")

    print(result.content)
    # → The weather in London is 16°C and cloudy. In Paris it is 14°C and sunny.

    print(f"Turns: {result.total_turns}")         # → 2 (1 tool turn + 1 answer turn)
    print(f"Cost:  ${result.total_cost_usd:.6f}") # → $0.000145

    # Inspect each tool-calling turn
    for step in result.steps:
        print(f"  Turn {step.turn_num}: {len(step.tool_calls)} tool call(s)")
        for tc in step.tool_calls:
            print(f"    {tc.name}({tc.arguments})")

except AgentMaxTurnsError as exc:
    print(f"Stopped after {exc.turns} turns — partial cost ${exc.partial.total_cost_usd:.6f}")

Application-Level Logging with `on_step`

Pass a callback to AgentConfig to see exactly what the agent does on each turn. This is the right place for logging — not inside library code.

import logging

logger = logging.getLogger("myapp.agent")

def log_step(step):
    for tc in step.tool_calls:
        args_str = ", ".join(f"{k}={v!r}" for k, v in tc.arguments.items())
        logger.info("[TOOL CALLED]  %s(%s)", tc.name, args_str)
    for tr in step.tool_results:
        logger.info("[TOOL RESULT]  %s → %s", tr.name, tr.content[:200])
    logger.info(
        "[STEP %d DONE]  in=%d  out=%d  cost=$%.6f",
        step.turn_num, step.usage.input_tokens, step.usage.output_tokens, step.cost_usd,
    )

config = AgentConfig(max_turns=8, on_step=log_step)
agent  = Agent(graph=graph, registry=registry, config=config)

The callback fires after each tool-calling turn completes. All fields of AgentStep are already set: tool_results, usage, cost_usd. The callback runs synchronously before the loop continues.

Async Agent

import asyncio

async def main():
    result = await agent.arun("What are the top sellers this month?")
    print(result.content)

asyncio.run(main())

arun() uses graph.achat() internally. When the model requests multiple tools in one response, they are dispatched concurrently with asyncio.gather.

Combining Agent with RAG

The retrieval_tool() helper wraps any BaseRetriever as a Tool, so the agent can decide when to search the knowledge base rather than always retrieving:

from pywrapai_graph import retrieval_tool

search_kb = retrieval_tool(
    retriever   = my_retriever,
    name        = "search_knowledge_base",
    description = "Search the internal knowledge base for relevant facts.",
)

registry.register(search_kb)
# The model now calls "search_knowledge_base" when it needs document context

This is more powerful than automatic RAG injection (GraphLLM(retriever=...)) because the model decides which query to use — not just the verbatim user question.

What Comes Next

Task	Path
Add a new AI persona	Create a new `.txt` file in `system_prompts_registry/`
Change the LLM model	Change `model=` in `LLM()`
Add a second provider	`LLM(provider="openai")` — the rest of the code is unchanged
Reduce context cost	Load fewer turns of history before calling `chat()`
Override model pricing	Pass `pricing="./my_pricing.json"` to `LLM()`
Cache repeated queries	Pass `cache=True` to `LLM()`
Add retry resilience	Pass `max_retries=3, fallback=backup_llm` to `LLM()`
Add tools to the LLM	`@tool` + `ToolRegistry` + `chat(tools=registry.to_schema())`
Force a specific tool	Pass `tool_choice="my_tool_name"` to `chat()`
Add document search (RAG)	See Part 4 — `Embedder` + `VectorRetriever` + `GraphLLM(retriever=...)`
Wrap a retriever as a tool	`retrieval_tool(retriever, name="search_kb")` + `ToolRegistry.register()`
Skip RAG for a specific call	Pass `rag=False` to `chat()`
Automate the tool loop	See Part 8 — `Agent` + `AgentConfig` + `agent.run()`
Change vector store backend	Swap `InMemoryVectorStore` for `SQLiteVectorStore`, `QdrantVectorStore`, etc. — same `BaseRetriever` interface