PyWrapAI — Library Reference

Single responsibility: Call an LLM and return a response.

PyWrapAI does two things: call an LLM and embed text. It provides a single unified interface to call any supported LLM provider and to produce embeddings from any supported embedding provider, and automatically tracks token usage and cost for both. It does not do RAG pipelines, agents, conversation history, vector stores, or document chunking — those live in PyWrapAI-Graph. Tool calling (passing tools= to the LLM and parsing the response) is handled here. Tool execution and the agent loop live in PyWrapAI-Graph.

---

Public API

Everything you need is exported from the top-level package:

from pywrapai import (
    LLM,             # main entry point — call an LLM
    Embedder,        # embed text (Phase 2) — openai/gemini/ollama
    Message,         # a single message in a conversation
    Role,            # enum: USER, ASSISTANT, SYSTEM, TOOL
    LLMResponse,     # what every chat() call returns
    TokenUsage,      # input_tokens + output_tokens inside LLMResponse
    ToolCall,        # a tool call requested by the LLM in its response
    ToolResult,      # your code's result, returned to the LLM
    TokenTracker,    # accumulated usage across multiple calls
    CallRecord,      # single call record inside TokenTracker.calls
    PromptTemplate,  # reusable prompts with {placeholders}
    BaseCache,       # abstract base — implement this for custom cache backends
    ResponseCache,   # built-in in-memory LRU cache (implements BaseCache)
    get_cost,        # calculate USD cost for a given token count
    load_user_pricing, # load a custom pricing JSON file
    # exceptions
    PyWrapAIError,          # base class for all library errors
    ProviderError,          # base class for all provider API errors
    AuthError,              # 401 — invalid API key
    RateLimitError,         # 429 — quota or rate limit hit
    InvalidRequestError,    # 400 — malformed request
    ServerError,            # 5xx — provider-side failure
    ProviderTimeoutError,   # connection timeout or deadline exceeded
)

---

Core Data Types

Role

from pywrapai import Role

Role.USER       # the human's message
Role.ASSISTANT  # the model's reply
Role.SYSTEM     # instruction that governs the whole conversation
Role.TOOL       # tool result sent back to the model (Phase 1)

Role is a Python Enum. Its .value is the lowercase string used by the provider APIs ("user", "assistant", "system", "tool").

Message

A single message in a conversation.

from pywrapai import Message, Role

msg = Message(role=Role.USER, content="Hello!")
msg = Message(Role.ASSISTANT, "Hi! How can I help?")   # positional also works

Fields:

• role: Role — who sent this message
• content: str — the text content
• tool_calls: list[ToolCall] | None — set when role is ASSISTANT and the model is requesting tool execution. None in all other messages.
• tool_result: ToolResult | None — set when role is TOOL and your code is returning a tool’s output to the model. None in all other messages.

Both tool_calls and tool_result default to None — all existing code that creates Message(role, content) continues to work unchanged.

Message is a dataclass. It is immutable by design — never modify a message after creation.

TokenUsage

Returned inside every LLMResponse.

usage = response.usage
print(usage.input_tokens)   # prompt tokens consumed
print(usage.output_tokens)  # completion tokens generated
print(usage.total_tokens)   # input + output

Token counts come directly from the API response. PyWrapAI never estimates or counts tokens manually.

LLMResponse

What every successful chat() or chat_structured() call returns.

response = llm.chat("What is Python?")

response.content          # str — the model's reply text (empty string when tool_calls is set)
response.usage            # TokenUsage
response.usage.input_tokens   # int
response.usage.output_tokens  # int
response.model            # str — e.g. "claude-haiku-4-5-20251001"
response.tool_calls       # list[ToolCall] | None — set when the model wants to call a tool
response.raw              # the raw provider response object (for advanced use)

When tool_calls is not None, the model has decided to call a function instead of replying with text. content will be an empty string in that case. The tool execution loop (Phase 1.2) reads tool_calls, runs each function, and sends the results back to continue the conversation.

ToolCall

Represents a function call requested by the LLM in its response.

from pywrapai import ToolCall

# You do not create ToolCall objects directly.
# They arrive inside LLMResponse.tool_calls after a chat() call.
tc = response.tool_calls[0]

tc.id          # str  — unique ID assigned by the API, e.g. "call_abc123"
tc.name        # str  — the function to call, e.g. "get_weather"
tc.arguments   # dict — arguments already parsed from JSON, e.g. {"city": "London"}

ToolResult

Wraps your function’s return value so it can be sent back to the LLM.

from pywrapai import ToolResult, Message, Role

# You run the function, then wrap the result
result_text = get_weather(city="London")   # "16°C, partly cloudy"

tool_result_msg = Message(
    role        = Role.TOOL,
    content     = result_text,
    tool_result = ToolResult(
        tool_call_id = tc.id,    # must match the ToolCall.id above
        name         = tc.name,  # "get_weather"
        content      = result_text,
    ),
)

The tool calling conversation flow (implemented by PyWrapAI-Graph in Phase 1.2):

1. llm.chat(messages)
   → response.tool_calls = [ToolCall(id="call_1", name="get_weather", arguments={"city": "London"})]

2. Your code runs: result = get_weather(city="London")  → "16°C, partly cloudy"

3. Append ToolResult as a TOOL message and call llm.chat() again:
   messages.append(Message(role=Role.ASSISTANT, content="", tool_calls=response.tool_calls))
   messages.append(Message(role=Role.TOOL, content=result,
                           tool_result=ToolResult("call_1", "get_weather", result)))
   final_response = llm.chat(messages)

4. final_response.content → "The weather in London is 16°C and partly cloudy."

PyWrapAI provides the data types. The loop that drives steps 1–4 automatically is PyWrapAI-Graph’s Agent class (Phase 3).

---

LLM — Constructor Parameters

llm = LLM(
    provider      = "anthropic",                # required — see Providers section
    model         = "claude-haiku-4-5-20251001", # optional — uses provider default
    api_key       = None,                       # reads from env var if not set
    base_url      = None,                       # override the provider's API base URL
    temperature   = 0.7,                        # 0.0 = deterministic, 1.0 = creative
    max_tokens    = None,                       # cap on output tokens (provider default if None)
    system_prompt = None,                       # prepended to every call automatically
    track_tokens  = True,                       # set False to disable TokenTracker
    pricing       = None,                       # path to a JSON pricing config file
    max_retries   = 0,                          # retry on failure (0 = no retries)
    retry_delay   = 1.0,                        # base seconds between retries (doubles each attempt)
    fallback      = None,                       # another LLM to try if all retries fail
    cache         = False,                      # True or a BaseCache instance to enable caching
)

Provider Defaults

Provider	Default Model
openai	gpt-4o-mini
anthropic	claude-haiku-4-5-20251001
gemini	gemini-2.0-flash-lite
ollama	llama3.2

---

LLM.chat() — Synchronous Chat

response = llm.chat(input, tools=None, tool_choice=None)

Parameters:

• input: str | list[Message] — a plain string (converted to a single user message) or a list of Message objects for multi-turn conversations.
• tools: list[dict] | None — provider-neutral tool definitions from ToolRegistry.to_schema(). Each provider converts this to its own wire format internally. Default None.
• tool_choice: str | None — one of "auto" (model decides), "required" (must call a tool), "none" (never call a tool), or a specific tool name to force. Only meaningful when tools is also set. Default None.

Returns: LLMResponse. When the model calls a tool, response.content is "" and response.tool_calls is a non-empty list.

Behaviour:

1. Checks the cache (keyed on all seven call parameters including tools and tool_choice). Returns the cached response if found.
2. Builds the full message list (prepends system prompt if set).
3. Calls the provider with retry + fallback logic.
4. Records token usage in TokenTracker.
5. Stores the response in cache.
6. Returns LLMResponse.

Examples:

# Single string — simplest form
response = llm.chat("What is machine learning?")

# Multi-turn conversation
from pywrapai import Message, Role

response = llm.chat([
    Message(Role.USER,      "My name is Alice."),
    Message(Role.ASSISTANT, "Hello Alice!"),
    Message(Role.USER,      "What is my name?"),
])
print(response.content)  # → Your name is Alice.

# Tool calling — use registry.to_schema() for the neutral format
from pywrapai_graph import ToolRegistry, tool

registry = ToolRegistry()

@tool
def get_weather(city: str) -> str:
    """Get the current weather for a city."""
    return f"16°C, cloudy in {city}"

registry.register(get_weather)

response = llm.chat("What's the weather in London?", tools=registry.to_schema())
if response.tool_calls:
    tc     = response.tool_calls[0]
    result = registry.execute(tc)   # ToolResult
    # Build the follow-up messages and call again...

---

LLM.stream() — Streaming

for chunk in llm.stream("Write a poem about the sea."):
    print(chunk, end="", flush=True)

Parameters: Same input as chat(). Does not accept tools= — passing it raises ValueError. Agent loops that need tool calling must use chat(). Streaming with interleaved tool-use deltas is deferred to Phase 3.

Yields: str — small text fragments as they arrive from the API.

Token tracking: Token usage IS tracked for stream() calls. After the generator is exhausted, the provider reads token counts from the final chunk’s metadata and records them in the TokenTracker. Counts will be 0 only if the stream was interrupted before completion.

---

LLM.chat_structured() — JSON Output

Forces the model to return valid JSON matching a schema. Where the provider supports it, the library uses the provider’s native JSON mode for a guaranteed clean response — no markdown fences, no stray text. Where native mode is not available, it falls back to schema-injection into the system prompt.

result = llm.chat_structured(input, schema=MyModel)

Parameters:

• input: str | list[Message] — the question or prompt
• schema: type | dict — a Pydantic BaseModel class or a plain dict

Returns: A validated Pydantic model instance (if schema is a class) or a parsed dict.

How it works:

1. The JSON schema is extracted from the Pydantic model (or from the dict directly).
2. A schema instruction is injected into the system message: “You must respond ONLY with valid JSON that matches this schema.” (done regardless of provider — tells the model what structure to produce).
3. The provider’s chat_structured() method is called. Each provider either uses its native JSON mode or returns None to signal no native support.
4. If None: falls back to a regular chat() call with the injected instruction. Any markdown code fences are stripped from the response.
5. The response text is parsed as JSON (fence stripping runs as a safety net even in native mode).
6. If a Pydantic class was passed: the result is validated and returned as a model instance.

Provider support table:

Provider	Native JSON Mode	API Parameter Used
openai	✅ Yes	response_format={"type": "json_object"}
gemini	✅ Yes	response_mime_type="application/json"
ollama	✅ Yes	"format": "json" in the request payload
anthropic	❌ No — prompt injection fallback	schema injected in system message

Native mode guarantees the provider outputs only valid JSON, with no surrounding text or markdown. Prompt injection relies on the model following the instruction — it works well for capable models but is less reliable with smaller or older models.

With Pydantic:

from pydantic import BaseModel
from pywrapai import LLM

class ExtractedFact(BaseModel):
    subject: str
    predicate: str
    object: str

llm    = LLM(provider="openai")
result = llm.chat_structured(
    "Marie Curie discovered polonium.",
    schema=ExtractedFact,
)

print(result.subject)    # → "Marie Curie"
print(result.predicate)  # → "discovered"
print(result.object)     # → "polonium"

With a plain dict:

schema = {"country": "string", "population": "integer", "capital": "string"}
result = llm.chat_structured("Tell me about France", schema=schema)
print(result["capital"])  # → "Paris"

---

LLM.achat() and LLM.astream() — Async

import asyncio
from pywrapai import LLM

llm = LLM(provider="anthropic")

async def main():
    # Async chat — same as chat() but awaitable; accepts tools= and tool_choice=
    response = await llm.achat("What is Python?")
    print(response.content)

    # Async chat with tools
    response = await llm.achat(
        "What's the weather in London?",
        tools       = registry.to_schema(),
        tool_choice = "auto",
    )

    # Async streaming — does NOT accept tools=
    async for chunk in llm.astream("Tell me a story"):
        print(chunk, end="", flush=True)

asyncio.run(main())

Implementation details:

• achat() accepts the same tools= and tool_choice= parameters as chat(). Uses the provider’s native async client (AsyncOpenAI, AsyncAnthropic, client.aio for Gemini, httpx.AsyncClient for Ollama). It runs an async retry loop (_do_achat()) with await asyncio.sleep() between attempts, so it never blocks the event loop.
• astream() does not accept tools= — same restriction as stream(). Calls the provider’s native async generator directly. After the stream ends, it reads _last_stream_usage from the provider and records token usage in the TokenTracker.
• All async features (retry, cache, fallback) work identically to their sync counterparts.

Gemini event loop note: The Gemini client.aio namespace binds its internal httpx connections to the active event loop. In production (FastAPI, aiohttp) there is one persistent event loop — no issue. In scripts, run all async calls on a single Gemini LLM instance inside one asyncio.run() call rather than calling asyncio.run() multiple times on the same instance.

---

Token Tracking

TokenTracker

Accessed via llm.tokens. Records every chat() and chat_structured() call.

llm = LLM(provider="openai")
llm.chat("Hello")
llm.chat("Tell me about Python")

tracker = llm.tokens

tracker.total_input    # int — total input tokens across all calls
tracker.total_output   # int — total output tokens across all calls
tracker.total_used     # int — total_input + total_output
tracker.total_cost     # float — total cost in USD
tracker.calls          # list[CallRecord] — one per call

tracker.reset()        # clear all records
tracker.summary()      # formatted string report

CallRecord

Each entry in tracker.calls:

for record in llm.tokens.calls:
    record.model          # str — model name
    record.input_tokens   # int
    record.output_tokens  # int
    record.cost           # float — USD
    record.total_tokens   # int — input + output
    record.timestamp      # datetime — UTC
    record.kind           # str — "chat" (default) or "embedding"

kind distinguishes chat calls from embedding calls in the same TokenTracker:

• "chat" — set by every LLM.chat() and LLM.achat() call
• "embedding" — set by every Embedder.embed() and Embedder.aembed() call

This allows TokenTracker.summary() to split the report by call type and lets analytics code filter tracker.calls by kind without ambiguity.

Disabling Tracking

llm = LLM(provider="openai", track_tokens=False)
llm.tokens   # raises RuntimeError

---

Pricing

Built-in Price Table

The library ships with hardcoded prices for all supported models (USD per 1 million tokens):

Chat models (USD per 1 million tokens):

Model	Input	Output
gpt-4o	$2.50	$10.00
gpt-4o-mini	$0.15	$0.60
claude-opus-4-8	$15.00	$75.00
claude-sonnet-4-6	$3.00	$15.00
claude-haiku-4-5-20251001	$0.80	$4.00
gemini-2.0-flash	$0.10	$0.40
gemini-1.5-flash	$0.075	$0.30
ollama (any local model)	$0.00	$0.00

Embedding models (USD per 1 million input tokens; output is always 0):

Model	Input
text-embedding-3-small	$0.02
text-embedding-3-large	$0.13
text-embedding-ada-002	$0.10
models/text-embedding-004 (Gemini)	$0.00
nomic-embed-text (Ollama)	$0.00

Unknown models return $0.00 — the library never raises an error for an unknown model. It does emit a logging.warning() so developers immediately see when a model is unpriced, rather than silently losing cost data in the analytics dashboard.

Custom Pricing

my_pricing.json

{
    "gpt-4o-mini": { "input": 0.15, "output": 0.60 },
    "my-fine-tuned-model": { "input": 1.50, "output": 4.00 }
}

llm = LLM(provider="openai", pricing="./my_pricing.json")

Or load globally:

from pywrapai import load_user_pricing
load_user_pricing("./my_pricing.json")
# All LLM instances now use the updated prices

get_cost()

Calculate cost without making an LLM call:

from pywrapai import get_cost

cost = get_cost("gpt-4o-mini", input_tokens=1000, output_tokens=500)
print(f"${cost:.6f}")  # → $0.000450

---

Prompt Templates

from pywrapai import LLM, PromptTemplate

llm = LLM(provider="openai")

translate = PromptTemplate("Translate the following text to {language}:\n\n{text}")

response = llm.chat(
    translate.render(language="Spanish", text="Hello, how are you?")
)

render() raises ValueError if any placeholder is missing or an unexpected key is passed.

Multi-message templates — render() returns a plain string, which chat() converts to a single user message. For more complex templates, build the message list yourself.

---

Response Caching

from pywrapai import LLM, ResponseCache

# Simple: enable with True (default capacity = 1000 entries)
llm = LLM(provider="openai", cache=True)

# Advanced: pass a configured instance
cache = ResponseCache(max_size=500)
llm   = LLM(provider="openai", cache=cache)

# Check cache state
print(llm.cache.size)    # number of cached entries
llm.cache.clear()        # remove all entries

Cache key: SHA-256 hash of all seven call parameters: {model, messages, temperature, max_tokens, system_prompt, tools, tool_choice} serialised as JSON. All seven must match for a cache hit. This means a tool-enabled call and a plain text call with the same prompt produce different cache keys — they are never accidentally served each other’s response.

Eviction: LRU — when the cache is full, the least-recently-used entry is dropped. Accessing an entry promotes it to most-recently-used. The backing store is an OrderedDict.

Thread safety: All cache operations are protected by a threading.Lock, so ResponseCache is safe to share across threads.

Backends: ResponseCache inherits from BaseCache. Future backends (SQLiteCache, RedisCache) implement the same interface and can be passed directly to LLM(cache=...).

Current limitation: In-memory only — does not survive process restarts. A SQLiteCache backend (Stage 1) and RedisCache backend (Stage 2) are planned.

---

Retry and Fallback

backup = LLM(provider="anthropic", model="claude-haiku-4-5-20251001")

llm = LLM(
    provider    = "openai",
    model       = "gpt-4o-mini",
    max_retries = 3,
    retry_delay = 1.0,    # 1s → 2s → 4s (exponential backoff)
    fallback    = backup,
)

Retry flow:

1. Attempt 1 → fails → wait 1.0s (only if error is retryable: RateLimitError, ServerError, ProviderTimeoutError)
2. Attempt 2 → fails → wait 2.0s
3. Attempt 3 → fails → wait 4.0s
4. Attempt 4 (final) → fails → fall through to fallback
5. If fallback is set → call fallback._do_chat(messages)
6. If fallback also fails → raise the last exception

Non-retryable errors (AuthError, InvalidRequestError) stop the retry loop immediately and jump straight to the fallback. This means the fallback is always tried on any primary failure, not just exhausted retries.

The fallback LLM also benefits from its own max_retries and retry_delay settings.

---

Providers

Design: Lazy Imports

All provider SDK imports happen inside method bodies, not at the top of the file. This means:

• import anthropic only runs when you first call a method on the Anthropic provider
• Installing anthropic is only required if you use provider="anthropic"
• No import errors at library load time for SDKs you have not installed

OpenAI

llm = LLM(provider="openai", model="gpt-4o-mini")

Reads OPENAI_API_KEY from environment. Supports all chat(), stream(), chat_structured(), and async variants.

Anthropic

llm = LLM(provider="anthropic", model="claude-haiku-4-5-20251001")

Reads ANTHROPIC_API_KEY from environment. Anthropic requires the system prompt as a separate system= parameter, not inside the messages array. The provider handles this conversion automatically — application code never needs to know about it.

Google Gemini

llm = LLM(provider="gemini", model="gemini-2.0-flash")

Reads GOOGLE_API_KEY from environment. Uses google-genai SDK (from google import genai).

Ollama (Local)

llm = LLM(provider="ollama", model="llama3.2")

No API key required. Requires Ollama to be running locally (ollama serve). Uses httpx for HTTP calls instead of a dedicated SDK. All calls go to http://localhost:11434 by default.

---

BaseLLM — Adding a Custom Provider

All providers implement BaseLLM. Four methods are required; one is optional.

from pywrapai.providers.base import BaseLLM
from pywrapai.core.base import LLMResponse, Message, TokenUsage
from typing import AsyncIterator, Iterator, Optional

class MyProvider(BaseLLM):
    default_model = "my-model-v1"

    # ── required: four abstract methods ───────────────────────────────────

    def chat(
        self,
        messages:    list[Message],
        tools:       Optional[list[dict]] = None,
        tool_choice: Optional[str]        = None,
    ) -> LLMResponse:
        # Convert neutral tools list to your provider's wire format here.
        # Return an LLMResponse with tool_calls populated if the model called a tool.
        return LLMResponse(
            content    = "response text",
            usage      = TokenUsage(input_tokens=10, output_tokens=20),
            model      = self.model,
            tool_calls = None,
            raw        = None,
        )

    def stream(self, messages: list[Message]) -> Iterator[str]:
        # stream() never receives tools= — LLM raises ValueError before reaching here.
        yield "response "
        yield "text"
        # Set _last_stream_usage after the stream ends so LLM.stream() can record it.
        self._last_stream_usage = TokenUsage(input_tokens=10, output_tokens=5)

    async def achat(
        self,
        messages:    list[Message],
        tools:       Optional[list[dict]] = None,
        tool_choice: Optional[str]        = None,
    ) -> LLMResponse:
        return LLMResponse(
            content    = "async response",
            usage      = TokenUsage(input_tokens=10, output_tokens=20),
            model      = self.model,
            tool_calls = None,
            raw        = None,
        )

    async def astream(self, messages: list[Message]) -> AsyncIterator[str]:
        yield "async "
        yield "response"
        self._last_stream_usage = TokenUsage(input_tokens=10, output_tokens=5)

    # ── optional: native JSON mode for chat_structured() ──────────────────

    def chat_structured(self, messages: list[Message]) -> Optional[LLMResponse]:
        """Use the provider's native JSON mode.

        Return an LLMResponse containing clean JSON, or return None to let
        LLM.chat_structured() fall back to the prompt-injection path.
        The schema instruction is already in messages by the time this is called.
        """
        # Example: add a provider-specific JSON parameter here
        return None   # remove this line and implement if your API supports native JSON

Register with the factory:

from pywrapai.providers import _PROVIDERS

_PROVIDERS["myprovider"] = MyProvider

llm = LLM(provider="myprovider")

---

Exception Handling

PyWrapAI raises typed exceptions so callers can handle specific failure modes:

PyWrapAIError                  ← base class for all library errors
├── ProviderError              ← base class for all provider API errors
│   ├── AuthError              ← 401/403 — invalid or missing API key
│   ├── RateLimitError         ← 429 — quota or rate limit exceeded
│   ├── InvalidRequestError    ← 400 — bad model name, invalid parameters
│   └── ServerError            ← 5xx — provider-side failure
└── ProviderTimeoutError       ← connection timeout or deadline exceeded

from pywrapai import (
    LLM,
    PyWrapAIError,
    AuthError,
    RateLimitError,
    InvalidRequestError,
    ServerError,
    ProviderTimeoutError,
)

llm = LLM(provider="openai")

try:
    response = llm.chat("Hello")
except AuthError:
    print("Check your API key")
except RateLimitError:
    print("Quota exceeded — retry later or switch provider")
except InvalidRequestError as e:
    print(f"Bad request: {e}")
except ServerError:
    print("Provider outage — try again shortly")
except ProviderTimeoutError:
    print("Request timed out")
except PyWrapAIError as e:
    print(f"Unexpected library error: {e}")

Retryable vs non-retryable:

• Retried automatically by max_retries: RateLimitError, ServerError, ProviderTimeoutError
• Stops retry loop immediately (goes straight to fallback): AuthError, InvalidRequestError

---

Embedder — Text Embedding (Phase 2)

Embedder lives in PyWrapAI (Library 1) because embedding is a provider API call, just like LLM.chat().

from pywrapai import Embedder

embedder = Embedder(
    provider     = "openai",                    # "openai", "gemini", or "ollama"
    model        = "text-embedding-3-small",    # optional — uses provider default
    api_key      = None,                        # reads from env var if not set
    track_tokens = True,                        # set False to skip TokenTracker
)

Provider defaults

Provider	Default model	Token counting
openai	text-embedding-3-small	From API response (exact)
gemini	models/text-embedding-004	4-chars-per-token heuristic (API does not return counts)
ollama	nomic-embed-text	4-chars-per-token heuristic

Anthropic is not supported for embeddings. Anthropic does not offer a public embedding API. Passing provider="anthropic" raises InvalidRequestError immediately with a message directing you to openai or ollama. If your entire stack is Anthropic, you will need a separate API key for embeddings — OpenAI text-embedding-3-small at $0.02/M tokens or local Ollama nomic-embed-text at $0.00 are both good choices.

Methods

# Embed a batch of texts (single API call per 100 texts)
vectors: list[list[float]] = embedder.embed(["Paris is nice", "London is rainy"])

# Embed a single text
vector: list[float] = embedder.embed_one("What is the capital of France?")

# Async variants
vectors = await embedder.aembed(["Paris is nice", "London is rainy"])
vector  = await embedder.aembed_one("What is the capital of France?")

embed() batches all texts into one API call (up to 100 per batch for OpenAI). embed_one() is embed([text])[0] — a convenience shortcut.

Token tracking

print(embedder.tokens.total_cost)   # float — USD cost of all embed() calls
print(embedder.tokens.calls)        # list[CallRecord] with kind="embedding"

Embedding CallRecord entries have kind="embedding" and output_tokens=0. They are stored in the Embedder’s own TokenTracker (accessible as embedder.tokens), not in LLM.tokens. To track both in one report, create a shared TokenTracker and pass it:

from pywrapai.tokens.tracker import TokenTracker

shared = TokenTracker()
llm      = LLM(provider="openai", _tracker=shared)
embedder = Embedder(provider="openai", _tracker=shared)

# Now shared.calls contains both "chat" and "embedding" records

Retry

Embedder inherits the same retry behaviour as LLM: pass max_retries=3, retry_delay=1.0 to enable exponential backoff on RateLimitError and ServerError.

---

What PyWrapAI Does NOT Do

These features deliberately do not exist in PyWrapAI. They live in the higher-level libraries:

Feature	Where it lives
Conversation history management	PyWrapAI-Graph
RAG pipeline / document retrieval / vector stores	PyWrapAI-Graph
Document chunking	PyWrapAI-Graph
Tool registration (@tool, ToolRegistry)	PyWrapAI-Graph
Tool execution / agent ReAct loop	PyWrapAI-Graph (Phase 3)
System prompt versioning (file-based)	PyWrapAI-Graph

Boundary between tool calling and tool execution:

PyWrapAI owns:

• The wire-level tool calling API — LLM.chat(tools=..., tool_choice=...) passes tools to the provider, parses the response, and returns response.tool_calls.
• The data types — ToolCall, ToolResult, Role.TOOL are part of the LLM response structure, the same way LLMResponse lives here.

PyWrapAI-Graph owns:

• Tool registration — @tool decorator, ToolRegistry, to_schema() to generate the neutral format.
• Tool execution — registry.execute(tool_call) runs the Python function and wraps the output in a ToolResult.
• The agent loop — the ReAct loop that drives chat(tools=...) → execute → send results back → repeat until plain text. This is Agent (Phase 3).