PyWrapAI

Constructor

llm = LLM(provider="anthropic", model="claude-haiku-4-5-20251001")

Provider defaults

Core methods

response = llm.chat("What is machine learning?")

response = llm.chat([
    Message(Role.USER,      "My name is Alice."),
    Message(Role.ASSISTANT, "Hello Alice!"),
    Message(Role.USER,      "What is my name?"),
])
print(response.content)  # → Your name is Alice.

for chunk in llm.stream("Write a poem about the sea."):
    print(chunk, end="", flush=True)

class ExtractedFact(BaseModel):
    subject: str
    predicate: str
    object: str

result = llm.chat_structured(
    "Marie Curie discovered polonium.",
    schema=ExtractedFact,
)
print(result.subject)  # → "Marie Curie"

async def main():
    response = await llm.achat("What is Python?")
    async for chunk in llm.astream("Tell me a story"):
        print(chunk, end="", flush=True)

asyncio.run(main())

Core data types

Token tracking & pricing

Every call is recorded in a TokenTracker, accessed via llm.tokens. Cost comes from a visible, overridable price table — never estimated.

llm.tokens.total_input     # int
llm.tokens.total_cost      # float — USD
llm.tokens.calls           # list[CallRecord]
llm.tokens.summary()       # formatted string report

Chat models — USD per 1M tokens

Embedding models — USD per 1M input tokens

Unknown models return $0.00 with a logging.warning() — never a hard error. Override anything with LLM(pricing="./my_pricing.json") or globally via load_user_pricing().

Response caching

llm = LLM(provider="openai", cache=True)          # default 1000 entries
llm = LLM(provider="openai", cache=ResponseCache(max_size=500))

llm.cache.size    # entries currently cached
llm.cache.clear() # wipe

Key: SHA-256 over all seven call parameters (model, messages, temperature, max_tokens, system_prompt, tools, tool_choice) — a tool-enabled call never collides with a plain one. Eviction: LRU via an OrderedDict. Thread safety: guarded by a threading.Lock.

Retry & fallback

backup = LLM(provider="anthropic")
llm = LLM(provider="openai", max_retries=3, retry_delay=1.0, fallback=backup)

Jitter: delay = base * (0.5 + random() * 0.5) — avoids thundering-herd retries. Retryable: RateLimitError, ServerError, ProviderTimeoutError. Non-retryable errors (AuthError, InvalidRequestError) skip straight to the fallback rather than wasting retry attempts.

Providers

All provider SDK imports are lazy — inside method bodies, not at module load. import pywrapai never fails just because a provider SDK isn't installed.

Exception hierarchy

AgentError and its subclasses are raised by Agent.run()/arun() in PyWrapAI-Graph, but defined here and re-exported — all three libraries share one exception hierarchy with no circular imports.

Embedder

embedder = Embedder(provider="openai")

vectors = embedder.embed(["Paris is nice", "London is rainy"])
vector  = embedder.embed_one("What is the capital of France?")

# async
vectors = await embedder.aembed([...])

Anthropic is not supported for embeddings — it offers no public embedding API. Passing provider="anthropic" raises InvalidRequestError immediately. embedder.tokens is a separate TokenTracker from llm.tokens, with kind="embedding" records.

Structured logging

Every call emits through the standard logging module — the library never calls basicConfig() or attaches its own handler. Attaching a handler and choosing a level is entirely up to your application.

What PyWrapAI does not do