Agent Observability

A trace is the observability unit of an entire agent run — the full story of what happened from the first input to the final output.

What a trace contains:

① Run metadata — session ID, start time, duration, total cost, model used
② Input — the user's initial request
③ All spans — every step taken during the run (LLM calls, tool calls, sub-agent calls)
④ Intermediate outputs — Thoughts, tool results, partial answers at each step
⑤ Final output — the agent's response to the user
⑥ Feedback (if collected) — human ratings, automated eval scores

Why traces are the fundamental observability unit:
• A single user interaction might involve 15+ LLM calls and 30+ tool calls — you need the trace to understand the full picture
• Unlike simple logs (text), traces are structured and queryable — you can filter by model, cost, step count, outcome
• Traces can be replayed, compared, and used as training examples

Key vocabulary:
• Trace ID — unique identifier for one complete agent run
• Run — synonym for trace in some frameworks (LangChain uses "run")
• Trace viewer — the UI that renders a tree of all steps in a trace
• Trace dataset — a collection of traces used for offline evaluation

A span is the individual building block that makes up a trace — one discrete unit of work.

Types of spans in an agent trace:

Span type	Contains
LLM span	Prompt, completion, input tokens, output tokens, latency, model
Tool span	Tool name, parameters, result, latency
Retrieval span	Query, retrieved documents, similarity scores
Agent span	Wraps a complete sub-agent invocation

Trace hierarchy:
A trace is a tree of spans:
Trace (root span)
├── LLM span: "What tool should I use?" (step 1)
│ └── Tool span: get_customer(id="C-8847") (step 2)
├── LLM span: "How do I format the response?" (step 3)
└── LLM span: Final answer generation (step 4)

Key vocabulary:
• Parent span — the span that contains/triggered a child span
• Span duration — how long the step took (latency)
• Token count per span — how many tokens that specific LLM call consumed
• Span metadata — tags and attributes attached for filtering

A token budget is a cost and safety control for preventing unbounded agent consumption.

Why agent token budgets are critical:
• Unlike single LLM calls, multi-step agents can run for many iterations
• Each iteration: system prompt + conversation history + tool schemas + tool results → grows the context window
• A 50-step agent run with GPT-4o can easily cost $2–5 USD or more
• Without a budget, a looping agent will exhaust your API quota

Token budget implementation types:

Type	Behaviour when exceeded
Hard limit	Stop immediately; return budget-exceeded status
Soft limit	Warn the agent: "You have X tokens remaining — wrap up"
Warning threshold	Trigger a notification at 80% usage; halt at 100%

Soft limit prompt injection pattern:
"SYSTEM: You have used 45,000 / 50,000 tokens. Prioritise completing the task in the next 1–2 steps."

Key vocabulary:
• Token budget — max tokens allowed for one run (hard or soft)
• Budget-exceeded status — the outcome code when a run hits the limit before completion
• Context window pressure — the growth of accumulated context across agent steps

LangSmith and LangFuse are purpose-built LLM observability platforms — not general logging tools.

Why traditional logging falls short for agents:
• Text logs show "agent called tool X" — but not why, what the full prompt was, or how many tokens it cost
• You can't easily compare two versions of a prompt from logs
• You can't search logs by "show me all runs where the agent took more than 15 steps"

What LangSmith / LangFuse add:

Capability	Use case
Trace viewer	See the full step tree, click any span to inspect input/output
Dataset + eval	Run saved traces through eval functions to measure regressions
Prompt playground	Edit prompts and compare outputs side-by-side
Cost analytics	Total spend per project, model, and individual trace
Feedback collection	Attach human ratings or automated eval scores to traces

Key vocabulary:
• LLM observability — the practice of making LLM and agent behaviour visible and measurable
• Prompt versioning — tracking and comparing iterations of a prompt over time
• Offline evaluation — running evaluations against a saved dataset (not real-time user traffic)

Option B demonstrates expert-level agent observability thinking — using traces and spans, not raw logs.

The debugging workflow for token optimisation:

① Open the trace in LangSmith (or LangFuse) by trace ID
② Navigate to the spans view — shows all 47 steps as individual spans in a tree
③ Sort spans by token count — instantly surfaces the most expensive LLM calls
④ Open the expensive span → inspect the prompt that generated it (what's making it large?)

Common causes of high-token spans:
• Tool results being passed in full (e.g. entire API responses) instead of summarised
• Conversation history growing without truncation
• System prompts with excessive instructions
• Tool schemas being re-sent in every step instead of being cached

Optimisation techniques triggered by this analysis:
• Tool result summarisation — pre-process large API responses before passing to the LLM
• Sliding window history — only keep the last N steps in context
• Schema caching — send tool schemas once; reference them by name in subsequent steps

Key vocabulary:
• Token optimisation — reducing token consumption without losing task capability
• Span-level token analysis — examining token counts for individual agent steps
• Context management — strategies for keeping agent context window usage under control