Tracer

Observability OpenAI SDK-inspired

Full observability system that captures execution traces with spans for LLM calls, tool invocations, handoffs, and guardrail checks for debugging and analysis.

Overview

The Tracer Agent implements comprehensive observability inspired by OpenAI's Agents SDK tracing. It captures structured trace data throughout agent execution:

Hierarchical Spans: Nested spans show parent-child relationships
Event Logging: Discrete events within spans
Performance Metrics: Duration, tokens, latency for each operation
Error Tracking: Failures captured with full context
Export Formats: JSON, OpenTelemetry, custom backends

Span Types

Span Type	Captures	Key Attributes
agent	Agent lifecycle	agent_name, task, duration
llm	LLM API calls	model, tokens_in, tokens_out, latency
tool	Tool invocations	tool_name, arguments, result
handoff	Agent transitions	source, target, context_size
guardrail	Validation checks	guardrail_name, result, violations

Trace Visualization

Trace: tr_abc123 | Duration: 45.2s | Status: Success

├─ [agent] conductor (45.2s)
│  ├─ [llm] claude-sonnet (2.1s) tokens: 1,200/800
│  ├─ [handoff] → architect (0.1s)
│  │
│  ├─ [agent] architect (12.4s)
│  │  ├─ [llm] claude-sonnet (3.2s) tokens: 2,100/1,500
│  │  ├─ [tool] Read (/src/auth.ts) (0.05s)
│  │  ├─ [tool] Write (/TODO/spec.md) (0.08s)
│  │  └─ [guardrail] output_validation ✓
│  │
│  ├─ [handoff] architect → auto-code (0.1s)
│  │
│  └─ [agent] auto-code (30.5s)
│     ├─ [llm] claude-sonnet (8.5s) tokens: 4,200/3,800
│     ├─ [tool] Write (/src/auth/jwt.ts) (0.12s)
│     ├─ [tool] Bash (npm test) (5.2s)
│     ├─ [guardrail] code_safety ✓
│     └─ [guardrail] pii_detection ✓

Commands

/trace list

/trace list

Recent Traces:
ID          | Task                    | Duration | Status  | Spans
------------|-------------------------|----------|---------|------
tr_abc123   | Add authentication      | 45.2s    | Success | 24
tr_def456   | Fix login bug          | 12.8s    | Success | 11
tr_ghi789   | Refactor database      | 128.4s   | Failed  | 47

/trace show

/trace show tr_abc123

[Full trace visualization as shown above]

Summary:
- Total LLM calls: 3
- Total tokens: 7,500 in / 6,100 out
- Tools invoked: 4
- Guardrails passed: 3/3
- Handoffs: 2

/trace export

/trace export tr_abc123 --format opentelemetry

Exported to: ./traces/tr_abc123.otlp.json
Compatible with: Jaeger, Zipkin, Honeycomb

Integration Points

System	Integration
All Agents	Automatic span creation for agent lifecycle
Guardrails	Guardrail check spans with results
Handoff	Handoff spans linking agent transitions
Time-Travel	Traces provide replay source data
Memory	Stores trace summaries for analysis

Use Cases

Performance Analysis: Identify slow operations and bottlenecks
Debugging Failures: Trace back through execution to find root cause
Cost Tracking: Monitor token usage per task and agent
Compliance: Audit trail of all LLM interactions and decisions

← All Agents Confidence Agent →