A user complains that the assistant took 40 seconds and then gave a wrong answer. Your logs show one HTTP request that took 40 seconds. Inside it, the agent made four model calls, ran three tools, retried one of them, and retrieved eight documents. Without traces, finding which step was slow or which step introduced the wrong fact is guesswork.

OpenTelemetry has semantic conventions for generative AI that standardise span names and attributes for model calls, tool calls and agent runs. Using them means your traces work with any OpenTelemetry backend and with instrumentation libraries that already follow the conventions. This guide shows how we instrument an agent service, what we record, and what we deliberately do not.

The shape of an agent trace

A useful trace mirrors the agent's work:

POST /chat                                   41.2 s
└─ invoke_agent support-assistant            41.0 s
   ├─ chat large-2026-01                      3.8 s   in=2,140 out=96   finish=tool_calls
   ├─ execute_tool search_docs                0.4 s
   ├─ chat large-2026-01                      4.1 s   in=6,880 out=120  finish=tool_calls
   ├─ execute_tool get_order_status          28.9 s   error=timeout (retry)
   ├─ execute_tool get_order_status           1.1 s
   └─ chat large-2026-01                      2.6 s   in=7,410 out=310  finish=stop

The slow step is obvious: an order service timeout. Without the tool spans, it would look like a slow model.

Span names and core attributes

The conventions define operation names such as chat, embeddings, execute_tool and invoke_agent. A model call span is named {operation} {model}, for example chat large-2026-01, with span kind CLIENT.

The attributes we always set on model call spans:

Attribute Example Why
gen_ai.operation.name chat Groups calls by type
gen_ai.provider.name openai Provider; older convention versions used gen_ai.system
gen_ai.request.model large-2026-01 What you asked for
gen_ai.response.model large-2026-01-0115 What actually answered
gen_ai.request.max_tokens 800 Explains truncation
gen_ai.request.temperature 0.2 Explains variability
gen_ai.usage.input_tokens 6880 Cost and context growth
gen_ai.usage.output_tokens 120 Cost and latency
gen_ai.response.finish_reasons ["tool_calls"] Stop, length, tool calls
error.type timeout Error category on failure

Tool spans use execute_tool with gen_ai.tool.name and the tool call id. Agent spans use invoke_agent with the agent name.

The conventions are still marked as development status, and names have changed between versions. Pin the convention version your instrumentation follows, and check the changelog when upgrading. Some libraries let you opt into the latest experimental names with the OTEL_SEMCONV_STABILITY_OPT_IN environment variable.

Instrumenting a call by hand

Instrumentation libraries exist for the major provider SDKs and frameworks, and they are the easiest start. When you call an API through your own client, wrapping it is short:

from opentelemetry import trace
from opentelemetry.trace import SpanKind, Status, StatusCode

tracer = trace.get_tracer("assistant")

def chat(model: str, messages: list[dict], **params):
    with tracer.start_as_current_span(f"chat {model}", kind=SpanKind.CLIENT) as span:
        span.set_attribute("gen_ai.operation.name", "chat")
        span.set_attribute("gen_ai.provider.name", "openai")
        span.set_attribute("gen_ai.request.model", model)
        if "max_tokens" in params:
            span.set_attribute("gen_ai.request.max_tokens", params["max_tokens"])
        try:
            resp = client.chat.completions.create(model=model, messages=messages, **params)
        except Exception as exc:
            span.set_attribute("error.type", type(exc).__name__)
            span.set_status(Status(StatusCode.ERROR))
            raise
        span.set_attribute("gen_ai.response.model", resp.model)
        span.set_attribute("gen_ai.usage.input_tokens", resp.usage.prompt_tokens)
        span.set_attribute("gen_ai.usage.output_tokens", resp.usage.completion_tokens)
        span.set_attribute("gen_ai.response.finish_reasons",
                           [c.finish_reason for c in resp.choices])
        return resp

Tool execution gets the same treatment with an execute_tool {name} span. Because spans nest through context, tool spans appear under the agent span automatically.

For streaming, start the span before the request, record time to first token as an event or attribute, and end the span when the stream finishes or fails, not when the first chunk arrives.

Prompts and completions: opt-in, redacted, sampled

Prompts and responses are the most useful debugging data and the most sensitive. The conventions treat message content as opt-in, recorded as span events or log records rather than attributes, and instrumentation libraries usually have a flag to turn content capture on. Our policy:

  • Content capture is off by default in production.
  • When on, content goes through the same PII redaction as our logs before export.
  • It is enabled per feature for a limited time when debugging, or sampled at a low rate.
  • The tracing backend's retention for content is shorter than for metadata, and access is restricted.

Metadata alone answers most operational questions: which step was slow, how big the context got, how often the model hit the length limit.

Metrics from traces

The conventions also define metrics, such as token usage and operation duration histograms. If your backend can derive metrics from spans, you get them for free. If not, record gen_ai.client.token.usage and gen_ai.client.operation.duration with the model and operation as attributes. Dashboards we look at weekly:

  • Input tokens per request per feature, to spot context growth.
  • Share of finish_reason = length, a sign of truncated answers.
  • p95 duration per model and per tool.
  • Tool error rate per tool.

Sampling

Agent traces are large. Head sampling at a fixed rate drops the rare, interesting cases. We use tail sampling in the OpenTelemetry Collector: keep all traces with errors, all traces slower than 20 seconds, all traces with a fallback, and 5 percent of the rest.

Common mistakes

Mistake Why it hurts Do this instead
One span for the whole request Cannot see which step was slow Span per model call and per tool
Prompts as span attributes by default PII in your tracing backend Opt-in content events with redaction
Custom attribute names per team Dashboards do not compose Follow GenAI semantic conventions
Ending stream spans at first chunk Durations look tiny End at stream completion
Head sampling only Errors and slow runs lost Tail sampling on errors and latency

Key takeaways

  • Model calls, tool calls and agent runs each get their own span.
  • Use the GenAI semantic convention names and pin the version you follow.
  • Record usage, finish reasons and response model on every call.
  • Capture content only when needed, redacted, with short retention.
  • Tail-sample to keep the traces you will actually look at.

References