Explore traces

The Trace Explorer is where you search, filter, and inspect every trace flowing through your AI applications. It's available in Studio at Observability → Traces.

Before you can explore traces, your application needs to be instrumented. See Send traces for setup instructions.

What traces contain

What traces contain

Every trace is a tree of spans. Each span represents one operation in the execution chain.

FieldDescription
Trace IDUnique identifier for the full execution chain
SpansIndividual operations: LLM calls, tool calls, workflow steps
DurationTime taken for each span and the total trace
Token countsInput and output tokens per LLM call
StatusSuccess or error, with error messages when applicable
SourceWhich product generated the trace (SDK, Workflows, Vibe Work)
TagsCustom tags from SDK usage, plus automatic metadata such as model name

Click any trace in the list to open the span tree. Expand individual spans to inspect inputs, outputs, latency, and token usage at each step.

Search and filter

Search and filter

Explorer accepts a search expression language: write conditions on trace fields and combine them with AND, OR, and NOT.

Operators

Operators

CategoryOperatorsExample
Comparison=, !=, <, <=, >, >=duration_ns > 5000000000
Pattern matchingLIKE, ILIKE, NOT LIKEservice_name LIKE '%agent%'
SubstringCONTAINS, NOT CONTAINSservice_name CONTAINS 'sdk'
RegexREGEXP, NOT REGEXPservice_name REGEXP '^my-'
RangeBETWEEN ... AND ...duration_ns BETWEEN 1000000 AND 5000000000
ListIN (...), NOT IN (...)status_code IN ('Error', 'Ok')
Null checkEXISTS, NOT EXISTSerror_message EXISTS
BooleanAND, OR, NOT, ()status_code = 'Error' AND duration_ns > 1000000
Available fields

Available fields

FieldTypeDescription
service_namestringName of the service that produced the trace
status_codestringTrace status: Ok or Error
duration_nsnumberTotal trace duration in nanoseconds
error_messagestringError message if the trace failed
start_timetimestampWhen the trace started (ISO 8601)
models_usedarrayModels called within the trace
span_attributes['key']map valueAny span attribute, using bracket syntax
Span attribute queries

Span attribute queries

Use bracket syntax to filter on any OpenTelemetry span attribute:

span_attributes['gen_ai.request.model'] CONTAINS 'mistral-large'
span_attributes['gen_ai.operation.name'] = 'chat'
span_attributes['custom.tag'] EXISTS
Array fields

Array fields

For array fields like models_used:

FunctionExample
CONTAINSmodels_used CONTAINS 'mistral-medium-latest'
has(field, value)has(models_used, 'mistral-medium-latest')
hasany(field, [values])hasany(models_used, ['mistral-large-latest', 'mistral-small-latest'])
hasall(field, [values])hasall(models_used, ['mistral-large-latest', 'mistral-small-latest'])
Common examples

Common examples

# All failed traces from a specific service
service_name = 'my-agent' AND status_code = 'Error'

# Traces slower than 5 seconds
duration_ns > 5000000000

# Traces using a specific model
span_attributes['gen_ai.request.model'] CONTAINS 'mistral-large'
Add spans to a dataset

Add spans to a dataset

Turn production spans into evaluation records. Use the resulting Studio Dataset to drive an offline evaluation.

i
Information

Adding spans to a dataset is part of offline evaluations, available to Enterprise-tier organizations only.

Add a single span

Add a single span

  1. Open a trace and select a span in the span tree.
  2. Open the span actions menu (⋯) and click Add span to dataset.
  3. Select a dataset, or create one from the dataset picker.
  4. Review the payload in Preview and edit. By default, it contains the span's resource and span attributes as JSON.
  5. Optionally click Keep input, output only to reduce the payload to the span's input and output messages, or edit the JSON directly. Click Reset changes to restore the original payload.
  6. Click Add to dataset.

The payload must be a valid JSON object. After the record is created, click View record in the confirmation to open it in the dataset.

Add multiple spans

Add multiple spans

  1. In Explorer, switch the view from Traces to Spans, then filter to the spans you want.
  2. Select spans with the row checkboxes. You can select up to 500 spans at once.
  3. Click Add to dataset in the selection bar, then pick a dataset or create one.
  4. Map span fields to record fields. For each mapping, pick a Span field (a span or resource attribute) and name its Record destination, either an existing field of the dataset or a new one.
  5. Check the Record preview and the import summary, then click Continue and confirm the import.

Each selected span becomes one record that contains only the mapped fields. Spans with no value for any mapping are skipped. The import runs in the background. A notification tracks its progress and links to the dataset when it completes.

Query traces programmatically

Query traces programmatically

All trace data is accessible via the REST API. The endpoints are documented in the API reference:

  • Traces: search traces, get a single trace, list fields and field options.
  • Spans: list spans for a trace, query span evaluations.
Access

Access

Reading traces requires an Enterprise organization and one of the following roles:

RoleAccess
Org AdminAll traces across all workspaces
Workspace AdminAll traces in their workspace
Observability ViewerAll traces in their workspace (read-only)

A Workspace Admin or Org Admin can assign the Observability Viewer role to grant read-only access without admin privileges.

FAQ

FAQ