If you run AI agents in production, the OpenTelemetry GenAI semantic conventions deserve your attention. They define a shared vocabulary, under the gen_ai.* namespace, for describing model calls, tool calls, retrieval, memory and complete agent runs. A run becomes a trace, each model call and tool call becomes a span, and token usage becomes a metric.
Adoption is uneven, but it is moving in one direction. VS Code Copilot already emits invoke_agent, chat and execute_tool spans that follow the conventions. Claude Code adds a handful of gen_ai.* attributes to its tracing, which is still in beta. Several older instrumentation libraries predate the conventions and are moving toward them gradually.
That is useful progress. But a shared way to describe what an agent did does not tell you whether it should have done it. The second question is the one most teams need answered, and the conventions leave it open by design.
What the conventions standardize
The central attribute is gen_ai.operation.name, which says what kind of work a span represents: chat, embeddings, invoke_agent, execute_tool, retrieval, or one of the other values the spec defines. A few more attributes do most of the remaining work:
gen_ai.provider.nameandgen_ai.request.modelrecord which provider and model answered.gen_ai.tool.nameandgen_ai.tool.call.idrecord which tool the agent called.gen_ai.conversation.idties the spans of one session together.
MCP fits the same model. The spec treats an MCP tool call as an execute_tool span, so a tool behind a protocol and a tool in your own process look the same in a trace.
The conventions also make a deliberate choice about content. Instrumentations should not capture prompts and responses by default, and should offer a way to opt in when you need them. As a result, the most useful operational signal, which tools an agent called, in what order, how long each took and whether it failed, is available without anyone storing what was said.
What a single span can't show
Each span describes one step on its own. Take the trace most guides use as an example: an invoke_agent run that calls the model, runs web_search, calls the model again, runs summarize_doc and calls the model one last time. Every span is well formed, and every attribute matches the spec.
Now suppose that on a later run, the same agent calls send_email at step four. Or it calls web_search eleven times instead of once. Or it pays a merchant it has never paid before. Those spans are just as well formed as the earlier ones. Nothing in them says the run was unusual, because unusual only means something relative to how this agent has behaved before. It is a property of the whole sequence, compared against a history, and a single span carries neither.
Why counting calls isn't enough
The spec's metrics come closest. Two histograms, gen_ai.invoke_agent.tool_calls and gen_ai.invoke_agent.inference_calls, record how many tool and model calls an agent makes per invocation. A rising count is a reasonable early sign that an agent is stuck in a loop.
Many meaningful changes don't move the count, though. An agent that used to call search, read and summarize, and now calls search, read and export, makes the same number of calls. Its latency and token spend may not change either. A dashboard built only on counts, latency and tokens would show nothing wrong.
This matters for security as much as for reliability. A successful prompt injection doesn't have to make an agent slower or more expensive. It only has to make the agent do something different, and that shows up in which tools the agent calls, not in how many.
What detecting drift requires
To catch these changes, you need three things the conventions don't try to provide.
A baseline for each agent. What is normal for a research agent is not normal for a billing agent. The baseline has to be learned from each agent's own runs, keyed on an identity the telemetry already carries.
Detectors that give the same answer every time. Sudden shifts and gradual drift are different statistical problems. CUSUM is designed for the first. Windowed Jensen-Shannon divergence over the distribution of tool calls suits the second. Neither needs a language model to guess what the agent intended, so the same input always produces the same score, and anyone can rerun the calculation to check it.
A contract for each task. Some changes shouldn't wait for statistics. If an agent calls a tool it has never used for this kind of task, that is worth flagging the first time it happens. A new payee is a good example: by the time a distribution has shifted, the money has moved.
None of this needs new instrumentation. It needs something that reads the spans agents already emit, in order, and compares them with what came before.
Building on a spec that's still changing
Every gen_ai.* attribute, span and metric is still at Development stability, the earliest of OpenTelemetry's three levels. In June 2026, with release v1.42.0, the conventions moved out of the core semantic conventions repository into a dedicated one, semantic-conventions-genai. That repository hasn't published a tagged release yet, and its schema URL is still listed as TODO.
That is no reason to wait, but it does suggest two habits. First, keep the attribute names you depend on in one place in your code. If gen_ai.tool.name is renamed, you want to change one line, not search through every query and dashboard.
Second, read only what you need. A behavioral score needs to know which tool was called, when, in which session and whether it succeeded. It doesn't need the prompt. If you extract those few fields and ignore everything else, you keep the spec's no-content default by construction, instead of relying on every downstream system to respect it.
Where this leaves you
The GenAI conventions are well on their way to solving the naming problem. Agents built on different frameworks can describe their work in one language, to any backend. That is a solid foundation.
The next layer is judgment. We think it should be deterministic: each run measured against that agent's own history, with a number as the result, rather than a second model's opinion of the first. A trace records what happened. A score tells you whether that was normal for this agent.
KYDE Score
Kyde Score works this way. It reads gen_ai.* tool spans and Claude Code tool events over OTLP, learns a baseline from each agent's first 30 runs over at least 3 days, and scores every later run from 0 to 100 using CUSUM, Jensen-Shannon divergence and a per-task contract. When an agent's score moves into a different band, it sends an HMAC-signed webhook. It never reads prompt content.