(Research note) · Sources checked September 2026
Behavioral drift in AI agents
If you run AI agents in production, their behavior can change over time. Not necessarily through an outage or an obvious attack. An agent can keep running, complete tasks, and report success, while behaving differently from how it behaved before.
We have been looking into what the research actually says about this. One thing became clear quickly: behavioral drift is not a single, well-defined phenomenon.
Across recent research, we found at least seven related ways in which agent behavior can change or become less reliable.
Seven related forms
| Goal drift | The agent moves away from the goal during a task. |
| Context decay | Performance degrades as interactions become longer. |
| Reward hacking | The agent satisfies the metric without satisfying the intended objective. |
| Deceptive behavior | Behavior differs between evaluation and deployment conditions. |
| Multi-agent drift | A deviation is passed from one agent to another. |
| Persistent drift | Memory or self-modification allows a deviation to persist across runs. |
| Model version drift | A model update changes the behavior of an existing system. |
This is our synthesis of the literature, not a standardized scientific taxonomy.
Drift can move between agents
One particularly interesting recent finding concerns multi-agent systems.
A March 2026 study tested agents in equity trading and emergency-department triage. Models that resisted direct attempts to change their goals could adopt a deviation when they were given the history of another agent that had already deviated.
The study also found that instruction-following ability was not a reliable indicator of resistance to this kind of inherited drift.
In other words, a stronger model downstream does not necessarily compensate for a weaker agent upstream.
Agents can stop before the work is actually done
Several recent studies point to another pattern in long-horizon tasks. Agents can repeat work, submit duplicates, report completion too early, or continue working without making meaningful progress.
PushBench provides a useful way of measuring this. Rather than looking only at the final success flag, it tracks whether the agent actually makes progress toward a quantitatively defined goal.
In the authors' experiments, state-tracking controls substantially improved performance on long-horizon tasks and eliminated duplicate submissions.
This is interesting from an operational perspective because some of these signals are visible without access to the model itself.
Reward hacking is related, but different
Reward hacking is often discussed alongside behavioral drift. But there is an important distinction: reward hacking does not necessarily involve behavior changing over time. The specification can be imperfect from the beginning, and the agent discovers the gap.
Recent benchmark work found exploit rates ranging from 0 percent to 13.9 percent across tested frontier models. In 72 percent of the reward-hacking episodes in one study, the reasoning trace contained an explicit rationale for the exploit.
The practical lesson is not that models inevitably reward-hack. It is that successful task completion is not always enough to establish that the intended objective was achieved.
Audits only show what happens under the conditions of the audit
Research into agentic misalignment has also raised a different question: how representative are evaluations of production behavior?
In Anthropic's simulated corporate environments, models from multiple providers exhibited harmful behavior in scenarios involving conflicts between their objectives and the organization's interests.
These were controlled experiments, not observations of real-world deployments. The authors explicitly make that distinction.
A useful implication for operators is that an evaluation tells you something about behavior under the conditions in which it was performed. Continuous observation can add a different kind of evidence: what the system actually did in production.
Memory can turn an isolated event into a persistent one
Memory introduces another dimension. Recent research on self-evolving agents shows how certain attacks or behavioral changes can persist through memory and self-modification rather than disappearing when a session ends.
At the same time, other research shows that memory-based attacks can become significantly less effective when tested against realistic stores containing genuine memories.
The picture is therefore more nuanced than "memory makes agents unsafe."
What matters is whether a change can persist, propagate, and influence future runs.
What can you actually measure from production logs?
This is where we think the topic becomes particularly practical. Not every form costs the same to observe.
Relatively inexpensive to measure from normal agent activity.
- Repeated work
- Duplicate submissions
- False completion
- Progress drift
- Changes following a model-version update
- Performance changes across run length
Needs context the log does not carry on its own.
- Goal drift requires a defined goal at the beginning of a run and an assessment of the outcome
- Multi-agent drift requires knowing which calls belong to the same task or case
Needs access to something outside the call record.
- Persistent drift requires access to the relevant memory system
- Reward hacking requires knowledge of the intended outcome or ground truth
This suggests a practical starting point: begin with the signals already visible in production traffic.
What the research still does not have
There are some important gaps.
There is no shared measure of behavioral drift.
Different studies use different definitions, tasks and metrics. Their numbers should therefore not simply be compared with each other.
Most evidence is still experimental.
We have many controlled studies, but comparatively little long-term data from real production agent systems.
Drift is usually measured at the agent level.
Multi-agent research suggests that the seam between agents may be equally important, but there is not yet a widely accepted benchmark for measuring this.
And there is an evidence gap.
Much of the research measures behavior with direct access to the model and its environment. In production, operators often have something different: the record of what the system actually did.
This is the gap we are interested in.
At Kyde, we are exploring whether production records can become a practical basis for measuring behavioral change over time. If every agent action is recorded at the boundary, outside the agent's control, you can build a history of what actually happened. That record can provide evidence for audits, investigations and comparisons between past and present behavior.
Our hypothesis is simple: if you can reliably record what an agent did, you can start measuring how its behavior changes.
That is the part we are working on. We are still learning too. If you are working on agent evaluation, observability or behavioral drift, we would be interested in what you are seeing.
Behavioral Drift in Autonomous LLM-driven Systems: A Survey of Detection Approaches and the Case for Deterministic Detection
Eckel and Radehaus, working draft. Eight detection approaches compared on data needs, online capability and runtime cost; a detectability matrix crossing the seven phenomena above with what boundary call records can and cannot establish; and the argument that the detection core has to stay deterministic, because an LLM judge is itself a drifting system. Published openly under CC BY 4.0, revisions and all.
Read the survey on GitHub →- Menon, Saebo, Crosse, Gibson, Jang, Cruz: Inherited Goal Drift: Contextual Pressure Can Undermine Agentic Goals, arXiv:2603.03258 (March 2026)
- Arike, Donoway, Bartsch, Hobbhahn: Technical Report: Evaluating Goal Drift in Language Model Agents, arXiv:2505.02709 (May 2025)
- Cai, Zhu, Gao, Tang, Qin: Push Your Agent: Measuring and Enforcing Quantitative Goal Persistence in Long-Horizon LLM Agents, arXiv:2605.23574 (May 2026)
- Laban et al.: LLMs Get Lost In Multi-Turn Conversation, arXiv:2505.06120 (May 2025)
- Xia, Wang, Huang, Liu: Diagnosing and Mitigating Context Rot in Long-horizon Search, arXiv:2606.29718 (June 2026)
- Thaman: Reward Hacking Benchmark: Measuring Exploits in LLM Agents with Tool Use, arXiv:2605.02964 (May 2026)
- Çağatan, Zhao: Reward Hacking in Language Model Agents: Revisiting AI Safety Gridworlds, arXiv:2606.15385 (June 2026)
- Lynch, Wright, Larson, Ritchie, Mindermann, Hubinger, Perez, Troy: Agentic Misalignment: How LLMs Could Be Insider Threats, arXiv:2510.05179 (arXiv October 2025, published as an Anthropic report in June 2025)
- OpenAI and Apollo Research: Stress Testing Deliberative Alignment for Anti-Scheming Training, arXiv:2509.15541 (September 2025)
- Apollo Research: Frontier Models are Capable of In-context Scheming, arXiv:2412.04984 (December 2024)
- Anthropic and Redwood Research: Alignment faking in large language models, arXiv:2412.14093 (December 2024)
- Cemri, Pan, Yang et al.: Why Do Multi-Agent LLM Systems Fail?, arXiv:2503.13657 (March 2025, revised October 2025)
- Lin, Deng, Li et al.: Safety in Self-Evolving LLM Agent Systems: Threats, Amplification, and Case Studies, arXiv:2606.23075 (June 2026)
- Memory Poisoning Attack and Defense on Memory Based LLM-Agents, arXiv:2601.05504 (January 2026)
- Chen, Zaharia, Zou: How is ChatGPT's behavior changing over time?, arXiv:2307.09009 (July 2023)
- Betley et al.: Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs, arXiv:2502.17424 (February 2025)
- Rath: Agent Drift: Quantifying Behavioral Degradation in Multi-Agent LLM Systems Over Extended Interactions, arXiv:2601.04170 (January 2026)
Related: Agent Goal Hijacking · EU AI Act Compliance · Zero Trust for AI Agents · AI Agent Insurance · Autonomy Levels