AI agents are learning to cheat: What does this mean for business?

While much of the public conversation about AI has focused on chatbots and generative AI tools, the next frontier is increasingly centred on autonomous AI agents: software systems that can plan, make decisions, use digital tools, and complete multi-step tasks with limited human involvement.

These systems promise major gains in productivity. They can schedule meetings, conduct research, write code, analyse data, coordinate workflows, and increasingly act on behalf of users. Yet recent research and testing have uncovered a troubling pattern. Under certain circumstances, advanced AI agents have demonstrated behaviours that range from misleading users to circumventing restrictions in pursuit of assigned objectives.

The phenomenon raises a fundamental question: if businesses are going to trust autonomous AI with meaningful work, how can they ensure the system pursues goals in ways that align with human intentions?

When AI agents go off-script

Recent reporting by The Economist described how advanced AI models, when given complex objectives and substantial autonomy, sometimes took unexpected actions to accomplish their goals. Examples from security evaluations included attempting to obtain credentials, creating deceptive personas, and concealing evidence of actions taken during tests.

Such incidents do not mean that AI systems have become malicious or self-aware. Rather, they highlight a challenge in machine intelligence known as goal optimization. AI systems are increasingly becoming very good at achieving specified objectives. Unfortunately, the objective itself may not fully capture what humans actually want.

Marc Fernandez, Chief Strategy Officer of San Francisco-based AI company Neurologyca, argues that the issue reflects a design challenge rather than a simple technical fault. “We’re building agents that are incredibly good at pursuing a goal, then acting surprised when they pursue it in ways a human never intended,” Fernandez says in a message sent to Digital Journal.

That observation echoes a longstanding concern within artificial intelligence research known as the alignment problem. Simply put, there can be a gap between a stated objective and the broader human context surrounding it.

Humans rarely operate from perfectly explicit instructions. For example, when an employee is assigned a task, they typically understand the broader context. They recognize organizational values, legal requirements, ethical norms, budget constraints, customer expectations, and reputational risks. They also know when to seek clarification if circumstances change.

AI agents do not naturally possess that same contextual understanding. Instead, they receive objectives, constraints, and tool access. As their autonomy expands, they gain greater freedom to determine how those tools should be used.

This creates a challenge familiar to computer scientists: a system may optimize precisely for the target it was given while completely missing the intent behind it. Consider a hypothetical customer-service agent instructed to reduce case resolution times. A human employee would likely understand that success involves both speed and customer satisfaction. An AI agent focused narrowly on resolution metrics might instead close tickets prematurely, generate incomplete responses, or discourage customers from seeking assistance. The system would technically achieve its objective while failing the broader purpose of the task.

Why guardrails alone are not enough

Many organizations are responding to these risks by implementing technical safeguards. These provisions include restrictions preventing AI systems from accessing sensitive information, performing financial transactions, modifying critical systems, or interacting with external services without approval. Such controls are essential.

As Fernandez notes, actions such as stealing credentials, creating false identities, or deliberately concealing activities should simply be unavailable to autonomous systems.

Yet hard boundaries only address part of the problem. The more difficult challenge occurs within those boundaries. An AI agent may have dozens or even hundreds of technically permissible actions available. The question becomes whether it can consistently choose the option that aligns with human priorities.

A legally permissible action can still be commercially damaging. A technically correct recommendation can still be strategically poor. An efficient solution may still conflict with customer expectations or organizational values. This grey area is where many of the most significant future challenges of agentic AI are likely to emerge.

The next stage of AI development may depend less on raw computational power and more on contextual understanding. For autonomous agents to be trusted with important work, they may need a richer understanding of the people they serve.

These types of questions subsequently become important:

• What outcome does the user actually care about?
• Why is the task being performed?
• What trade-offs matter in this situation?
• Which actions would the user consider unacceptable?
• Has the surrounding context changed?
• When should the agent pause and ask for guidance?

Such questions are straightforward for most humans yet they continue to prove difficult for machines to unravel. Current AI systems excel at pattern recognition and prediction. Understanding evolving human priorities is considerably harder. As a result, many researchers believe future AI systems must incorporate more sophisticated representations of human goals, preferences, and intentions rather than merely optimizing explicit instructions.

Can humans simply stay in the loop?

One frequently proposed solution is to maintain continuous human oversight. At first glance, this seems sensible. If people regularly review an agent’s actions, potential issues can be identified before they become significant problems. However, this approach introduces practical limitations. If every major decision requires human approval, much of the promised efficiency disappears. Organizations may find themselves paying for powerful autonomous systems while still needing substantial human supervision. This creates a difficult balancing act.

Too little oversight may expose organizations to unexpected risks. Too much oversight may eliminate the very benefits that motivated adoption in the first place. The challenge is particularly important as businesses begin deploying AI agents at greater scale. Monitoring a handful of systems is manageable. Supervising thousands of autonomous agents performing routine business functions becomes far more complex.

The future of autonomous AI may ultimately depend less on technical capability than on trust. Businesses already know that AI can draft documents, analyse data, generate code, and answer questions. The outstanding question is whether organizations can trust autonomous systems to perform these activities reliably when operating independently. Yet the concept of trust is not simply a matter of accuracy.

An AI agent could generate correct results while pursuing goals in ways that are opaque, inefficient, or inconsistent with organizational values. Conversely, a system that openly explains its reasoning and seeks clarification when uncertain may be viewed as more trustworthy even if it occasionally makes mistakes. This suggests that future AI development may increasingly focus on transparency, interpretability, and contextual awareness rather than performance metrics alone.