AI agents are getting smarter, but autonomy still has a trust problem

Artificial intelligence is rapidly shifting from technology that generates information towards systems designed to take action. Yet, as businesses consider allowing AI agents to make purchases, submit information and execute multi-stage workflows, new research points to a continuing gap between documented capabilities and dependable real-world performance.

The Agentic Capability Report from web data infrastructure provider Decodo evaluated 45 AI browsing tools against ten capabilities. These included form filling, transactions, multi-step task completion, awareness across browser tabs, memory, cross-site execution, confirmation before irreversible actions, background tasks and third-party integrations. According to Decodo’s scoring, Claude for Chrome and Amazon Nova Act jointly achieved the highest overall result, at 18 points out of a possible 20. Fellou and ChatGPT Desktop followed on 17, although Fellou’s score includes a two-point deduction following hands-on testing. Skyvern scored 16 and Vercel Agent Browser 15. No product achieved the maximum 20 points. For businesses, however, the most informative finding may not be which product finished first. More significant is what happens when an AI system moves from gathering information to performing actions that have real consequences.

The Decodo research makes an important distinction between documented capability and practical performance. The researchers first examined how clearly suppliers described each capability across public documentation, pricing pages and changelogs. Each capability received between zero and two points, producing a maximum documented score of 20. Decodo then subjected higher-stakes claims to hands-on testing, recording problems including crashes, freezes, timeouts, session drops and unsuccessful task completion.

This means the results should not be interpreted simply as a laboratory comparison in which every function of all 45 products underwent identical testing. The headline scores primarily reflect documented functionality, with adjustments made where Decodo’s practical testing identified contradictions, credibility problems or reliability failures. For example, Fellou initially received 19 points on the documentation assessment. Decodo reported that its practical testing encountered difficulties with complex layouts, loops and CAPTCHAs, leading the company to reduce the product’s multi-step capability score and its overall result to 17. However, something appearing on a product’s feature list does not, in itself, establish how reliably that capability will perform within an organisation’s particular environment.

AI can browse, but can it safely buy?

Transactions produced the lowest average capability score in Decodo’s research, at just 0.43 out of two. The category refers to completing a purchase or checkout, rather than merely adding something to a basket. Only six products received full marks for transactional actions: Amazon Nova Act, BrowserOS, ChatGPT Desktop, Minded, Sigma Browser and Skyvern.

The result highlights an important distinction between an AI assistant and an autonomous AI agent. Finding products and presenting options leaves the final decision with a person. Completing a purchase turns the AI-generated decision into an external action involving money. Safeguards consequently become particularly important. Decodo reports that ten tools document a pause before an irreversible action. However, there is an anomaly in the published report: the accompanying passage names only nine products. This discrepancy means the claimed total should be treated cautiously unless clarified by the researchers.

More significantly, Decodo found that just two products, Amazon Nova Act and ChatGPT Desktop, document both the ability to complete a purchase and a mechanism for stopping to obtain confirmation before carrying out an irreversible action. The distinction has implications for organisations considering agentic AI. Capability and appropriate autonomy are not necessarily the same thing. The amount of independence granted to an agent needs to reflect the potential consequences if it makes the wrong decision.

The tension between autonomy and oversight becomes more apparent with multi-stage workflows. Of the 35 tools Decodo says fully document multi-step task completion, 18, or 51 per cent, have no documented confirmation step before an irreversible action.

This matters because one of the central attractions of agentic AI is its capacity to chain activities together towards a goal without requiring a person to issue a new instruction at every stage. Yet removing those interruptions also removes opportunities for human oversight. Background activity increases that concern. Seventeen products in the Decodo assessment document an ability to operate without an active user session. Seven of these, Airtop, Axiom AI, BrowserOS, Browserbase, HARPA AI, OpenHands and Opera Neon, were assessed as combining background operation with no documented confirmation step before irreversible actions.

This does not establish that these products will necessarily perform a harmful action autonomously. Rather, it identifies where the suppliers’ documentation, as assessed by Decodo, does not describe the type of approval checkpoint included in the study. For businesses, the question should therefore extend beyond whether an agent can execute a workflow to when it should be required to stop and return control to a person.

Another relatively weak area was cross-tab awareness, which averaged 0.66 out of two in Decodo’s scoring. The tools receiving full marks were predominantly browser-native extensions, including Claude for Chrome, ChatGPT Chrome Extension, Chrome Gemini, Sigma Browser, Opera Aria, Opera Neon and Brave Leo. Decodo reports that standalone automation platforms and cloud-browser infrastructure generally scored lower in this category. Cross-tab operation is significant because real-world digital work frequently involves comparing information or maintaining context while moving between sources. Consequently, strong performance within an individual webpage does not necessarily translate into effective handling of more complicated browsing workflows.

Greater autonomy also increases the significance of cybersecurity. Decodo points to industry security research concerning prompt injection attacks against browser agents. Such attacks involve malicious or untrusted content attempting to influence an agent’s behaviour while it is carrying out another task. The report references research associated with Anthropic, OpenAI and Brave in discussing this problem. The risk becomes more consequential as agents gain permission to access private information or perform actions outside the AI system itself. This does not mean autonomous browsers are inherently unsafe; rather, it reinforces the importance of permissions, confirmation mechanisms and human oversight when agents are connected to sensitive information or consequential workflows.

Hence, Decodo’s findings should be interpreted with an important qualification. This is vendor-produced research rather than an independent academic benchmark, and its scoring model places substantial weight on what suppliers publicly document. The hands-on component adds evidence about real-world behaviour, but it should not be taken as demonstrating how every product will perform under every condition. Nonetheless, the research highlights a useful problem for organisations adopting agentic AI: the most sophisticated agent may not necessarily be the most appropriate product for a particular workflow.

Gabriele Vitke, Product Marketing Team Lead at Decodo, argues that businesses should focus on suitability rather than the breadth of a product’s advertised capabilities. “Someone on your team will still have to own what happens when the agent gets it wrong,” Vitke states, before concluding: “Match the tool to the job, not to the longest feature list,” she states in a statement sent to Digital Journal.